AI Models Predict Cancer Patients' Mental Health Needs From Doctor Notes
-
Used natural language processing (NLP) models on 47,625 cancer patients' initial oncology consult notes to predict seeing a psychiatrist or counsellor in the next year. Best models achieved balanced accuracy over 70% and AUC over 0.75.
-
Convolutional neural network and long short-term memory models outperformed simpler bag-of-words models, suggesting complex language understanding helped.
-
Model interpretations showed use of relevant features like symptoms, mental health history, cancer history. Also found some differences in factors predicting psychiatry vs counselling referrals.
-
Developed a new NLP model interpretation technique using integrated gradients and BERTopic to understand model predictions over multiple documents.
-
Future work needed for external validation and investigating implementation barriers, but technique could help identify patient psychosocial needs and improve equity in cancer care.