1. Introduction
Text-based mental health support services increasingly rely on triage systems to route the highest-risk conversations to trained counsellors quickly, and automated risk detection can support this without replacing human judgement for escalated cases.
2. Methodology
A DistilBERT encoder was fine-tuned on 22,000 de-identified support-chat transcripts labelled by trained annotators into five escalation levels, from routine to imminent-risk. The resulting classifier was integrated into a response pipeline in which only low-risk turns were eligible for automated templated responses, with all higher-risk turns immediately flagged for human handoff.
3. Results
The fine-tuned classifier achieved a macro F1-score of 0.88 across the five risk levels, with recall for the two highest-risk categories at 0.95, prioritising sensitivity for escalation over precision. In a blinded review of 300 low-risk automated responses, two independent counsellors rated 91 percent as clinically appropriate.
4. Conclusion
Fine-tuned transformer classifiers can support responsible triage in mental health chat platforms when paired with conservative human-handoff thresholds. Future work will evaluate the system in a live, IRB-approved pilot deployment.
References
[1] Sanh V. et al., DistilBERT, a distilled version of BERT, NeurIPS Workshop, 2019. [2] Milne D. N. et al., Detecting suicidality in online communities, ACL Workshop, 2016.