AI Student Monitoring Software Raises Accuracy, Privacy, and Equity Concerns
-
AI-based monitoring software tracks students' online activity to try to identify suicide risk, but lacks evidence supporting its accuracy and poses privacy concerns.
-
Monitoring could exacerbate inequalities if it disproportionately flags marginalized students' activity or fuels existing disparities in school discipline.
-
Schools often direct alerts to law enforcement, risking violent interactions, instead of counselors who could provide support.
-
The software’s actual benefits and harms are unknown due to a lack of follow-up on flagged students and school responses.
-
More regulation is needed to protect students as the software could do more harm than good without appropriate safeguards.