AI Tools to Treat Schizophrenia Struggle in Reliability Testing
-
Computer algorithms designed to help treat schizophrenia fail to adapt to new, unseen data sets in a study. These AI tools are central to precision medicine.
-
Researchers found AI models can predict treatment outcomes accurately on data they were trained on. But performance drops to nearly random chance on different data.
-
Models need to be consistently accurate across different cases to be effective. The study shows algorithms must be rigorously tested on multiple samples.
-
Only about 20% of clinical prediction models undergo validation testing on samples other than ones they were developed on.
-
Algorithms should be tested much like drug development, with discipline across building and testing to ensure reliability.