Study Finds Neural Networks Follow Surprisingly Similar Paths When Learning to Classify Images
-
Neural networks follow a surprisingly uniform learning path, identifying the same low-dimensional image features like ears and eyes when classifying cats vs dogs.
-
This common learning pathway hints at the possibility of cheaper, faster AI training algorithms needing fewer computational resources.
-
The study used information geometry tools to compare different networks on an equal footing, revealing their similarities.
-
The learning path is like a 3-5 dimensional manifold in probability space, despite neural networks' high complexity.
-
Possible explanations are that classifying real-world images is easier than expected and that human category labels are themselves low-dimensional.