Apple Research Enables More Advanced AI on iPhones While Protecting User Privacy
-
Apple's new research shows an innovative approach to efficiently run large language models (LLMs) on devices with limited memory like iPhones. This could enable more advanced AI capabilities on iPhones.
-
The techniques discussed include windowing, row-column bundling, sparsity exploitation, and memory management strategies to minimize data transfers and overhead.
-
In tests, the proposed approach delivered 4-5x and 20-25x speed improvements on CPU and GPU compared to traditional methods when running models like OPT and Falcon.
-
For iPhone users, this research could mean enhanced on-device AI with better language processing, more sophisticated voice assistants, improved privacy, potentially lower internet bandwidth usage.
-
While promising, experts caution Apple to practice great care and responsibility when incorporating these findings into real-world use cases considering aspects like privacy protection and mitigating misuse.