Posted 4/13/2024, 12:22:15 AM
Apple Developing AI to Better Understand iPhone Screens for Accessibility, Testing, and New Features
- Research paper details new Apple AI system, "Ferret-UI," to understand and interact with iPhone interface elements like icons and text
- Ferret-UI could enable richer accessibility features, complex hands-free tasks through Siri, and testing tools for developers
- Code discoveries point to upcoming "Safari Browsing Assistant" and "Encrypted Visual Search" features potentially utilizing AI
- These tie into Ferret-UI capabilities described in research paper, though specifics remain unclear
- Apple playing catch-up to Google's public AI push; iPhone 16 expected to showcase Apple's AI approach at June developer conference