Posted 7/25/2023, 2:05:39 PM
Small Devices Could Soon Handle Large Language Models
- Recent technological advancements have made it possible to run AI models on devices, known as edge AI.
- Engineers from Carnegie Mellon University, University of Washington, Shanghai Jiao Tong University, and OctoML have collaborated to run large language models on smartphones, PCs, and browsers.
- If this research can be implemented in everyday use, it could greatly expand the applications of generative AI.
- Currently, most AI models run in the cloud, requiring users to rent servers from providers like Azure, AWS, and Google.
- The ability to run AI models on devices could potentially reduce costs and increase accessibility for users.