AI still not great at generating clean code in API study

Large Language Models (LLMs) like GPT-3.5 and GPT-4 have been found to have high rates of API misuse when answering Java coding questions from StackOverflow, while the open model Llama 2 exhibited a failure rate of less than one percent due to its lack of code suggestions.

theregister.com

Relevant topic timeline:

8/3/2023

Developers Ask GPT-4 What it Thinks About Google’s AI

- Startups and developers are questioning the trustworthiness of large-language models (LLMs) like OpenAI's GPT-4. - Recent research suggests that while LLMs can improve over time, they can also deteriorate. - Evaluating the performance of LLMs is challenging due to limited information from providers about their training and development processes. - Some customers are adopting a unique strategy of using other LLMs to assess the reliability of the models they are using. - Researchers at companies like OpenAI are becoming less forthcoming at industry forums, making it harder for startups to gain insights.

8/24/2023

Introducing Code Llama, an AI Tool for Coding

Code Llama, a language model specialized in code generation and discussion, has been released to improve the efficiency and accessibility of coding tasks, serving as a productivity and educational tool for developers. With three variations of the model available, it supports various programming languages and can be used for code completion and debugging. The open-source nature of Code Llama encourages innovation, safety, and community collaboration in the development of AI technologies for coding.

8/25/2023

10X coders beware: Meta’s new AI model boosts coding and debugging for free

Meta has introduced Code Llama, a large language model (LLM) designed to generate and debug code, making software development more efficient and accessible in various programming languages. It can handle up to 100,000 tokens of context and comes in different parameter sizes, offering trade-offs between speed and performance.

Topics

Posted 8/29/2023, 7:02:00 PM