Derived from Llama 2, Code Llama has been optimized to produce computer code.
Researchers and developers can now get Code Llama, Meta‘s most sophisticated large language model, for free. This generative AI model was first revealed in August 2023 and has 70 billion parameters.
Today we’re releasing Code Llama 70B: a new, more performant version of our LLM for code generation — available under the same license as previous Code Llama models.
Download the models ➡️ https://t.co/fa7Su5XWDC
• CodeLlama-70B
• CodeLlama-70B-Python
• CodeLlama-70B-Instruct pic.twitter.com/iZc8fapYEZ— AI at Meta (@AIatMeta) January 29, 2024
“We’re open-sourcing a new and improved Code Llama, including a larger 70B parameter model. Writing and editing code has emerged as one of the most important uses of AI models today. The ability to code has also proven to be essential for AI models to process information in other domains more rigorously and logically. I’m proud of the progress here and look forward to including these advances in Llama 3 and future models as well,” said Meta CEO Mark Zuckerberg in a Facebook post.
Meta states that Code Llama is the most advanced and best-performing model in the Llama family, available for free in three versions: CodeLlama – 70B, the foundational code model, CodeLlama – 70B – Python, 70B specialized for Python, and Code Llama – 70B – Instruct 70B, which a version fine-tuned with human instruction and self-instruction code synthesis.
Code Llama is built on top of Llama and is capable of generating code. According to the company, the model has scored 67.8 on HumanEval, a generative AI benchmark, while the GPT-4 Turbo, a much bigger model, has scored 81.7. Meta also claims that Code Llama is tuned for code generation, and the best part is that it is an open-source model.
The latest generative AI model from Meta can write code in C++, Java, PHP, and Python. The model has been trained using 500 billion tokens of code and related data, and Meta has also expanded the context window, which can now accept inputs with up to 100,000 tokens or around 75,000 words.
It is worth noting that Code Llama is based on Llama, a 175-billion-parameter general-purpose AI model. Users can access Code Llama through platforms like Hugging Face or PyTorch, and the company has also confirmed that some of these advancements will be included in the upcoming Llama 3 and future models by Meta.
