OpenLLaMA is a fully open-source LLM, now ready for business

New LLM Connections are open for business, have been trying for a while, though are still a bit choppy.

OpenLLaMA is a fully open-source LLM, now ready for business

OpenLLaMA is an open-source reproduction of Meta’s LLaMA language model and can be used commercially.

Since the unveiling of Meta’s LLaMA family of large language models and the subsequent leak, the development of open-source chatbots has exploded. Models such as Alpaca, Vicuna, and OpenAssistant use Meta’s models as the basis for their various forms of instruction tuning.

However, LLaMA models are licensed for research use only, which prevents commercial use of those models.

OpenLLaMA reproduces Meta’s language models

Alternatives based on other freely available models do not match the quality of Meta’s models, as LLaMA follows Deepmind’s Chinchilla scaling laws and has been trained on particularly large amounts of data.

THE DECODER Newsletter

Researchers at Berkeley AI Research want to replicate Meta’s LLaMA models in the OpenLLaMA project. The team is using Together’s RedPajama dataset for the project. The open-source platform also announced its intention to reproduce the LLaMA models in April, releasing the 1.2 trillion parameter dataset as a first step.

The Berkeley team is now releasing an early version of the 7-billion-parameter OpenLLaMA model, which has so far been trained on 300 billion of 1.2 trillion tokens. Performance is already said to be approaching the level of LLaMA, and the team is confident that the fully trained OpenLLaMA will be competitive with Meta’s original .... '