Meta Is Already Training a More Powerful Successor to Llama 3

Zuckerberg took to Instagram today to explain that Meta would incorporate the new Meta AI assistant, powered by Llama 3, into products that include Whatsapp, Instagram, Facebook, and Messenger.

Meta said in its blog post announcing Llama 3 that it had focused heavily on improving the training data used to develop the model. It was fed seven times as much data as its predecessor, Llama 2, the company said. Some AI experts noted that figures released by Meta also showed that creating Llama 3 required huge amounts of energy to power the servers required.

The growing capabilities of open source AI models have spurred some experts to worry that they could make it easier to develop cyber, chemical, or biological weapons—or even become hostile toward humans. Meta has released tools that it says can help ensure Llama does not output potentially harmful utterances.

Others in the field of AI say that Meta’s Llama models are not as open as they could be. The company’s open source license on the models places some restrictions on what researchers and developers can build.

“It’s great to see more and more models openly releasing their weights,” said Luca Soldaini, senior applied research scientist at Allen Institute for AI, a nonprofit lab, n a statement after Llama 3’s release. “But the open community needs access to all other parts of the AI pipeline—its data, training, logs, code, and evaluations. This is what will ultimately accelerate our collective understanding of these models.”

Stella Biderman, an AI researcher involved with EleutherAI, a nonprofit open source AI project, says Meta’s license for Llama 2 limited the experiments that AI researchers can run with it, and adds that the Llama 3 license looks even more restrictive. “Meta releases weights but is famously restrictive about what you can do with them,” Biderman says.

One part of the model’s license says that companies with “greater than 700 million monthly active users” must seek a special license from Meta—a clause apparently designed to prevent the project from helping the company’s closest rivals.

Even so, Llama 3 seems likely to spark a new burst of AI experimentation. Clement Delange, CEO of HuggingFace, a repository for open AI models, including Llama 3, says developers created more than 30,000 variants of Llama 2. “I’m sure we’ll see a flurry of new models based on Llama 3 as well,” he says. “Awesome community move by Meta.”