Meta’s Next Llama AI Models Are Trained on the Bigger Than Anything GPU Cluster More

Managing such a gigantic array of chips for Llama 4 development will likely present unique engineering challenges and require enormous amounts of power. On Wednesday, Meta executives dodged questions from analysts about limiting access to energy in parts of the US that have hindered companies’ efforts to develop more powerful AI.

According to one scorea cluster of 100,000 H100 chips requires 150 megawatts of power. The largest national laboratory supercomputer in the United States, El Capitanon the contrary, it requires 30 megawatts of power. Meta expects to spend $40 billion of capital on data centers and other infrastructure this year, up 42 percent from 2023. Next year, the company expects an even greater increase in these costs.

Meta’s total operating expenses are up about 9 percent this year. But overall sales — driven largely by advertising — rose more than 22 percent, leaving the company with higher margins and higher profits even as it pours billions of dollars into Llama’s efforts.

Meanwhile, OpenAI, considered the current leader in advanced AI development, is spending money despite charging developers to access its models. What remains a non-commercial enterprise for now stated that it is training GPT-5, the successor to the model currently running ChatGPT. OpenAI said that GPT-5 will be larger than its predecessor, but did not say anything about the computer cluster it uses for training. OpenAI also said that in addition to scaling, GPT-5 will include other innovations, including a newly developed one approach to reasoning.

CEO Sam Altman said that GPT-5 will be a “significant step forward” over its predecessor. Last week, Altman responded to news that OpenAI’s next advanced model would be released by December in writing on X, “fake news is out of control.”

On Tuesday, Google CEO Sundar Pichai said that the company’s latest version The Gemini family of generative AI models is under development.

Meta’s open approach to artificial intelligence has proven controversial at times. Some AI experts worry that free access to more powerful AI models could be dangerous because it could help criminals carry out cyberattacks or automate the development of chemical or biological weapons. Although Llama was configured before release to limit misbehavior, removing these restrictions is relatively trivial.

Zuckerberg remains positive about the open source strategy, even as Google and OpenAI push proprietary systems. “It seems pretty clear to me that open source is going to be the most cost-effective, customizable, reliable, performant and easy-to-use option available to developers,” he said Wednesday. “And I’m proud that Lama is a leader in that.”

Zuckerberg added that the Llama 4’s new capabilities should be powered a wider range of functions in Meta services. Today, Llama’s signature model-based offering is a ChatGPT-like chatbot known as Meta AI, which is available on Facebook, Instagram, WhatsApp and other apps.

According to Zuckerberg, more than 500 million people use Meta AI every month. Over time, Meta expects to generate revenue through advertising in this feature. “The set of requests that people will use it for will expand, and monetization opportunities will come over time,” Meta CFO Susan Lee said on a call Wednesday. With the potential for ad revenue, Meta would just be able to subsidize Llama for everyone else.