Meta AGI
Implications span much farther than AI
It kind of seemed odd that Mark Zuckerberg would announce one of the boldest goals of humanity in a crammed room with low lighting and poor camera quality
Meta’s goal? To create Artificial General Intelligence (AGI), a term thrown around by everyone because “ultra intelligence” sounds too sci-fi to be taken seriously. In fact, AGI is quite often not well defined, even Mark’s The Verge interview points out that Mark himself does not know the definition.
“I don’t have a one-sentence, pithy definition,” he tells me. “You can quibble about if general intelligence is akin to human level intelligence, or is it like human-plus, or is it some far-future super intelligence. But to me, the important part is actually the breadth of it, which is that intelligence has all these different capabilities where you have to be able to reason and have intuition.”
Way to set success criteria Mark! All kidding aside, the real news behind the news here is that Meta is perhaps able to catch up to OpenAI with creating the most advanced model humanity has ever seen. By purchasing roughly 350,000 H100s (Nvidia’s super GPU), Meta is investing somewhere between $7-10B on GPUs alone. And it may surprise you that their existing compute was already 76% of that capacity. Mark said they will have roughly 600,000 H100 equivalent compute at their disposal when combined with their older GPUs. This number is quite staggering since Meta will be able to train a GPT4 sized model within a few days, whereas it took OpenAI with A100s likely many months to roll out their first version. As the time compression of advanced models narrows, even within a few months period we could see rapid iterations of extremely advanced models within this year, certainly by 2025.
AGI is not really a bold claim by any means, there will be some LLM’s rolled out this year that will be able to be indistinguishable from humans, even with vision and language recognition and response, with very low latency. This is not sci-fi anymore, it’s roadmap, but what people might not be thinking about are the implications of such technology being created from Meta.
There are two main things to really think about here, Meta’s role in the open source community, and Meta’s current business model. By creating advanced models rapidly, Meta could potentially disrupt both the AI industry and what we call an amalgamation of social media, entertainment, and really just user engagement.
Open Source models
With open source, Llama has already caused a massive surge of research and productivity around LLMs. By having a commercially viable and very good model released to the public, there has been a new edge against what OpenAI and other generalized LLM companies can compete against. When OpenAI releases GPT5, Meta will release Llama 3.
But wait, that’s the obvious part. The unobvious part will happen to be the army of startups that have purchased tons of H100s for training LLMs. Some will be buying them for renting out the compute, and the end customers will be startups trying to train their own models. If Meta has the ultimate cluster and releases their trained models for free use, what is the point for other clusters to be made? If some supercomputers will end up training inferior models, what happens to the business of companies that loan GPU compute, the businesses that train new models, or even Nvidia in 2025? The same argument could be made for data accessibility, at some point, all the data that is publicly available will be trained on, and the battle of the best LLMs could come from having unique datasets that have LLM’s with very specific strengths. Meta in particular has access to very “human” data and posts, which could behave nearly indistinguishable from even your best friend. After all, they have all their posts already.
However, Meta can’t boil the ocean, and they certainly don’t have every dataset. Startups with use case specific LLMs could survive since they could focus their product teams to really understand a specific customer group to provide more value. Obviously, even having these models does not solve the next part, which is scaling LLMs to thousands (and eventually millions) or users. I think that will be my next blog post…but back to business.
Meta’s Business model
With Meta’s business model, the obvious goal is having AI be able to create content that can stretch a user’s attention span longer and longer. AI could rapidly iterate and improve its own performance, or tailor itself for a specific persona. It would likely be many orders of magnitude better at this than a human can, because it could create extremely rapid tests of what works and what doesn’t and propagate this type of data to other AI content. By the way, Meta already does this with existing user posts. Now this AI content can take many forms, you could have an AI influencer that’s just perfect in every way, even having an AI influencer that can communicate and respond back to followers. You could have AI photos or videos (or even memes) that quickly grab user attention while putting ads in between. Basically anything the AI can learn to get more followers and views will happen very quickly.
For a human, it can take a lot of effort and time between posts, especially to iterate and get things right for followers. For an AI, it’s rapid testing and iterations within seconds across millions of users, get ready for Meta’s AI content to grab your attention over human content. I could see something like this happening, by end of this year, AI content will probably take a pretty good percentage of images and videos:
However, by end of 2025, the amount of AI content generated could be so easy that it would not only become a larger percentage of content, but it may start discouraging human content as well:
While ChatGPT and Midjourney are already doing this with a lot of online content, Meta’s business model benefits immensely from creating AGI and applying it to their current products to optimize revenue. At some point, it won’t even need humans to post content anymore.