The Grandmother of AI embraces World Labs to unleash a Spatial Computing Renaissance

Hey Everyone,

Everyone wants to be a world builder in the era of Generative AI.

May I just say what the video games World Labs. I've been waiting for an AI startup that could impact the future of gaming at scale, and I might just have found it.

World Labs was founded by visionary AI pioneer Fei-Fei Li along with Justin Johnson, Christoph Lassner, and Ben Mildenhall; each a world-renowned technologist in computer vision and graphics.

Fei-Fei Li, the Stanford professor many deem the “Godmother of AI,” has raised $230 million for her new startup, World Labs, from backers including Andreessen Horowitz, NEA, and Radical Ventures.

It's a fairly serious amount of funding as well, read a16z's note here. World Labs is valued at over $1 billion, and the capital was raised over two rounds spaced a couple of months apart.

The first part of the funding came in July, 2024. The new venture on 3D spaces counts venture firm Andreessen Horowitz, as well as actor Ashton Kutcher and Google’s Eric Schmidt as investors. They will build models aimed at improving our spatial computing and the way we think of 3D models and digital environments.

The Advent of Large World Models

World Labs is developing what it calls "large world models" (LWMs), which go beyond the current capabilities of language and image generation models. These LWMs aim to create and manipulate 3D virtual spaces with realistic physics and semantics. The goal is to enable computers to perceive and reason about the physical world in three dimensions, much like humans do.

TechCrunch sounds a bit skeptical and why not. World Labs, which was reportedly founded in April and went from founding to unicorn in four months, suggests that investors continue to place large bets on AI startups founded by prominent AI scientists, even if the startups businesses are unproven. 

It will be a fascinating AI startup to watch. It's all very vague but I think the end-game is actually video gaming and the impact of Generative AI at least in the commercial sense. Think about it, the company aims to build software that can use images and other data to make decisions about the three-dimensional world, building what it calls “large world models”.

The startup is also aptly named compared to the book Fei Fei released in 2023:

See the book on Amazon.com.

The range of the investors is also pretty wild, wide and far-reaching. Corporate venture capital arms like Adobe Ventures, AMD Ventures, Databricks Ventures, NVentures of NVIDIA, and Shinrai Investments LLC have also participated. This is actually the sort of thing Nvidia thinks a lot about with its partners.

Another way to consider what they are trying to do is the following:  World Labs is seeking to create AI models that can accurately estimate the three-dimensional physicality of real-world objects and environments, enabling detailed digital replicas without the need for extensive data collection. Got that? 😕

They are calling themselves a spatial intelligence company building Large World Models (LWMs) to perceive, generate, and interact with the 3D world.

Language models changed how we interact with computers because they enabled software to speak and understand natural languages. What might be next? Li, widely known as a “Godmother of AI,” discussed how machines can be trained to develop human-like “spatial intelligence” in a TED talk earlier this year.

What the video games, at the end of the day the best products for 3D LLMs will be the video gaming and perhaps the Metaverse, if that ever decides to actually become a thing. Consumers remain skeptical about VR & AR worlds and products.

“We aim to lift AI models from the 2D plane of pixels to full 3D worlds — both virtual and real — endowing them with spatial intelligence as rich as our own,” the company said in a Friday post on LinkedIn.

Fei-Fei Li wants to Build the Matrix

Indeed her LinkedIn posts are maybe some of the best insights into what this startup is trying to do. The team plans to have their first product ready by 2025, initially focusing on generating virtual 3D spaces where users can control elements like physics and object interactions.

Will this be a text-to-pixel revolution? Alongside language, there has been a “pixel” revolution in generative AI, allowing us to create images and videos from text descriptions of what we want to see. Unlike language models, these generated images are reflections of the physical world. What digital twins with capabilities to simulate and obtain probabilities of future events might this lead to?

Whatever it is, it's a big bet. World Labs has attracted substantial investment from leading venture capital firms. Fei-Fei has a lot of powerful friends in the AI world as well. Individual investors comprise a who’s who of the tech industry: Marc Benioff (Salesforce CEO), Reid Hoffman (LinkedIn co-founder), Eric Schmidt (former Google CEO), Jeff Dean (Google DeepMind chief scientist), Geoffrey Hinton (machine learning pioneer), and actor-turned-venture capitalist Ashton Kutcher.

Space, the final frontier.

The AI startup aims to lift AI models from the 2D plane of pixels to full 3D worlds – both virtual and real – endowing them with spatial intelligence as rich as our own. Human spatial intelligence evolved over millennia; but in this time of extraordinary progress, we see the opportunity to imbue AI with this ability in the near term.

AI is an emerging field with the potential to transform our world. What role will spatial computing and the future of 3D models and Generative AI play? This immediately sounds like a startup either Meta or Nvidia might want to acquire if they have difficulty monetizing or are actually a success.