Meta’s new world model lets robots manipulate objects in environments they’ve never encountered before

Although the text of large language models (and other points) have mastered the text (and some to some degree), it should not be a physical “mind” to operate in real environments. This restricted the placement of AI in areas such as production and logistics, where the cause and effects of understanding are limited.

The last model of methane, V-Jepa 2Studying a world model from video and physical interactions, this gap is one step to close the bridge.

V-Jepa 2 can help create AI apps that require predictability of results and planning actions in unexpected environments with many outsiders. This approach can give you an open way in the direction of advanced automation in more skilled robots and physical environments.

A ‘World Model’ learns how to plan

People observe their surroundings and develop early physical intuition in life. If you throw a ball, you can recognize the trajectory as instinct and where it will be located. V-Jepa 2 studies a similar “world model” with the internal simulation of an AI system how the AI ​​system works in the physical world.

The model is based on three main opportunities for enterprise applications: to understand what happened in a stage, plan the sequence of measures to change the stage and achieve a certain goal. As in a position of meta macal“Long-term vision will allow world models to show plan and cause in the physical world of EU agents.”

The architecture of the model of the joint embedding forecasting the video jointly (V-Jepa) consists of two main parts. An “encoder” follows the video clip and summarizes a compact number known as one placement. These accommodation seize basic information about objects and their relationships. A second component, “Predictor”, then takes this summary and predicts how the scene will develop, how the next summary will appear.

This architecture is the most recent evolution of JEPA frame, initially applied to images I-dipp Now, a consistent approach to the construction of world models is moving forward.

Unlike the generative AI models that are trying to predict the exact color of each pixel in the future – the calculation is an intensive task – V-Jepa operates in 2 abstract space. It is more effective than the position and trajectory of an object, not only the forecast of one’s high-level characteristics, texture or background details, but more effective than only 1.2 billion paramels

This turns the reports that is more suitable for reducing report accounts and place in real world parameters.

To learn from observation and action

V-Jepa 2 studies in two stages. First, it builds a basic concept with physics Self-controlled learningwatching more than a million hours of mobile videos. Simply observing how objects are moving and interrelated, it develops a common world model without any human leadership.

In the second stage, this pre-trained model is beautiful in a small, specialized database. Looking at only 62 hours of video showing the robot’s robot’s work, along with appropriate control commands, V-Jepa 2 learns to connect specific actions to physical consequences. This results in a model that can plan and control the actions in the real world.

These two-stage training allows a critical ability to automate the real world: zero stroke robot planning. A robot supplied by V-Jepa 2 can be placed in a new environment and can successfully manage objects that have never encountered before retreating for this particular setting.

This is an important progress on previous models that require training information exact the robot and environment they will work. The model was trained in the open source database, and then was successfully placed on different robots in the laboratories of meter.

For example, to complete an object, the robot is given the goal image of the desired result to complete an assignment. Later, using V-Jepa 2 predictors, the next number of possible actions are the internal simulation. After every imagined movement achieves the goal, he fulfills the highest rated movement and repeats the process until the task is completed.

Using this method, the model reached the rates between 65% and 80% in the selection and location assignments with unfamiliar objects in new settings.

The real world of physical justification

The ability to plan and move in novels in situations, directly affects the operations. In logistics and production, it allows more adaptable robots that can manage changes in products and storage plans without extensive programming. This is especially useful because they are studying the placement of companies Humanoid Robots in factories and mounting lines.

The same world model can enhance high real digital twins, allows companies to imitate new processes or prepare another AIS in a physically accurate virtual environment. Based on the video broadcasts of techniques in industrial parameters, based on the notion of video broadcasts of technology and the learned concept of physics, security problems and failures can be followed before starting.

This study is the main step for planning to learn how the “AI Systems” called “Developed Machine Invisible”, “AI Systems”, and how to perform their unfamiliar tasks and the constant world-changing world. ”

The meta model and its training code and its training code and the study hopes to build a wide society around the study, hoping to move towards our last goal to develop the EI’s physical world. ”

What does the enterprise mean for technical decision makers

V-Jepa 2, Robototics approaches the model identified by the program recognized by the recognized cloud teams: once before the train, place it everywhere. The model learns the general physics to public video and only a few ten hours of employee is needed, institutions, usually pilot projects can usually score the collection of data. From a practical point of view, you can prototype a choice and place robot in a suitable desktop and then rub the same policy on a factory floor without collecting thousands of fresh samples or special motion scripts.

The heading of low education also changes the cost equation. With 1.2 billion parameters, a high-level GPU of V-Jepa 2 is comfortable and reduces abstract forecast targets. This allows the teams to control the cloud delay, the cloud delay and the compatibility from the flowing video outside the plant, to manage the head and beyond the headache. Once the budget going to mass-paying groups can finance additional sensors, reserve or faster iteration periods.