Key insights

• ⚙️ Launching first AI agent called Operator for completing tasks using a web browser
• 🌎 Available to pro users in the US first, with plans to expand to other countries and user types
• ⚠️ AI may make mistakes but can be valuable for interacting with various brands
• 🎉 Demo showed successful booking of a table on Open Table
• 🍽️ Using Operator for making dining reservations and grocery shopping
• 📝 Custom instructions and delegation in Operator
• 👁️ Vision capabilities for understanding images and preferences
• 💻 Operator aims to remove the need for specialized APIs by teaching a model to use the basic interface of a computer

Q&A

• Is Operator perfect?

  No, Operator is a research preview and may make mistakes. However, benchmarks show promising outcomes for navigating operating systems and websites. It is being gradually rolled out to Pro users in the US with plans for API integration in the coming weeks.

• How does Operator ensure safety in task execution?

  The discussion includes safety measures to avoid misalignments in task execution and the human in the loop interaction mode. It involves confirmation as a mitigation for misalignment in user, agent, and website, as well as prompt injection monitor as a layer to observe and pause suspicious instructions, and an iterative approach to deployment for learning and improving safety measures.

• What is takeover mode in Operator?

  Takeover mode provides complete privacy for the user, and the operator cannot view the user's activities. The user can conduct multiple tasks simultaneously using a variety of apps and websites with the help of the remote browser, allowing for independent browsing and multitasking.

• What is the goal of Operator?

  The goal of Operator is to remove the need for specialized APIs by teaching a model to use the basic interface of a computer, enabling it to navigate and act in the digital world. It uses screenshots and visual information to make decisions and perform tasks, creating sub-plans and interacting with the interface to complete tasks. Users can take control at any point, similar to passing a laptop back and forth.

• What capabilities does Operator have?

  Operator can be used for making dining reservations and grocery shopping, featuring custom instructions, delegation, confirmations, and vision capabilities. It also introduces the research model Kua behind Operator.

• What is Operator?

  Operator is the first AI agent that uses a web browser to complete tasks. It's an early research preview designed to be available to pro users in the US first. The AI system may have some mistakes but can be valuable in interacting with various brands. A demo showcased the AI successfully booking a table on Open Table.

• 00:07 We're launching our first AI agent called Operator, which can use a web browser to complete tasks. It's an early research preview and will be available to pro users in the US first. The AI system may have some mistakes but can be valuable in interacting with various brands. A demo showed the AI successfully booking a table on Open Table.
• 03:30 A demonstration of using Operator to make dining reservations and grocery shopping, showcasing its custom instructions, delegation, confirmations, and vision capabilities. It also introduces the research model Kua behind Operator.
• 07:09 Operator is a research project that aims to remove the need for specialized APIs by teaching a model to use the basic interface of a computer, enabling it to navigate and act in the digital world. The model uses screenshots and makes decisions based on the visual information it receives, creating sub-plans and interacting with the interface to perform tasks. Users can take control at any point, similar to passing a laptop back and forth.
• 10:50 During takeover mode, the operator cannot see the user's activity, making it a completely private session. The user can perform multiple tasks simultaneously, such as making purchases and finding services, using various apps and websites. The remote browser allows for independent browsing and multitasking.
• 15:04 A discussion about using a model as an agent to perform tasks, focusing on safety measures to avoid misalignments in task execution and the human in the loop interaction mode.
• 19:18 The Operator feature is a research preview and not perfect, but benchmarks show promising outcomes. It can navigate operating systems and websites, and is being gradually rolled out to Pro users in the US with plans for API integration.