Google’s vision for future AI assistants will become a reality in the next few months via the Gemini Live conversational chatbot interface.
This was a revelation at the conclusion of Tuesday’s Made by Google event in Mountain View, California, where the company also unveiled its new Pixel 9 phones (including the Pixel 9 Pro Fold), Pixel Watch 3 and Pixel Buds Pro 2.
Rick Osterloh, senior vice president of platforms and devices at Google, said its next AI assistant, an AI agent called Project Astra, will provide Gemini Live with a contextual understanding of where we are and what we’re doing through our phones’ cameras.
Although Project Astra sounds like a top-secret NASA mission, it is actually a prototype from Google’s AI research lab DeepMind. It expands the concept of an AI assistant from a mere question answerer to a so-called agent that can take actions on our behalf, such as checking appointments in a calendar or sending a message to a friend – all with our permission, of course.
The idea is that once we have AI agents, we won’t need to open other apps – we can just talk to Project Astra (or a similar agent) while it fetches the necessary information from elsewhere on our devices. This is a big opportunity for Google and its competitors as AI and search converge and the way we access information changes. And while Google may win the award for most futuristic sci-fi name, consumer loyalty to an AI agent is still uncertain.
Project Astra + Gemini Live
The upcoming integration has a small catch: Gemini Live, and thus Project Astra, are only available to Gemini Advanced subscribers, who pay $20 per month for access to Google’s latest AI model, Gemini 1.5 Pro.
If you’re in this group, you’ll soon be able to share your camera while talking to Gemini to ask questions about what’s in front of you, whether it’s a math problem you don’t know the solution to or furniture you’re having trouble assembling.
Gemini Live can also pull information from apps like Google Calendar and Gmail to answer your questions and share information without leaving the Gemini Live interface, Osterloh said.
We’ve seen similar features from AI startup OpenAI. In its Spring Update in May, OpenAI introduced conversational interactions with its chatbot ChatGPT, as well as the ability to share photos, videos, and documents to support those conversations.
The voice feature, known as Advanced Voice Mode, went live to a small group of testers earlier this month.
Both Project Astra and Gemini Live were unveiled at the Google I/O developer event, which also took place in May.
“We’re evolving Gemini to make it even more actionable and tackle complex problems with advanced reasoning, planning and memory, so you can think several steps ahead and Gemini will do things for you under your supervision,” Osterloh said at the conclusion of Made By Google. “That’s the promise of a true AI assistant.”
