Google has announced its Gemini AI assistant will be an intrinsic part of the Android operating system from Android 15, bringing a ton of perceived benefits to users.
Breaking Google I/O tradition the company decided against revealing Android 15 in full, but did tell us what’ll underpin the new version of the operating system.
During the keynote, the company explained how a new overlay will enable contextual interaction with key apps on your Android device. Circle to Search is getting better, while it’ll also be better at weeding out potentially dangerous spam calls.
Google says there’s plenty more to come from Gemini within Android, which for all intents and purposes will replace the existing Assistant in time, but here are the key new features we’ll get later this year.
Gemini Overlay
During the Google I/O keynote address on May 14, Google said Android 15 will feature a new Gemini overlay capable of helping you interact with whichever Android app you’re currently using.
So, for example, Google says you’ll be able to “Ask This Video” in YouTube in order to find the specific information you’re looking for without watching the whole thing.
For Gemini Advanced users “Ask This PDF” will be more powerful than the existing text search function, enabling users to pull out exactly what they need from the document.
Google says this is part of the mission to give users dynamic suggestions based on what’s on your display.
Google also says it’ll be possible to drag and drop images generated by Gemini into your correspondence in Gmail and Google Messages. The presenter during Google I/O told Gemini to help it create a pickle playing tennis in answer to a request from a friend to play pickleball.
Google is giving the excellent new Circle to Search feature on Android to harness the information contained in just about anything on your screen – including helping with homework. Effectively, it’ll be fantastic for cheating on maths tests.
“When students circle a prompt they’re stuck on, they’ll get step-by-step instructions to solve a range of physics and math word problems without leaving their digital info sheet or syllabus,” Google says. “Later this year, Circle to Search will be able to help solve even more complex problems involving symbolic formulas, diagrams, graphs and more.”
Better spam detection
Google says this Gemini Nano-powered feature could cut down on fraud by deleting “patterns commonly associated with scams” in real time during phone calls.
So if someone from “the bank” asks you to transfer some money or requests personal information, you’re more likely to be alerted to a “likely scam” and be prompted to end the call.
There’ll also be a helpful message informing you “banks will never ask you to move your money to keep it safe.”
Big TalkBack Boost
Google TalkBack, which helps visually impaired users identify images on their phone, is getting a helping hand thanks to Gemini AI. Later this year, the multimodal capabilities will enable clearer descriptions of what’s happening within the image. This will help, Google says, with shopping online or when receiving a pic sent by family.
