Android’s built-in AI, Smart Text Selection, will know what you need to copy and paste in

One of the new features in Android O that was recently announced at I/O is Smart Text Selection. Selecting the right text can be hard on a small screen, but Google’s data shows that most of the time when we are selecting information it’s something like an address or phone number. Now in Android O you can double tap text, and the selection automatically expands as needed. It even shows a convenient predictive shortcut so we can use it in another context, or save it. Of course, we can use the Assistant to do all those things if we remember, but it’s good to have multiple ways of solving the same problem.

Google previewed a way of using AI to copy and paste without actually having to copy and paste on a mobile device. The feature was teased as a way Android will use context to take what you’ve been researching in Chrome and apply it to other apps. Well, there’s a bit more to it than that.

First up is Smart Text Selection, which eliminates the need to long-press a bit of text; Now when you double tap on a relevant bit of text or the stuff you want, Google’s machine learning software is able to figure out where the relevant information starts and ends, and automatically selects all if it, rather than just the tiny part you tapped. Specifically, It works for physical addresses, email addresses, names, phone numbers, and even suggests contextually relevant means of interacting with the data right next to the other options that appear like Cut and Format. From there, Android will suggest calling with the phone app or getting directions to the business via Maps based on the bit of selected text. It happens in real-time without leaving the device — just like the code release from last month suggested.

Tensor Flow Lite, a version of Google’s open-source machine learning tools, enables LSTMs and other AI bits on your phone. It’s a neural network API that uses hardware acceleration to tap into “silicon-specific accelerators.” Autofill from Chrome is coming to Android too. It uses your autofill data from your browser, and in the example shown onstage, it was used to log into the Twitter app.

In conclusion, all of the information is processed on-device, so the privacy conscious need not worry too much, and from a technical level it’s very cool that the machine learning software is snappy enough to do it all locally. All the tiny bits of precious time this can save will add up, even if you forget to use the Assistant. This and the other neural/”fluid” experiences in Android O will be available later this year.