How to use the 'Look and Ask' AI feature to identify objects

Overview

Look and Ask is one of the most powerful features of the Ray-Ban Meta AI glasses for users who are blind or have low vision. When you prompt Meta AI, it will describe what is in front of you, read out signs or even identify people. To ensure the feature is working correctly there are some settings you will need to turn on and important prompts to remember, which we’ll go through here.

What does the ‘Look and Ask’ feature do?

With the Look and Ask feature on your new AI glasses, you’ll be able to take advantage of Meta AI and the inbuilt cameras to learn about your surroundings, read menus or signs and more.

For example, if you’re at a restaurant, you can pick up the menu and ask Meta to read out the menu in your hands. It will then scan the text using the cameras and read it back to you through the inbuild speakers.

If you’re in a new location, you can ask the AI to describe your surroundings. It will then use the cameras to see what you’re looking at and describe it back to you.

You can save people’s face and name into Meta AI’s long-term memory. So, when you ask “Who is in front of me” Meta AI will be able to recall that memory and let you know.

How to enable detailed descriptions

A key part of getting the most out of this feature is to enable detailed descriptions. This is a feature that will allow to AI to provide longer, more descriptive responses when you ask it to do things like describe your surroundings. By default this feature is turned off.

To turn detailed descriptions on you should do the following:

1. Open up the Meta AI app on your phone.
2. Head to Device settings.
3. Then tap on Accessibility.
4. Within the accessibility settings there will be a toggle for detailed descriptions. Ensure this is switched on.

The prompts you need to know

To ensure you get the results you want out of the Look and Ask feature there are some specific prompts you should know. Each will prompt the Meta AI to provide information in a specific way that may be best for your scenario. Here’s a breakdown of these prompts:

To get Meta AI to describe something:

• “Hey Meta, look and tell me what you see”: Provides identification of objects or people in view.
• “Hey Meta, look and give me a detailed description”: Offers a thorough analysis of visible surroundings.
• “Hey Meta, look and tell me everything you see”: Gives a comprehensive overview of the environment.

To get Meta AI to read something:

• “Hey Meta, look and read this”: Reads aloud visible text such as signs, menus, or documents.
• “Hey Meta, translate this”: Translates text into your preferred language.
• “Hey Meta, look, and give me a summary of the document I’m holding”: Provides a general idea of the printed item before you.
• “Hey Meta, look, and give me a detailed description of anything on this envelope.”
• “Hey Meta, look and summarize, and describe in detail the items on this menu.”
• “Hey Meta, look and give me detailed pricing information on this menu for the beverages, appetizers, main courses, and desserts.”
• “Hey Meta, look and give me a detailed summary of the main courses on this menu with pricing for each item.”

To get Meta AI to remember someone:

• “Hey Meta, remember this person as my (mother, father, brother, sister, cousin, aunt, uncle, grandmother or grandfather).”
• “Hey Meta, remember this person as (my friend or neighbour, etc.) and store this in your database for future reference.”

Need support?