Meta's smart glasses can now see what you see

Meta is quietly rolling out the multimodal AI features for its second-generation Ray-Ban smart glasses way ahead of schedule. While Mark Zuckerberg originally hinted these visual smarts wouldn't land until 2024, an early-access beta is already reaching test users alongside a new Bing-powered voice search update.

The premise is simple: the glasses can now see what you are looking at and answer questions about it in real time.

Look and tell me

The feature turns Meta's $300 shades into a face-mounted, voice-activated visual assistant. To use it, you speak a specific prompt starting with "Hey, Meta, look and..."

You can point your gaze at ingredients and ask for a recipe, glance at a houseplant to find out what it is, or look at a menu and ask for spicy dishes. Once you speak the command, you hear an audible camera shutter click in your ears. The glasses snap a still photo, pause for a few seconds while generative AI digests the image, and then read the answer back to you through the open-ear speakers.

Everything you ask, along with the snapshot the glasses took, is automatically saved inside the companion Meta View phone app like a digital visual notebook.

For now, it works on photos rather than continuous live video. Meta CTO Andrew Bosworth noted that this voice-activated snapshot method is necessary to avoid destroying battery life. The long-term dream, according to Bosworth, is developing sensors low-power enough to automatically trigger the AI when relevant events happen.

Hallucinations and awkward pauses

Because this is an early beta, the quirks are front and center.

First, there is the latency. Having to say "Hey, Meta, look and" followed by a multi-second pause before hearing an answer feels robotic. You also cannot escape the typical quirks of generative AI.

In testing, the glasses looked at a fruit bowl and successfully spotted oranges, bananas, dragonfruit, and apples, but hallucinated pomegranates that were never there. Asked to write a funny caption for a giant stuffed panda sitting by a window, it generated a line about someone feeling lonely and staring at their phone.

Language tasks are also hit-or-miss. The glasses successfully parsed a Spanish menu to highlight spicy options, but when asked about meat dishes, they simply recited the raw Spanish text back aloud.

Meta also confirmed that anonymized query data is being collected during this beta to train its models, which will inevitably raise privacy questions for onlookers and users alike.

Why it matters

We think Meta is playing this wearable AI race much smarter than the rest of the hardware field. While startups like Humane are trying to sell expensive, novel lapel pins that demand radical lifestyle changes, Meta piggybacked on an existing, stylish $300 accessory that millions of people already wear.

Is it worth buying right now solely for the AI features? Probably not. The shutter delay, rigid verbal triggers, and occasional hallucinations mean it is still very much an experimental lab demo on your face.

Yet the implications are huge. For years, the tech industry assumed smart glasses were useless without bulky heads-up displays or full augmented reality waveguides. Meta is proving that a simple camera, an earphone, and a fast multimodal model might be all the hardware wearable AI actually needs to be useful in daily life.