Apple's ReALM sees your screen without looking at images
Apple researchers just unveiled ReALM, a lightweight AI model that decodes your screen context by converting visual layouts directly into text. Despite having far fewer parameters than GPT-4, it substantially outperforms OpenAI's flagship on screen-specific tasks—offering a blueprint for a lightning-fast, on-device Siri that can truly take action on what you are looking at.