ChatGPT on Mac can now read your coding apps

Most developers know the muscle memory by heart: run into a bug, highlight a block of code, switch windows, paste it into ChatGPT, and ask what went wrong.

OpenAI moved to eliminate that chore. The company introduced a new feature called "Work with Apps" for the macOS ChatGPT desktop app, allowing the chatbot to look directly inside your coding tools.

At launch, it supports a handful of developer staples, including VS Code, Xcode, TextEdit, Terminal, and iTerm2. Once enabled, the chatbot automatically pulls the code you are currently working on into your prompt as context.

Leaning on a 20-year-old Apple API

Teaching an AI to understand what is happening across your desktop is one of the hardest challenges in software right now. According to Alexander Embiricos, OpenAI desktop product lead, the team solved it by mostly relying on macOS's built-in accessibility API.

That screen-reading framework has powered Apple's VoiceOver for nearly two decades. It is reliable at grabbing raw text from standard windows, though some tools require extra plumbing: VS Code users, for instance, need to install a dedicated extension before ChatGPT can query editor contents.

Because the accessibility layer only handles text, ChatGPT remains blind to visual elements like images, canvas layouts, or videos.

In practice, Work with Apps sends your last 200 lines of code alongside your prompt for certain apps, while for others it grabs everything in your active window. You can highlight a specific snippet to guide the model, but it still sweeps in surrounding text, which means it will chew through input tokens quickly.

It reads your project, but won't type for you

In a demonstration, an OpenAI employee opened a simple Xcode project modeling the solar system that was missing Earth. Within ChatGPT, the employee selected the Xcode tab and prompted the model to add the missing planet.

ChatGPT generated a matching line of code to represent Earth. But unlike dedicated AI coding environments such as Cursor or GitHub Copilot, it cannot write code directly into your files. The tester still had to copy and paste the snippet back into Xcode by hand.

The feature is out today for Plus and Team subscribers, with Enterprise and Edu rollouts planned over the coming weeks. OpenAI says text-based writing apps are next.

Why it matters

We think OpenAI took a pragmatic detour that trades grand ambition for immediate reliability.

Compare this to Anthropic, which recently released an experimental agent that takes screenshots of your desktop to navigate apps visually. While Anthropic's approach is more general, it is noticeably slow and error-prone. OpenAI instead piggybacked on a battle-tested accessibility framework, creating a fast, text-only bridge that actually works today.

For developers who prefer staying in Xcode or native terminals rather than switching to an AI-first editor like Cursor, this removes significant friction from everyday debugging. Just keep in mind that shoveling hundreds of lines of context into every prompt will eat into your usage limits.

More importantly, this lays the groundwork for what comes next. With reports pointing to an upcoming OpenAI agent codenamed "Operator" in early 2025, getting the chatbot to comfortably inspect other running software is the crucial first step. It cannot touch the keyboard yet, but your desktop is no longer a black box to it.