No jailbreak, no developer account: this project drives a real iPhone by reading pixels off a Mac window
phone-harness, published on 7 August, had gathered more than 1,500 stars in five days. It lets a language model operate an actual iPhone with no jailbreak, no Xcode and no WebDriverAgent. The trick is macOS Sequoia's iPhone Mirroring: the harness screenshots that window, runs Apple's built-in Vision framework over it to find text and its coordinates, then sends taps and keystrokes as low-level CGEvents, taking another screenshot to check the action landed. The limits are honest — one phone at a time, no pinch gestures, no Face ID or camera flows, DRM video shows up black, and the OCR reads words but not unlabelled icons.
Why it mattersPhone automation has always meant enterprise device management or a jailbreak. This is a few hundred lines built entirely from features Apple already ships to every Mac owner, under an MIT licence. It also relocates the security question: the agent is not calling an API that can be rate-limited or audited — it is a synthetic finger on your logged-in banking app, and the only thing standing between it and a mistake is the screenshot it takes afterwards.
Read in the app — free, in 9 languages
Related stories
Apple Music will tell you when a song was made by a machine — but the uploader decides whether to say so2026-08-21DeepSeek's cheap workhorse can now see — and on agent tasks that need eyes it says it is close to Anthropic's best
2026-08-21Stripe has just paid $7.5 billion for a model router. Days later Ramp built one and is giving it away until January.
2026-08-21Meta's assistant is now a Mac app that reads your screen and types into any window — and what it sees can train the model
2026-08-21One click on a news site now tells Google to show you more of it — and you stay on the page you were reading
2026-08-21