No jailbreak, no developer account: this project drives a real iPhone by reading pixels off a Mac window
phone-harness, published on 7 August, had gathered more than 1,500 stars in five days. It lets a language model operate an actual iPhone with no jailbreak, no Xcode and no WebDriverAgent. The trick is macOS Sequoia's iPhone Mirroring: the harness screenshots that window, runs Apple's built-in Vision framework over it to find text and its coordinates, then sends taps and keystrokes as low-level CGEvents, taking another screenshot to check the action landed. The limits are honest — one phone at a time, no pinch gestures, no Face ID or camera flows, DRM video shows up black, and the OCR reads words but not unlabelled icons.
Why it mattersPhone automation has always meant enterprise device management or a jailbreak. This is a few hundred lines built entirely from features Apple already ships to every Mac owner, under an MIT licence. It also relocates the security question: the agent is not calling an API that can be rate-limited or audited — it is a synthetic finger on your logged-in banking app, and the only thing standing between it and a mistake is the screenshot it takes afterwards.
Read in the app — free, in 9 languages
Related stories
TikTok put a shopping chatbot inside the video you are watching.2026-10-06Wikipedia's owner says OpenAI agents may have caused a May outage.
2026-10-06Meta's assistant keeps an hourly file on everyone in your life.
2026-10-05Two senators want prison time for bosses whose AI agents hack.
2026-10-05One command restores the Apple Intelligence off switch Apple deleted.
2026-10-05