Alpha Phone Use 0.1.2 is an early release. Report a problem

Phone Use

Phone Use

Let an AI agent, or you, see and operate a real iPhone from your computer.

The iPhone sends its screen to your computer over AirPlay, and your computer taps and types on it as a Bluetooth mouse and keyboard. Agents drive it from the command line or over MCP.

Download for Linux Download for macOS
  • No jailbreak
  • No developer account
  • No cable
  • Free and open source
AirPlay: the iPhone's screen, to your computer Bluetooth: taps and keys, to the iPhone Wi-Fi: to the bridge, every packet signed Left: your iPhone, unmodified. Right: your Linux PC or Mac, and the optional bridge. Illustration

Two links to the phone

Phone Use relies on two features every iPhone has, Screen Mirroring and AssistiveTouch, so the phone needs no jailbreak and no developer setup. The Phone Use iPhone app adds the controls for allowing and stopping agents.

The screen comes in over AirPlay

On the iPhone, open Control Center, tap Screen Mirroring and choose Phone Use on your computer. The computer receives the picture the way an Apple TV would, and shows it in a window and in your browser.

~100 ms from a pointer move to the mirror showing it

Taps go out over Bluetooth

Your computer presents itself to the iPhone as a Bluetooth mouse and keyboard, with its own Bluetooth or through the bridge. With AssistiveTouch on, iOS shows a pointer: a click taps, a drag swipes, and your keyboard types.

114–215 reports per second over the computer's Bluetooth

Setting it up

  1. Install Phone Use

    On Linux or a Mac. The window walks you through the rest, and on Linux it offers to open the firewall ports the iPhone needs.

  2. Pair the keyboard and mouse

    In the iPhone's Bluetooth settings, tap Phone Use for your computer's Bluetooth, or Phone Use Bridge for the bridge.

  3. Pair the iPhone app

    The app gets a one-time link from your computer. Agents are refused until an app is paired.

  4. Start Screen Mirroring

    From Control Center, to Phone Use on your computer.

  5. Turn on the pointer

    AssistiveTouch, in Accessibility settings. Phone Use can also ask Siri to turn it on, out loud through your computer's speakers.

Give your agent the phone

Every action is a phoneuse command, and the MCP server offers the same actions to Claude Code and other MCP clients. After each action, Phone Use waits for the screen to react, then returns frames from the change and the screen's text with where each line sits.

Example session
$ phoneuse open Settings
$ phoneuse text
0.500 0.081  Settings
0.331 0.312  Wi-Fi
0.876 0.312  Home
0.360 0.358  Bluetooth
0.893 0.358  On
$ phoneuse tap-on "Wi-Fi"
0.331 0.312
$ phoneuse done
# the phone goes back to you

Connect Claude Code

In the Phone Use app, Connect Claude Code adds the MCP server, installs the Phone Use skill and puts a phoneuse command on your path. Any other MCP client can start the server itself:

Linux flatpak run --command=phoneuse-mcp co.lxe.PhoneUse
macOS "/Applications/Phone Use.app/Contents/Resources/core/bin/phoneuse-mcp"

Reading the screen as text costs a few hundred tokens, against about 1,300 for a screenshot. The text comes from OCR that runs on your computer.

You decide when an agent gets the phone

The Phone Use iPhone app is where you allow and stop agents. Until it's paired, your computer refuses every agent action.

Agent control
Let agents start at once, after a 5 to 30 second countdown you can cancel, or only when you tap Allow.
Live Activity
While an agent works, your lock screen and Dynamic Island show its current action, its step count and how long it has been going, with a Take over button.
Hand-back
A session ends when the agent hands the phone back, when you take over, or after 60 seconds with no action.
3 Monday, November 9 9:41 Agent is using your iPhone Opening Settings Step 3 · 0:42 Take over
Illustration of the Live Activity.

The bridge

The bridge is a small ESP32 board that the iPhone pairs with as its keyboard and mouse. Phone Use reaches it over Wi-Fi, so the board can sit next to the phone on a battery or a USB charger while your computer's Bluetooth stays free. The Mac app sends all its input through the bridge.

Phone Use Bridge Wi-Fi Home 2.4G −58 dBm iPhone 38 reports/s USB-C 2 BUTTONS 1.14-INCH SCREEN
LilyGO T-Display, drawn. Get the original ESP32 T-Display, not the T-Display S3, which has a different chip.
Board
LilyGO T-Display: an ESP32 with a 1.14-inch screen, two buttons and USB-C. Nothing needs soldering.
Flashing
Plug it in over USB and the Linux app flashes the firmware for you. Or flash the release's .bin with esptool.
Setup
Over Bluetooth LE, no cable: your computer and the bridge show the same 6-digit code, then you pick a 2.4 GHz Wi-Fi network.
Security
Every packet to the bridge is signed with a key from setup, so nobody else on the network can type on your phone.
Speed
15 ms median round trip per report, measured on a −80 dBm Wi-Fi signal.
Get the board from LilyGO LilyGO on Tindie
Phone Use iPhone · connected USB-C STATUS SCREEN CONCEPT · NOT FOR SALE
Concept drawing. A finished bridge doesn't exist yet.

A finished bridge, if there's interest

A ready-made bridge would be the same ESP32 firmware in a small case with a status screen: plug it into any USB charger near the phone and set it up from the app. Leave your email and you'll hear from us if it gets made. Nothing is for sale yet.

Download

Phone Use 0.1.2. Your iPhone and your computer need to be on the same Wi-Fi network.

Checksums, release notes and earlier versions are on GitHub releases. To run it from source, see the README.