Controlling a phone

Endpoints#

A phone held by a run or another key answers 409 leased. If someone takes over the phone in the console's live view, your commands fail with preempted until they let go.

Observe#

screenshotobject

Include one. maxEdge is 240 to 2000 pixels (960 by default), format is jpeg or png, and quality is 30 to 95.

uiTreeboolean

Include the elements on screen, up to 400. Off by default, because apps can notice it being read.

foregroundboolean

Include the app and activity in front, and whether the keyboard is up. On by default.

settleobject

Wait for the screen to stop changing first: up to maxMs (3,000 by default) for quietMs of stillness (500 by default).

The screenshot comes back as base64 with its width and height. Send coordinates to act in that picture's pixels with its size as frame, and the phone scales them to its screen. Without a frame, they are the screen's own pixels.

Act#

An action is an object with a type and the fields for it. Add then, which takes the same fields as observe, to see the result without a second call.

curl https://api.distilled.cx/v1/phones/phone-1/act \
  -H "Authorization: Bearer $DISTILLED_KEY" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: tap-search-0001" \
  -d '{
    "action": { "type": "tap", "x": 216, "y": 212 },
    "frame": { "width": 432, "height": 960 },
    "then": { "screenshot": {} }
  }'

Send an Idempotency-Key header with an action you might retry, and a repeat of the same request is not played twice.

Files#

Put a video, a picture or any other file on a phone. A picture or video goes to the phone's camera roll, so its gallery and every app's media picker show it, and anything else goes to Downloads. The answer comes once the phone has the file, and says where it is and whether the gallery lists it. Unlike observe and act, this holds no lease, so it works while someone else drives the phone.

The phone takes JPEG, PNG, WebP, GIF, HEIC, MP4, MOV, WebM, MKV, MP3, M4A, WAV, OGG, PDF, text, CSV, JSON and ZIP files, up to 100 MB each. Start an upload, send the file in parts of 16 MB to the address the answer names, each with its token, and finish the upload with every part's number and etag. A part that fails can simply be sent again. The parts go straight to storage, not through this API.

# Start. The answer names where to send the parts (uploadUrl), the token to send them with, and partBytes.
curl https://api.distilled.cx/v1/phones/phone-1/file-uploads \
  -H "Authorization: Bearer $DISTILLED_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "name": "long.mp4", "size": 80000000 }'

# Cut the file into parts of partBytes (16 MB) and send each straight to the file store, numbered from 1.
# Each answers { "part": 1, "etag": "..." }.
split -b 16777216 -d long.mp4 part-
n=0; for p in part-*; do n=$((n+1))
  curl -X PUT "$UPLOAD_URL/parts/$n" \
    -H "Authorization: Bearer $UPLOAD_TOKEN" \
    --data-binary @"$p"
done

# Finish with every part's number and etag. The file is on the phone when this answers.
curl https://api.distilled.cx/v1/phones/phone-1/file-uploads/pfu_0123456789abcdef/finish \
  -H "Authorization: Bearer $DISTILLED_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "parts": [{ "part": 1, "etag": "a1b2c3" }] }'

• An upload stays open until expiresAt, an hour plus two minutes for each part, and its token is never renewed.
• A workspace can have 5 open at once; a sixth answers 429 busy until one is finished or dropped. It can start 20 GB of uploads a day (UTC); past that a start answers 429 quota-exceeded.
• Every part is partBytes long except the last. A part that failed can be sent again with the same token, in any order, and the last copy is kept. Once you finish, the upload is closed whatever the answer.
• The upload's token lets in parts of that one upload and nothing else, so a web page on any site can send the parts from the browser without your API key: start and finish the upload from your server, and hand the page only uploadUrl and token.