On 2026-10-11 the user asked for a plan (not code yet) for Orbit to use the PC for them: drag Orbit onto a window and tell it a task (book a flight, comment on X/TikTok, chess), voice plus typed prompts, picks windows and opens apps itself. Plan written as a Claude Doc "Orbit, do it for me": https://claude.ai/code/artifact/21267e72-27a1-4757-b068-9dcc28da1309

Key design: brain = Claude Code headless (claude -p, stream-json, --mcp-config with an island MCP server) on the user's own subscription, like the 0.5 chat runner (commit 17b3e72). No API key (Agent SDK docs: third-party apps may not offer claude.ai login to other users without approval, so sharing Island would need API keys). Hands = new UI Automation functions in the C# bridge behind a safety gate in main; ears = whisper.cpp multilingual small (English + Urdu), hold-to-talk; milestones 1 Hands (typed) → 2 Ears → 3 Perch (drag onto window) → 4 Smarts (memory, skills, show-me-once) → 5 Extras.

Boundaries stated in the plan: no playing online chess against people for the user (coach / vs computer only), social posts one at a time only after the user taps Post, never types passwords/cards, stops at payment.

Why: the user wants Orbit to act, not just show; the CLAUDE.md "Not now" rule (no AI API, Claude Code only) still holds because the brain is their Claude Code.

How to apply: milestone 1 shipped as 0.11.0; 0.11.1 (2026-10-11) made it native like Bluey (OCR eyes, real clicks, no extension). Next: milestone 2 voice, 3 perch. Keep Island 1.0 blueprint constraints. See Orbit native, no extension.

User decisions 2026-10-11 (while reading the plan): any window, not just Chrome; Orbit finds or opens the right window/tab himself; SPEED is the top requirement ("if it's slow I have no use for it") — targets in the doc: quick command <0.3 s, one-link task <2 s to Send lit, one screen 2-4 s; model = Haiku 5.5 (step escalates to Sonnet 5.5 only after two failures); comments: post or "draft me a comment"; messages/posts: Orbit types it and highlights Send/Post (ui.highlight) so the user just presses Enter — this replaces the island confirm card; hands "whatever Bluey uses" = accessibility layer + simulated input (UIA + SendInput on Windows). Speed levers: warm claude -p session, short custom prompt, UIA text snapshots, batched ui.run, deep links in skills, pre-reading the dropped window.

Milestone 1 built and installed as 0.11.0 on 2026-10-11 (branch feat/orbit-hands, commits 940bd1a..196c39b, not merged to main, not pushed). Typed tasks via Ask Orbit, gate, quick commands, Opera extension (extension/, user must Load unpacked from %LOCALAPPDATA%\Programs\island\resources\extension). Orbit's tasks use the user's Claude plan: a smoke test hit "session limit" — tasks fail until it resets. Next milestones: 2 voice, 3 drop on window, 4 memory/skills, 5 extras.

Status 2026-10-11 (end of day): 0.11.4 installed (native OCR eyes, memory, research playbook, own tab, tidy-up). Next session the user will share Bluey's source code to study how it works, and wants Orbit to do hard tasks off-screen (a VPS, or a browser of Orbit's own) so the user's screen only shows the final result. Pick whichever is quickest while staying very smart. Open flaws: sometimes types links into the address bar, long answers truncated on the island (idea: a full comparison page), lit Send in apps doesn't notice Enter.

Status 2026-10-12: a fresh thread built 0.12.0, committed as 86a0ebf on local branch feat/orbit-browser (off feat/orbit-hands at 9a8d57b), and installed it on the user's PC: (1) two brains in one warm session: Haiku 5.5 for quick tasks, Sonnet 5.5 (medium) for research/hard tasks, switched with a stream-json apply_flag_settings control line (40 ms, works mid-task); a quick task that fails twice, wants to hand over, or takes 10 steps is switched mid-task. (2) Orbit's own hidden browser on the PC (offscreen Electron window, in-page reader in an isolated world, real input events, browse tool, page = window 0, nothing sent/posted out of sight, best page opened on screen after research). Measured: cheapest flight to Dubai, two sites compared, 12 s (was 26 s). The VPS idea was not needed. The same day the old build thread added the GitHub remote github.com/mfxruk1-del/island and was pushing main and feat/orbit-hands.

Later on 2026-10-12 the user shared Bluey's source (Downloads\bluey-by-riley-main; Swift Mac + iPhone, OpenAI realtime brain). Study notes are in docs/ORBIT-TASKS.md "What Bluey taught". Taken into Orbit as 0.12.1 (commit f212ef1, installed): report card on hover, point tool (purple ring), "this" = what the mouse is on (bridge pointer), no narration between steps, lock/task-manager keys refused. The user said no to Bluey's visible flying hand: clicks stay instant (see Instant, no hand). Word-level OCR refs not taken yet.

The user then asked whether "no talking between steps" makes Orbit dumber (answer: no; his thinking is a separate step, transcripts show thinking blocks) and wants him "the most smartest and rapid" (calls Haiku "dumb"). Measured per step on easy steps (2026-10-12): Haiku low ~0.95 s, Haiku medium ~1.0 s, Sonnet low or medium ~1.6 s; first step Haiku ~1.2 s, Sonnet ~2.1 s. Offered: Haiku medium for quick tasks (recommended), Sonnet for everything (+0.6 s/step, more plan use), or keep. On 2026-10-15 the user said "ok do it quick": quick tasks now run Haiku medium (0.12.2, commit c02616a, installed). The user also wants changes explained and approved BEFORE building; don't code until they say go.

2026-10-15: the user asked Orbit to learn from coucou (Downloads\coucou-main; its agent is "OK Coucou", a voice assistant: energy VAD + offline Parakeet via sherpa-onnx on Windows, a fixed phrase parser first, Claude Haiku with an API key and a few tools otherwise; no screen control). Study only, nothing built. Offered, ranked: 1 plan usage on the island (Orbit's claude -p stream already sends rate_limit_event with five_hour/seven_day utilization), 2 follow-ups (remember the last exchange ~90 s), 3 more instant commands (volume, reminders, last download, undo, two commands in one sentence), 4 typo-proof names, 5 voice (Parakeet has no Urdu; Whisper for Urdu), 6 dictation anywhere. Waiting for the user's pick.

2026-10-15, later: the user said "i thought orbit has memory, we will fix that later" (his notes in memory.json persist, but every task is a fresh Claude session, so follow-ups land cold), then asked to learn from OpenDots (Downloads\OpenDots-main; CopilotKit web template: a Docker computer per assistant via OpenBot with human takeover, approval cards that run only the stored request, repeat tasks with leases and "retry after review", owner-managed memories passed as untrusted data, voice = OpenAI realtime talker handing work to a compute agent; brain, voice and search all need API keys). Study only. Offered, ranked: 1 two small safety fixes (his hidden browser refuses localhost and private addresses unless the person typed them, since webOnly allows any http(s); a task cap of ~3 min or 60 steps, since the runner passes no --max-turns), 2 take over his hidden browser for robot checks (same partition; he stays signed in to nothing), 3 repeat tasks / price watch, 4 task history (answer, report, steps, never typed text), 5 a report line when a planned source blocked him. For the later memory fix: let the person see and delete his notes. Waiting for the user's picks.

2026-10-15, picks: follow-ups (from coucou), see/edit/add/remove notes, OpenDots #2 robot-check takeover; "leave 1" (no home-network block, no step cap). Built as 0.13.0 (commit a821ffa on feat/orbit-browser, installed, not pushed): session kept 10 min after a good task (SessionKeeper), recent.json note for fresh sessions, Memory page in the dock, robot-check window with ClickFix defenses (site lock, clipboard restore, ends at sign-in, inactive window). Live: robot check cleared and price read in ~8 s; follow-up answered from memory in 1.6 s. Also fixed Snippets Paste (Electron 44 clipboard is async). The user then wants deals "that wow" them: offered 1 several pages at once, 2 deal playbook, 3 price history, 4 price watch, 5 deal card (recommended 1-3 together); waiting for their pick. The coucou list (plan usage etc.) is still open too.

2026-10-15 evening: the user reported the lit Send never notices their Enter and said "do the 1 to 3". Built and committed as WIP f878003 on feat/orbit-browser (NOT installed; installed is still 0.13.0; 445 tests pass): bridge/SendWatch.cs (LL keyboard/mouse hooks while lit; Enter not Shift+Enter, or a click on the lit rect judged by the foreground window 150 ms later), browse urls (2-4 sites side by side), deals.ts playbook (+ price history) on Sonnet, Cloudflare pages read from the check window ("seen"), reportFromAnswer fallback card. Live: RTX 5070 hunt found Rs 214,990 with a report card. Left: code review, docs (ORBIT-TASKS, CLAUDE.md), version bump to 0.14.0, installer + install, and a live check of the send watcher in Island itself. Lesson: tests that click must use the topmost test window (a click at a test button's spot once landed on the user's Opera); kill dev Island by PID tree, never by image.

2026-10-17: 0.14.0 finished (review fixes: Ctrl+Enter for posts, same tab only, software presses ignored), committed 6aa3909 and installed. The user then asked how notifications and Orbit's answers should share the island and how to make Orbit an assistant for big, difficult jobs. Their picks: the notification fix "artistic style", big-job mode (asked how it decides: Orbit decides by writing a plan, like Claude Code's to-do list; "take your time"/"deep dive" force it), full reports as pages, reading their files, and helpers "as long as the hands or helping workers are haiku 5.5". Built as 0.15.0 on feat/orbit-browser: drops at the island's edge while hovering, ending time counted only in view, last answer chip on Home, plan tool + rail, big brain (Sonnet high), helper subagents (Haiku 5.5, read-only, CLAUDE_CODE_DISABLE_BACKGROUND_TASKS=1 or they finish after Orbit's turn), report pages in Documents\Orbit (the user's Documents is Desktop\Documents), find_files/read_file (bridge Paper.cs for PDFs/pictures). Live: earbuds report page 336 s, three cities 38 s, three sights with helpers 83 s. Lesson: auto mode refuses running claude -p with Orbit's flags directly; test through Island. 0.15.1 (commit 01f4f4a, installed 2026-10-17 23:08): code-review fixes. read_file reads a file straight away only if it's on the shelf or its name shares a word with the task; otherwise the island asks "Read X from Downloads?" once per file per task (prompt-injection exfiltration defense). WebFetch removed from Orbit's brain. Office parsing hardened (linear scans, 24 MB cap). The user was already using 0.15.0 live: a Karachi flight (31 s), finding their paid Anthropic invoice PDF in Downloads, UK weather table. 0.16.0 (commit dc07eaf, installed 2026-10-18 ~12:00): the user picked "reminders and timers" and "plan usage on the island" from my suggestions, plus "make Home more modern, better fonts" after a screenshot of the First Light sunrise stuck behind the panel. Built: reminders/timers parsed without AI (main/reminders/), ringing card, timer pill, Up next; plan use from Orbit's rate_limit_event (shared/planUsage.ts), shown on Home's Claude widget; Home rebuilt as phone-style widgets; content clipped by the moving shape (.content-clip); welcome watchdog. Font still OPEN: I asked Inter (my pick) vs Plus Jakarta Sans (close to Gilroy, which they have, only ExtraBold installed); needs their OK to install a package. Fonts are tokens (FONT in tokens.ts), so the swap is one import plus two strings.