Alpha2026Concept, architecture and development

ASTRO

A voice assistant that lives inside Ableton Live. It understands what you ask and carries it out through checked actions.

By voice
you ask Ableton Live to do something, and it does it
Almost 200 actions
the assistant can carry out inside the software
Hands on the instrument
while you play or record, without reaching for the mouse
ASTRO's Ponte interface, with the voice command bar above a keyboard cockpit.

The problem

People who make music on their own do many jobs: they compose, arrange, mix and play live. In Ableton Live, one of the most widely used programs for producing music, every move goes through menus, shortcuts and windows. When your hands are on the instrument, stopping to look for a command breaks the flow.

What it does

ASTRO is an assistant that lives inside Ableton Live. You ask it for something out loud or in writing, even from your phone, and it does it: it works on tracks, instruments, the mixer, clips, scenes and automation, and knows almost 200 different actions. A discreet panel on the Mac shows what is happening and then disappears.

It grew out of the way I work in the studio and on stage, for Ableton Live 11 and 12 on macOS.

The choices that matter

  • It talks to the software from the inside. Instead of faking mouse and keyboard, which is fragile, it uses the channel Ableton provides for being controlled from within.
  • Rules first, then AI. Simple commands are recognised and carried out straight away, without AI. AI comes in only when the request is more complex.
  • AI proposes, the software executes. The AI never touches Ableton directly. It drafts a plan made only of planned actions, waits for confirmation, and every action is checked before and after. You always know what it can do and what it has done.
  • When something goes wrong, it says so. If the AI service fails several times in a row, the assistant pauses it for a minute and says so clearly, instead of pretending all is well.

Problems solved along the way

  • Google blocked the addresses of the server hosting ASTRO, and the AI stopped until I routed the requests through different infrastructure.
  • Measuring real use. In June, automated tests and real use were mixed together in the statistics, and the success rate swung between 79% and 21% within a few days. Since then the two have been measured separately.
  • Not everything can be driven from the inside. Of the 193 catalogued actions, 110 go through the direct channel. The others need shortcuts or sequences of steps, which are more fragile, and a review found 41 still to be verified.

Where it stands

ASTRO is a private alpha for macOS. My assessment in July was “about 80% functional, not yet ready to sell”. Going further needs a signed app, an Apple Developer account and a beta period. The latest features date from July 2026; since then I have focused on other products.

What I learned

  • A closed list of actions makes AI explainable. You always know what it can do and what it has done.
  • Make it safe before you simplify. Before removing the old parts I added automated tests, and that surfaced two hidden bugs in the metronome.
  • Automated tests and real use must be measured separately.
  • Check before you rebuild. What you need often already exists and is more complete than it looks.

Try it

There is no video yet. The page for musicians is at silvestronelcosmo.com/astro, and I can show ASTRO live on a call.

How the information moves

The method I use with clients, applied to this project: follow the information from where it starts to how you know everything works.

  1. Where it startsAn Ableton Live session with tracks, instruments and clips, and the musician at the Mac or with a phone in hand.
  2. How it arrivesA spoken or typed request. Meanwhile the software keeps the assistant up to date on the state of the session.
  3. How it is describedEvery request becomes a clear intention, picked from a closed list of actions, each with its own rules.
  4. What happens to itSimple commands are recognised straight away; for complex ones the AI drafts a plan made only of actions from the list.
  5. Where it livesThe Mac remembers the recent context; accounts and statistics live in an online service.
  6. Who decidesThe musician confirms the plan, and every action is checked and must be one of the permitted ones.
  7. What happens nextThe assistant carries out the actions inside Ableton Live, from within the software whenever possible.
  8. How we know it worksEvery action reports how it went, and a before-and-after check flags when it had no effect.
For those who want the technical details

How it is built, for people who work in software or want to know what is underneath. Every number has a source.

  • A Python service on the Mac receives commands, spoken, typed or from the phone, and forwards them over TCP, using the NDJSON astro/v1 protocol, to a Remote Script inside Live.
  • Rule-based recognition in 22 modules in priority order. Only when no rule matches does Gemini 2.5 Flash step in, through the backend.
  • The Mentor proposes a plan with its tools switched off. After confirmation, only skills with a validated Pydantic schema run (130, of which 13 high-risk), chosen from a server-side allowlist.
  • FastAPI backend on Render with Supabase authentication and Stripe payments; model calls through a Cloudflare Worker; optional local mode with faster-whisper, Ollama and Piper.
  • iOS app with Capacitor, desktop panel in PySide6, interface in six languages. Contracts, design tokens and translations come from a single source, guarded by tests.
  • Built with several AI coding agents taking turns (Claude, Gemini and Codex), coordinated through a handover document with shared rules.
Actions available to the AI
130 skills with a validated schemasource: contratto astro_v2_skills.json
Catalogued Live actions
193, of which 110 via the native scriptsource: action_kb.json, 15/07/2026
Automated tests
570 in the core and 258 in the backendsource: HANDOFF.md, 10/07/2026
Commits
428 in the monorepo and 314 in the backendsource: git log, maggio–settembre 2026
Status
Private alpha on macOSsource: silvestronelcosmo.com/astro

Tools used: Python, PySide6, Ableton Live Remote Script, NDJSON su TCP, FastAPI, Pydantic, Supabase, Stripe, Gemini, Cloudflare Workers, TypeScript, Capacitor, PyInstaller

Have a project in mind?

Tell me in a few lines: what you'd like to build, who it's for, and what's in the way today. I reply within one working day, and the first conversation is free.

Write to meinfo@pietromey.com