Specifying an AI composer, frame by frame
The composer is the bar where users of an investment platform type to its AI. It showed every AI feature at once, so users faced nine controls before they could ask anything. I respecified it as twenty-six frames in six families of states, and gave each frame one line an engineer can test as pass or fail. The same composer serves both AI modes: analysis, when the user asks about what they are reading, and edit, when they change their work.
The composer, the bar where users type to the AI, showed nine controls before anyone could ask a thing. I respecified it as twenty-six frames, each with one line an engineer can test.
- Role
- Senior Product Designer. Interaction design, working prototypes, delivery spec.
- Users
- Institutional allocators and fund managers: time-poor, sceptical of automation.
- Timeframe
- 2026, ongoing.
- Built
- In Claude Design on the current design system, handed to engineering through Claude Code.
01 — Context & my role
One component sits between the user and every AI feature on the platform. This is where it lives and what it has to serve.
The composer is the way into every AI feature on the platform: one bar on the dashboard, and the same component inside reports, documents and meeting notes. It serves both of the platform's AI modes (the AI surfaces case covers them in full): in analysis, it is shown on top of the page the user is reading; in edit, it sits inside the report, document or notes the user is changing.
I was the designer on it: the audit, the interaction patterns, coded prototypes and the delivery board, the set of annotated screens engineering built from. The work is still shipping, so the figures are wireframes from that board, with invented labels.
02 — The problem
The old bar asked users to make too many choices before typing, and then mixed those choices into their question.
My audit found nine controls on the first layer, the part of the bar users see before they type: shortcuts, a shortcuts grid, context, source, research, web, history, workflows and send. Deciding which of them to use took more effort than typing the question. And anything the user selected appeared inside the text field, so they could no longer tell their question apart from the context they had added.
Ask anything…
The composer before the redesign. Nine controls surround the field before the user has typed a word. It is drawn in grey here and throughout the page; the redesigned bars below are in ink.
03 — Constraints
The redesign had to make the composer simpler without taking anything away. These are the limits every frame was checked against.
04 — The decision
One structural move: empty the first layer and tag the context below the input. Everything else in the spec follows from it.
05 — The states
The spec covers every state a user can meet, from the empty field to rare failures such as a file that can't be read. Each family below groups the states of one part of the composer.
Most states behind the first layer had never been drawn: a failed upload, a picker with nothing to pick, a missing microphone, a model control hidden by a flag.
Where the code sets exact values, the spec states them: a recording stops at 120 seconds, a countdown appears when fifteen seconds are left, and the waveform has 44 bars. The seven keyboard shortcuts are written down exactly, so engineering builds them as specified. One issue is still open with the Escape key: two panels can be open at once, and pressing Escape should close only the inner one.
First layer
First layer, then the second. On the first layer the user sees only the field and a few controls. The second layer, with every other feature, opens when they type / or press +.
Why: Keeping most features on a second layer lets the composer stay a single bar. Users start general questions from the dashboard, which had no room to add a column for the AI, so the composer had to fit the existing layout.
Field and send
Seven states between an empty bar and a failed send. Only the first and last existed as frames.
TypingSend turns live on the first character and the placeholder is gone, not faded.
Multi-line at the capThe field grows to its cap, then scrolls. The toolbar stays at the bottom and the attachment bar stays beneath it.
SendingThe field is inert but still legible, the mic is gone, and the only live control is Stop.
Could not send. Check your connection and try again.
Send failedThe message stays, the reason sits below the bar (not in a toast), and send is live for the retry.
InertWhile the AI is carrying out a task on the page, the composer is shown as inactive and none of its controls can take focus.
Attachments
The attachment bar, the row of attachment tags (drawn as badges) under the field, is the only place where the files a user has attached appear. Four of its five states had no design, and nobody had decided what happens when there are more attachments than fit.
One attachedOne badge on a bar that shares the composer border.
OverflowPast the bar width, the rest collapses into one +N count that opens the full list.
Overflow openedThe count opens a second row and no more. What still does not fit stays behind a +N badge.
UploadingA muted badge with a spinner. Send stays live and queues behind the upload.
board-pack-final.pdf could not be read. Remove it or try a different file.
Upload failedThe badge turns destructive and keeps its ×. The reason sits below the bar, not in a toast.
Pickers
The picker is the list where the user chooses what to attach, such as a workspace. The Picker component in the design system already had these states, but the delivery board showed only a full list, so they were missing from the spec.
Plus menu, in placeThe + opens the menu in the picker's slot: bottom-left, 8px above the composer, 360px wide. Escape closes it and returns focus to the field.
Where it opensWhen the user chooses Workspaces, the picker replaces the menu in the same spot, so nothing on screen moves. When they can select several items, the picker stays open.
SearchingAs the user types, both groups of results (funds and companies in this example) narrow to match. An emptied group disappears, heading and all.
No resultsThe term is quoted back and the field keeps focus, so the user can correct it without the mouse.
First run: nothing to pickA new user has no workspaces yet, which is different from a search with no results. The panel needs a button to create one, and the Picker component has no place for it. Still to decide: add that action to the component, or build this panel by hand.
Voice
Voice input has five states in the code. The delivery board had drawn only its settings popover.
Voice transcription is starting…
StartingA tinted overlay on the field and a spinner; the control is disabled and says transcription is starting.
RecordingThe waveform (44 bars, 2px minimum) replaces the field, with elapsed time on the right. The mic button shows a filled square, the stop icon.
Transcription stops in 0:13
Last fifteen secondsAfter 105 seconds of recording, a countdown shows the seconds left before the 120-second limit. Screen reader users also hear it announced at fifteen, ten and five seconds left.
Transcribing audio…
TranscribingSame tinted overlay as starting. The mic and settings chevron are disabled.
No microphone detected
UnavailableWhen voice input can't be used, the control shows one reason, checked in this order: the browser is not supported, there is no microphone, another composer is transcribing, voice input is still starting, or it is still transcribing. The control stays visible but can't be used, so the user can see why.
Source
Source, the setting for where the AI reads from, moved inside the + menu. Moving it was harder than it looked, for two reasons. The user picks only one source but can add many attachments, so the two behave differently inside the same menu. And one source, Meetings, had no row in the new menu.
Plus menu with MeetingsMeetings sits beside Workspaces as a source. Without that row, users lose the ability to ask across their meetings.
Modes and model
The composer has its own mode switches, such as Web, separate from analysis and edit. They moved into the bar as badges, and their off state, which is what users see first, had never been drawn. The model menu had been drawn from memory and was wrong three ways: a deprecated model, one not in the registry, and the wrong default.
Both modes offThe toolbar is the plus, model read-out, mic and send. Nothing fills the badges' space.
One mode onA mode is one badge with an × to turn it off, the same state as its plus-menu switch.
Model menu: multi-model onTen active models from four providers, deprecated ones filtered out, GPT-5.5 as default. Open question: grouped by provider as drawn, or flat as the component is today.
Model menu: flag offWhen the multi-model feature flag is off, users see only the OpenAI models, with no provider headings.
Model control hiddenWith the flag off, the read-out is absent and the mic sits beside send, with no gap.
Existing parts
Each of the eight parts of the old composer is marked remove, move, keep or decide, so engineering knows what happens to each one. Two are marked decide because nobody had settled them yet.
06 — Mapped for AI
The spec is written for the next builder, usually an AI tool, so every state is a rule it can follow rather than a picture to copy.
On this platform, new screens are usually generated with AI tools first, then refined by the team. A composer that existed only as a picture would be redrawn differently each time. So it lives in the design system as one component, with its states and pass-or-fail lines in its rules.
AI reuses it instead of drawing a new input.
How the design system is written for AI to follow is its own case: Design system.
07 — Impact
The composer is still being built, so the gains below come from the specification itself.
Expected gains
Adoption and time-to-answer will be measured as the composer rolls out.
08 — What I learned
Three lessons: one about the product, and two about the practice of specifying a component for engineers and AI tools.
09 — What this case doesn't cover
The composer is still in build. These are the things this case can't show yet, and why each one is still open.
Let’s build AI into a product with the structure to grow it.
Adding AI changes how a product behaves as well as how it looks. I design where the AI appears, specify every state it can be in, and leave components with rules that the team can build on.
Let’s talk