Bryan Falkowski All writing

Steering Claude Code Without a Keyboard

Bryan Falkowski · October 19, 2026 · Updated October 20, 2026 · Builds on Robert Kovacs's post on coding hands-free with Claude Code and Wispr Flow

When we make delegating work to AI effortless, do we also make careful supervision less likely?

I ran into that question by building an interface for Claude Code from speech and an 8BitDo Micro game controller. Speech says what I want, and a thumb press approves what the agent did. It works. Building a small web app with it took one keyboard press. It also makes approving a change cost almost nothing, while reading the change costs what it always did.

This post covers what I built, what the HCI course I took says is good and bad about it, and how to test whether easy approval makes reviewers careless. It separates what I saw from what I only suspect.

Two inputs

Don Norman wrote The Design of Everyday Things about doors, stoves and light switches. He could not have pictured Claude Code. His principles still apply to it, because they are about people, and people have not changed.

Not long ago, writing code without opening an editor was new. With agents like Claude Code, you describe the change and approve what the agent does. "Writing code" is a loose term for that, since Claude writes it and I steer. If steering is the work, the input devices can be designed around it, and the keyboard is one choice among several. My guess, and it is only a guess, is that keyboards will one day look as dated as typewriters.

In a post from April 13, 2026, Robert Kovacs described steering Claude Code this way daily for several weeks, with the same controller and Wispr Flow for dictation. He found that talking makes him state the intent, constraints and edge cases up front, so Claude's first attempt lands closer. He also found that speech cannot approve anything. Saying "yes" types the word "yes" into the chat box, so approvals need a real key.

Kovacs noticed that he explains himself better when he talks. That matters, because explaining what you want is a skill, and it is the one part of the work Claude cannot do for you. I suspect speaking builds that skill. When you say a vague request out loud, you can hear that it is vague. A typed request is easier to send without noticing.

That is the lens for the rest of this post. An interface for working with AI changes how fast you go, and it also changes what you practise. Speech may train you to say what you mean. One-press approval may train you to stop reading.

So the goal is not to take all the effort out of working with AI. It is to remove the effort that gets in the way of good work and keep the effort that builds skill. If speaking makes me clearer about what I want, that effort is worth encouraging. If one-press approval makes me less likely to examine a change, the effort it removed was the useful kind. Both are usability questions, but only one is about whether I can operate the interface. The other is about what the interface helps me get better at.

The prototype

The prototype has three parts, all on a Mac:

An 8BitDo Micro held in one hand, small enough that the fingers wrap around it. It has a D-pad and A, B, X and Y buttons.
The 8BitDo Micro. The shoulder buttons L and R sit on the top edge, under the fingers.

Each button does one thing in Claude Code:

ButtonKeyIn Claude Code
LRight OptionStart FluidVoice listening, and press again to stop
AEnterAccept, submit, or pick the highlighted option
BEscReject, or interrupt Claude while it works
D-padArrow keysMove between options in a question or a permission prompt
X/Open the command menu
RShift+TabSwitch Claude Code's permission mode
Plus, minusCmd+Shift+], Cmd+Shift+[Next and previous Terminal tab, one agent per tab
YCmd+TabSwitch back to the last app, such as from the browser to the terminal

The appendix has everything needed to copy it.

Three rounds to get it working

Round one: every button typed a letter. In keyboard mode, A types g, B types j, and the D-pad types c to f. Claude Code listens for Enter, Esc and the arrow keys, so out of the box the controller did nothing useful. 8BitDo's remapping app has no macOS version, and the iPhone version had a one-star rating when I looked, so I wrote a Karabiner rule instead.

Round two: the remap failed without saying so. After I approved Karabiner's permission prompts, the buttons still typed letters. Karabiner's log said the permissions were not granted, because its setup had stopped at the Accessibility step. Later, quitting Karabiner to restart it turned the remap off, and the buttons went back to letters with no warning. Both times the screen looked exactly like a rule with a typo. Only the log said what was wrong.

Round three: slash commands. Claude Code runs a command only when the message starts with a typed /. When I said "slash", FluidVoice typed the word. The fix was to map X to /. Pressing X opens Claude Code's command menu. Then I press L, say one word from the command's name, such as "setup", and press L again. The menu shows only the commands that match. The D-pad moves to the right one, and A runs it. Dictation only has to get one ordinary word right, and picking from a short list is what the controller does well.

Using it

To test the interface on real work, I built a small web app with Claude Code from the controller: the Micro Tester. It draws the Micro, labels each button with its action, and lights a button up when you press it. That makes it the first item on the next-version list below, built with the thing it fixes. The build took about twelve minutes, from an empty folder to a tested, deployed page. The counts come from Claude Code's session file, apart from the keyboard presses, which I kept track of myself.

The tester below is a display, not a controller. If you have a Micro paired with your computer in keyboard mode, click the box and press the real buttons. The matching button in the drawing lights up and shows what it does. Without a Micro, the keyboard keys it sends work too, such as g for A and j for B.

The Micro Tester shows presses from a real controller. Clicking the drawing does not press anything.
What happenedCount
Files Claude wrote or edited12
Shell commands Claude ran, for installing, testing, building and deploying13
Design decisions I made with the D-pad and A4
Actions I rejected0
Keyboard presses1

The one keyboard press came when Claude opened the page in my browser. Getting back to the terminal took Cmd+Tab, and no button sent it. Afterwards I mapped the unused Y button to Cmd+Tab, which closes that gap.

The more interesting number is a pattern. Claude asked its questions as multiple choice with a recommended option first, and I took the recommendation every time. Every approval was A. I knew the D-pad could move to another choice, and I never needed it. That could mean the recommendations were good. It could also be the start of the habit the rest of this post worries about, and from inside the session I could not tell which. The first time I moved off a recommendation came after the build, when I used the D-pad to turn down Claude's suggested next step and point out a problem. The embedded tester looked like on-screen controls, which led to the explanation above it.

I usually run several Claude Code agents at once, each in its own git worktree and its own Terminal tab. Plus and minus move to the next and previous tab, so going from one agent to the next is a single press. Getting to an agent is easy. Knowing which one is waiting for me is harder, because a tab does not say so from across the room.

Reading it as an HCI problem

I took Foundations of Human Computer Interaction at RIT during my master's degree. It was an elective, and one of the more interesting subjects in the program. The course splits human error into two kinds. A mistake is choosing the wrong goal. A slip is having the right goal and doing the wrong thing, and the fix for a slip is better interface design. Almost everything that went wrong here is a slip, and the course has a name for each one.

These principles answer one question, whether people can use an interface. The next section asks a different one, what the interface trains people to do. Usability tells you whether you can approve with one press. It does not tell you whether you read first.

The next version

The prototype is usable, not polished. Its value so far is that it works well enough to expose real design problems. One choice held up and is worth keeping. Speech for intent and a short menu for decisions give each input the part it handles well, which is why the slash command fix works. The next version would add four things:

Easier is not the same as better

With this interface, approving a change costs one press of the thumb. Reading what you approve costs the same as it did before. I expect that gap to pull toward approving without reading. Kovacs's post does not look at it.

The course has names for this risk too. A capture error happens when a frequent action takes over a rarer one that starts the same way. If A approves fifty routine prompts in a row, the press on the fifty-first prompt, the one that needed reading, is the same thumb movement. Reviewing AI output is also a vigilance task, meaning continuous attention to a long, monotonous stream where the rare problem is easy to miss. Universal Design principle 5d says what to do about it: "Discourage unconscious action in tasks that require vigilance."

Speed is the wrong measure for a review step, and the cost of careless review reaches past one change. Teams check AI systems against human judgments. My primer on LLM judges measures a judge against human labels, and its numbers are only as good as those labels. A reviewer who approves without reading makes the judge look better than it is, and every decision based on that number inherits the error. The interface a reviewer uses is part of the measurement, so how it shapes their attention is a measurement question as well as a design one.

If approving is cheap, the interface has to protect the reading that should come before it. These changes would help, and none of them slows down routine work:

Whether the controller makes reviewers worse can be tested instead of guessed. A fair test needs more care than counting mistakes.

Even then, a short study measures performance, not capability. It could tell me whether the controller changes how well people review in one session. It would not tell me whether the controller helps or hurts them as reviewers over time. That would take measuring whether people improve with practice, keep what they learn, and carry it over to unfamiliar kinds of change.

What I saw and what I suspect

ClaimStatus
Building a small web app with Claude Code, with prompts, approvals, menus and commands, needs almost no keyboardObserved, one keyboard press in the build
The setup fails silently when permissions are missing or Karabiner is not runningObserved
With a recommended option shown first, I accepted it every timeObserved, in one twelve-minute build
Speaking makes prompts state intent and edge cases more clearlyReported by Kovacs, not tested by me
Speaking trains that skill over timeMy guess
One-press approval leads to approving without readingHypothesis, based on capture errors and vigilance. It needs the test above.
Keyboards will look datedA prediction, not a finding

Time will tell

I do not know yet whether I will keep using this. Habits are strong, and some things, like an exact file path, may still be easier to type. What the experiment shows is narrower than a prediction about keyboards. For one workflow, steering Claude Code, from prompts to approvals to commands, the keyboard turned out to be almost optional, and the one press it still needed has a fix. Whether that is good for the person steering is the question this post leaves open.

None of this is new for everyone. People who cannot use a keyboard have had other ways in for a long time, such as voice control and switch access, both built into macOS. What has changed is the work. When what we do at the computer becomes describing and deciding, the rest of us can step back from the keyboard too, and the old HCI principles tell us what to watch for when we do.

The controller is a small experiment, but the question applies to any AI tool that separates doing the work from approving it. Such tools should be judged on more than whether people can finish the task. They should also be judged on whether they help people make better decisions about the result.

Appendix: set it up yourself

You need the three pieces above and Claude Code in a terminal. I bought the Micro on Amazon.

Connect the controller.

  1. Slide the Micro's mode switch to K.
  2. Hold the pair button on the top edge for about 3 seconds, until the light blinks fast.
  3. On the Mac, open System Settings, then Bluetooth, and connect to "8BitDo Micro gamepad".
  4. Open TextEdit and press each button in a known order to see what it types.

Mine typed these:

ButtonSends
A, B, X, Yg, j, h, i
D-pad up, down, left, rightc, d, e, f
L, Rl, r
Plus, minuso, n
The two remaining buttonsk, m

Install Karabiner. The installer adds a keyboard driver, so it asks for your password.

brew install --cask karabiner-elements

Open Karabiner-Elements and follow its prompts. In System Settings, under Privacy & Security, turn on Karabiner-Core-Service in both Input Monitoring and Accessibility. Karabiner sees every key you type once you grant these, which is how any remapper works. Read its source if that matters to you. If the buttons still type letters, check /var/log/karabiner/core_service.log. It says when Karabiner has taken over the controller.

Add the rule. It goes in ~/.config/karabiner/karabiner.json, under profiles[0].complex_modifications.rules. Each line turns one letter into one key. The device_if condition limits the rule to the Micro by its vendor and product IDs, so your real keyboard still types g. Run system_profiler SPBluetoothDataType to confirm the IDs for your unit. Mine reported vendor 0x2DC8 (11720) and product 0x9021 (36897).

{
  "description": "8BitDo Micro: A=Enter, B=Esc, D-pad=arrows, L=FluidVoice, R=Shift+Tab, X=/, plus and minus=tabs, Y=Cmd+Tab",
  "manipulators": [
    { "from": "g", "to": "return_or_enter" },
    { "from": "j", "to": "escape" },
    { "from": "c", "to": "up_arrow" },
    { "from": "d", "to": "down_arrow" },
    { "from": "e", "to": "left_arrow" },
    { "from": "f", "to": "right_arrow" },
    { "from": "l", "to": "right_option" },
    { "from": "r", "to": "tab", "modifiers": ["left_shift"] },
    { "from": "h", "to": "slash" },
    { "from": "o", "to": "close_bracket", "modifiers": ["left_command", "left_shift"] },
    { "from": "n", "to": "open_bracket", "modifiers": ["left_command", "left_shift"] },
    { "from": "i", "to": "tab", "modifiers": ["left_command"] }
  ]
}

That is shorthand. Karabiner wants each line written out in full, with the device condition on every one. Here is the first line in Karabiner's format. The others follow the same shape.

{
  "type": "basic",
  "from": { "key_code": "g", "modifiers": { "optional": ["any"] } },
  "to": [{ "key_code": "return_or_enter" }],
  "conditions": [{
    "type": "device_if",
    "identifiers": [{ "vendor_id": 11720, "product_id": 36897 }]
  }]
}

Karabiner reloads the file when it changes. Press A in TextEdit to check. It should start a new line instead of typing g.

Notes. FluidVoice's hotkey was already right Option, set to start and stop on separate presses, so the L button only sends that key. Kovacs used a key combination for Wispr Flow because Wispr's default key, fn, could not be assigned to a controller button in 8BitDo's app. I left the two remaining buttons alone. They still type letters, so a stray press puts a letter in your prompt. Map them to vk_none in the same rule if that happens to you.

References