Bryan Falkowski

I'm a Principal Software Developer who builds AI features for enterprise applications. That includes generative AI workflows for requirements capture, application planning, test generation, and developer assistance.

Most of my recent work is about making AI behavior testable and ready for production through instrumentation, evals, error analysis, and quality gates. I have an M.S. in Data Science from RIT, where my graduate work focused on machine learning, NLP, and evaluation reliability.

Posts

Topic:

Earning the Next Rung

My office of Claude Code sessions is building Exhibit, a tool that decides what evidence a pull request needs and learns its rules from the bugs that got through. The design, and the first real numbers on the way to letting Autopilot merge some changes with nobody reading the code.

Notes, November 2026

Making a Realistic Trailer From a 3D One

How I remade the 3D Building58 trailer as a realistic AI video for about 250 Higgsfield credits. Claude priced two plans before anything was made, then ran Higgsfield and did the editing. All ten cast prompts are in the post.

Notes, November 2026

Making a Short Film From a Short Story

How I made a first-draft short film of my story Doctor Bones in one day. Higgsfield made the pictures, video and voices, and Claude Code ran the tools and did the editing. The biggest lesson was to make long takes and cut them up in the edit.

Notes, November 2026

Building58 Trailers

Short films of one room of my Claude Code sessions at work, on a break and at a party, with its own 80s music.

Video, November 2026

The Second Office's First User

I gave a stalled eval tool of mine to a second room of Claude Code sessions. Its PM used the tool, filed what was broken, and the room fixed it. The bugs that mattered most showed up only in real work, and the open question is whether simulated users reach their goals the way real users would.

Notes, November 2026

The Office Builds Itself

At 2:48 on Saturday morning, while I slept, my office of Claude Code sessions merged a fix to its own code and restarted itself to run it. What changed in a week, and what broke when the agents started running their own loop.

Notes, November 2026

An Office for My Claude Code Sessions

A 3D loft office where each developer is one of my Claude Code sessions. They type while it works, raise a hand when it needs my OK, and get coffee when it is done. By the afternoon its own sessions were filing, building and reviewing its tickets, and nothing moved on the board unless I let it. Then I wrote it a constitution, and a path toward a software factory.

Notes, November 2026

Building a 3D World for a Short Story

How I built Run 4,113, the 3D kitchen for my short story Doctor Bones: a three.js scene written with Claude, characters made with Meshy from a ChatGPT reference sheet, and what still looks wrong.

Notes, November 2026

Who Owns the Process? Spec Frameworks, Skills, and the Trust Ladder for Coding Agents

Spec frameworks like GitHub Spec Kit establish intent, skill packs like Matt Pocock's capture process, and orchestration coordinates the agents. None of them tells you whether the work is right. Why verification closes the loop, and how Lauren Tan's trust ladder lets agents take on more as the evidence grows.

Notes, October 2026

Steering Claude Code Without a Keyboard

A hands-free interface for working with Claude Code, built from speech and an 8BitDo Micro game controller. What it took to get working, what it shows about designing controls for AI work, and why the measure that matters is whether reviewers stay careful, not how fast they approve.

Notes, October 2026

Can You Trust Your Judge's Score? A Primer on TPR and TNR for LLM Judges

What an LLM judge's score really tells you, worked through one small example: is this customer service reply polite? Label a sample, build the confusion matrix, measure TPR and TNR, improve the judge, correct its raw score, and report the result with a range.

Primer, October 2026

Is This Run Valid? Attributing Pipeline Failures Before Scoring LLM Evaluations

Checking whether an eval run worked before trusting its score. A failure taxonomy, a run fingerprint, OpenTelemetry traces and a validity gate, tested with injected faults across 235 real runs and a GitHub Actions pipeline that breaks itself.

Paper, October 2026

Can a Browser Test Agent Tell When It Can't Tell? Ambiguity and Calibration in Element Selection

An empirical study of whether LLM test agents refuse when the element they are asked to act on is missing or cannot be told apart from look-alikes. Jev, Claude Haiku 4.5, Claude Opus 5.5 and a playwright-mcp agent, on recorded and live pages.

Paper, September 2026

Agent Simulation via Trace Replay: Raindrop Approach + OpenAI Paper

How replaying real production traces against a change to an agent works in practice, using Raindrop's approach and evidence from OpenAI's deployment simulation paper.

Notes, September 2026

LLM-as-a-Judge Repeat Stability with MT-Bench

How consistently LLM judges reproduce the same score on the same response. Seven frontier judge models, three prompting conditions, and five repeats per item on an MT-Bench subset.

M.S. capstone, Rochester Institute of Technology

Projects

Topic:

Building58

A building of 3D loft rooms where Claude Code sessions work as a squad: a Lead, Devs, reviewers and a PM, with me as the manager. Everything a room shows comes from real sessions, tickets and pull requests.

JavaScript, three.js, Claude Code hooks

Evaluation-Driven Design Platform

A deployable evaluation-driven design system for building, testing, and improving AI agents.

Python

Eval Run Validity Gate

Checks whether an eval run worked before its score is trusted: run fingerprints, OpenTelemetry traces and a validity gate, tested with injected faults in a GitHub Actions pipeline.

Python, OpenTelemetry, GitHub Actions

Browser Agent Dashboard (Jev vs Claude)

A split-screen dashboard that runs a purpose-built decision model (Jev) and a Claude agent driving playwright-mcp on the same browser test, with the element picked, confidence, latency, cost and a screenshot for each step.

Python, Playwright

EDD Agent Lab

A local workspace for building LangGraph agents through evaluation-driven design: targets, rules, eval contracts, runs, failure evidence, and verified fixes.

Python, LangGraph

LLM Evaluation Service Stack

Containerized service for running LLM evaluation jobs: a FastAPI base with async processing, tracing, and persistence, a Streamlit operator console, and a Helm/Kubernetes deployment.

Python, Helm

Other writing

Doctor Bones

I finished reading Player Piano by Kurt Vonnegut and felt inspired to write a short story.

Short story, October 2026