K
For a while, my workflow for building ML applications with coding agents looked something like this:
At the beginning, this worked surprisingly well.
The changes were small, the codebase was familiar, and I could still keep the whole thing in my head.
Then the application grew.
A seemingly simple feature could now involve preprocessing, model inference, postprocessing, and application logic.
The agent might touch several modules and add a few hundred lines of code in a single session.
My habit didn’t change.
I was still reviewing the code after every session.
And that became the problem.
When a coding agent changes a few lines of code, reviewing the diff is easy.
When it changes several hundred lines, it is still manageable.
Once you get to +1000 lines everything starts to fall apart…
You can read the code without really understanding whether the application is working properly.
At some point I realized that I had become the bottleneck.
Looking back, I think I was trying to solve the wrong problem.
When an AI agent writes code, the obvious way to stay in control is to review the code.
So that’s what I did.
I would look at the diff and ask:
I was trying to control the means.
The code was just the mechanism by which the agent produced the application, so I treated the code as the thing I needed to control.
The code was never the end goal.
I should never have cared whether the agent used one helper function or three. I didn’t care whether a transformation lived in one module or another.
That distinction became increasingly important as agents got better at generating code.
The problem is that once I stop reviewing the code, I have very little visibility into what the application is actually doing.
Without reviewing the code, all I had left was this:
I had to find another way.
I needed something more efficient than code review — something that would keep the computation from disappearing into arbitrary functions and modules.
So I started asking the agent to compose the application into meaningful, explicit steps.
For example:
The agent did still write the implementation behind those steps.
The code was still there.
But I no longer relied on the entire codebase to understand the computation.
Once the computation was expressed as meaningful steps, the app stopped looking like a black box.
I could inspect the result of each step.
Instead of only seeing the final output, I could inspect the intermediate data:
Now, when something goes wrong, I can ask a much more useful question:
That immediately narrows the problem.
The important interface is no longer just the source code. It is the data moving through the computation.
This is especially valuable for ML applications because the intermediate results often tell you more about correctness than the implementation does.
This also changed what I ask the agent to do.
I no longer need the agent to merely produce code that seems plausible.
I started asking it to structure the code so that its computation was inspectable.
The agent can handle implementation details within the steps.
I can reason about the application at the level of those steps.
And when I find a problem, I can point the agent to the problematic step instead of asking it to debug an entire application from scratch.
I still review code.
There are plenty of situations where implementation details matter: performance, concurrency, resource management, security, and many other concerns.
But I no longer think code should always be the first thing I review.
For many ML workflows, I first want to know:
This feels less like replacing code review and more like putting it in the right place.
The results tell me what is wrong.
The code tells me why it is wrong.
The agent simply made an existing problem much more visible.
ML applications are fundamentally computations over data, with some state management.
Yet our primary tools for understanding them are often source code, logs, and final outputs.
That can leave a large gap in the middle.
Consider:
The final output tells you what you got.
The source code tells you how it was implemented.
But the intermediate results tell you what actually happened along the way.
That middle ground is where I increasingly want to work.
I want ML applications where the important transformations are explicit and where the results at those boundaries can be inspected.
That is what I mean by making an ML application inspectable.
That idea led me to build a small framework around explicit, inspectable pipelines.
The goal is simple: compose applications as pipelines that you can:
By making the computation itself a first-class object, a pipeline provides:
I’ve put the code and examples in the repository so you can see how this works in practice:
GitHub — trained-by-humans/ml-pipes
The examples are probably the best way to see the idea in practice: they show the pipeline structure, the data flowing through it, and how the computation can be inspected.
In an AI-assisted development workflow, that becomes particularly important.
Coding agents have advanced quickly.
Generating useful code is becoming less of a bottleneck.
That doesn’t mean software development becomes trivial.
It means the bottleneck can move somewhere else.
For me, the more interesting question is now:
That question becomes especially important for ML applications.
An agent can generate a large amount of technically valid code very quickly. But correctness still depends on how data moves through the application and what the computation produces.
As code generation gets better, I don’t think the answer is simply to review more generated code.
We need better ways to observe and evaluate the computation itself.
At the beginning, this worked surprisingly well.
The changes were small, the codebase was familiar, and I could still keep the whole thing in my head.
Then the application grew.
A seemingly simple feature could now involve preprocessing, model inference, postprocessing, and application logic.
The agent might touch several modules and add a few hundred lines of code in a single session.
My habit didn’t change.
I was still reviewing the code after every session.
And that became the problem.
The code review trap
When a coding agent changes a few lines of code, reviewing the diff is easy.
When it changes several hundred lines, it is still manageable.
Once you get to +1000 lines everything starts to fall apart…
You can read the code without really understanding whether the application is working properly.
At some point I realized that I had become the bottleneck.
I was trying to control the wrong thing
Looking back, I think I was trying to solve the wrong problem.
When an AI agent writes code, the obvious way to stay in control is to review the code.
So that’s what I did.
I would look at the diff and ask:
I was trying to control the means.
The code was just the mechanism by which the agent produced the application, so I treated the code as the thing I needed to control.
The code was never the end goal.
I should never have cared whether the agent used one helper function or three. I didn’t care whether a transformation lived in one module or another.
That distinction became increasingly important as agents got better at generating code.
The Problem Is the Black Box
The problem is that once I stop reviewing the code, I have very little visibility into what the application is actually doing.
Without reviewing the code, all I had left was this:
Code:
Input → [ ? ? ? ? ? ] → OutputI had to find another way.
I needed something more efficient than code review — something that would keep the computation from disappearing into arbitrary functions and modules.
So I started asking the agent to compose the application into meaningful, explicit steps.
For example:
Code:
Pipeline([
Preprocess(...),
Inference(...),
Postprocess(...),
ApplicationLogic(...),
])The agent did still write the implementation behind those steps.
The code was still there.
But I no longer relied on the entire codebase to understand the computation.
The Unit of Review Has Changed
Once the computation was expressed as meaningful steps, the app stopped looking like a black box.
I could inspect the result of each step.
Instead of only seeing the final output, I could inspect the intermediate data:
Code:
Input → Preprocess → Inference → Postprocess → Application Logic → Output
│ │ │ │ │
▼ ▼ ▼ ▼ ▼
data input result result stateNow, when something goes wrong, I can ask a much more useful question:
That immediately narrows the problem.
The important interface is no longer just the source code. It is the data moving through the computation.
This is especially valuable for ML applications because the intermediate results often tell you more about correctness than the implementation does.
This changed the way I use the agent
This also changed what I ask the agent to do.
I no longer need the agent to merely produce code that seems plausible.
I started asking it to structure the code so that its computation was inspectable.
The agent can handle implementation details within the steps.
I can reason about the application at the level of those steps.
And when I find a problem, I can point the agent to the problematic step instead of asking it to debug an entire application from scratch.
Code Review Still Matters
I still review code.
There are plenty of situations where implementation details matter: performance, concurrency, resource management, security, and many other concerns.
But I no longer think code should always be the first thing I review.
For many ML workflows, I first want to know:
This feels less like replacing code review and more like putting it in the right place.
The results tell me what is wrong.
The code tells me why it is wrong.
From code-centric to result-centric development
The agent simply made an existing problem much more visible.
ML applications are fundamentally computations over data, with some state management.
Yet our primary tools for understanding them are often source code, logs, and final outputs.
That can leave a large gap in the middle.
Consider:
Code:
Input → Preprocess → Inference → Postprocess → Application Logic → OutputThe final output tells you what you got.
The source code tells you how it was implemented.
But the intermediate results tell you what actually happened along the way.
That middle ground is where I increasingly want to work.
I want ML applications where the important transformations are explicit and where the results at those boundaries can be inspected.
That is what I mean by making an ML application inspectable.
Putting the idea to practice
That idea led me to build a small framework around explicit, inspectable pipelines.
The goal is simple: compose applications as pipelines that you can:
By making the computation itself a first-class object, a pipeline provides:
- Gives the agent a meaningful structure in which to compose the application.
- Gives the developer meaningful boundaries at which to inspect the results.
I’ve put the code and examples in the repository so you can see how this works in practice:
GitHub — trained-by-humans/ml-pipes
The examples are probably the best way to see the idea in practice: they show the pipeline structure, the data flowing through it, and how the computation can be inspected.
In an AI-assisted development workflow, that becomes particularly important.
The next problem isn’t code generation
Coding agents have advanced quickly.
Generating useful code is becoming less of a bottleneck.
That doesn’t mean software development becomes trivial.
It means the bottleneck can move somewhere else.
For me, the more interesting question is now:
That question becomes especially important for ML applications.
An agent can generate a large amount of technically valid code very quickly. But correctness still depends on how data moves through the application and what the computation produces.
As code generation gets better, I don’t think the answer is simply to review more generated code.
We need better ways to observe and evaluate the computation itself.