From Bug Report to Production: How I Fix Bugs with AI

[inDrive.Tech]

Hi! I’m Tim, QA Engineer at inDrive. Some time ago, when I was updating tests for new endpoints, one of them behaved strangely. The service rejected a request, but on the next attempt it said the action had already been completed. In fact, nothing had happened.

The problem was with an idempotency key: the service reserved it before validating the request. If validation failed, the key remained reserved anyway. I prepared a fix, went through code review, and shipped the change. After the release, duplicate-key errors on the affected endpoints dropped by about 30%.

Previously, my work on a bug usually ended with a detailed report and retesting someone else's fix. Now, with Claude, I can go through the whole process by myself: understand unfamiliar code, check a hypothesis against logs, prepare a patch and tests, submit the review change, and take it through to release. Today, I independently ship an average of three to five small fixes and improvements a week.

I distinguish three levels of this approach based on my experience at inDrive, which I'll describe below. It's important to note that this is not a mandatory career path for every QA engineer, nor does it mean handing responsibility over to an agent. In this approach, AI helps investigate systems faster, but the engineer still checks the facts, assesses the risks, and takes responsibility for the result.

Level 1. A Bug Report That Lets a Developer Start Fixing Right Away​

A typical bug report answers the question, “What is broken?” But the developer still has to find the cause, understand the related parts of the code, and choose a solution. To save developers time, I started adding the results of that investigation to my bug reports.

First, I ask an AI agent to trace the flow of data and find a possible cause. Then I check every conclusion in the repository: whether that piece of code actually runs in the scenario, how similar cases are handled elsewhere, and what other parts of the system the patch might affect. If a hypothesis isn't confirmed, I don't present it as a fact.

As a result, each ticket includes six sections:

1. The problem and the conditions that trigger it;
2. A specific proposed fix;
3. Acceptance criteria and a regression test;
4. Risks and solutions that look plausible at first but are actually wrong;
5. Impact on users;
6. References to files, lines of code, and related tickets.

This kind of report doesn't guarantee that the proposed solution is correct. But it gives the developer a verified starting point instead of a task that still needs to be investigated from scratch.

Example: Passenger Requests That Never Reached the Driver​

The inDrive order service has separate ride options for an assistive device, a service animal, and a wheelchair-accessible car. When the service built its response, all three values were mistakenly read from the same field: assistiveDevice.

As a result, the driver might not see a request for an accessible car if the passenger hadn't selected an assistive device. And when the device option was selected, its description appeared under the other options as well. No tests covered this case.

Before the fix:

response.assistiveDevice = describe(order.assistiveDevice)
response.serviceAnimals = describe(order.assistiveDevice)
response.wheelchairCar = describe(order.assistiveDevice)

After the fix:

response.assistiveDevice = describe(order.assistiveDevice)
response.serviceAnimals = describe(order.serviceAnimals)
response.wheelchairCar = describe(order.wheelchairCar)

The cause was two copy-and-paste mistakes. In the ticket, I pointed to the exact location and specified the necessary changes and checks. The first available developer quickly made the fix, and the ticket was closed a couple of days later.

What matters here is not only the size of the patch but also the consequences of the bug: because of two incorrect field references, passengers who needed special ride arrangements might not get the service they required. That’s frustrating, and it had to be fixed fast.

What to Try in Your Team​

Take a bug you can already reproduce:

1. Ask the agent to trace the flow of data from the incoming request to the system's response.
2. Ask it to separate conclusions confirmed by the code from hypotheses, and to provide references to specific files and lines.
3. Check those findings yourself.
4. Then add the smallest possible fix, potential side effects, and a test that should stop failing after the patch to the ticket.

In another investigation, the agent helped find an endless recursive call in an error handler: for an unknown error, the function called itself with the same arguments. This could cause a stack overflow and crash the process. I explicitly noted in the ticket that the defect had been found by reading the code, not reproduced on a running instance. It's essential to preserve that distinction.

Level 2. Prioritize Bugs Using Production Data​

Even when you've found the cause, a question remains: how serious is the problem? How many requests or inDrive users does it affect? Can you confirm it not just from the code but also from how the system behaves?

For this, I analyze logs and metrics. The agent helps identify which events the relevant code writes, build queries, collect results across clusters, and group errors by cause. Where access to logging and metrics tools is available, MCP connectors are a convenient option.

But the results still need to be checked: the wrong filter or denominator can easily turn a convincing number into a false conclusion.

Example: Nearly 1.8 Million False Server Errors in One Week​

While analyzing logs from the inDrive order service, I found a large number of 500 errors occurring during calls to the bidding service. To understand the causes, I used AI to build log queries and collect one week of statistics across 13 clusters.

In total, I found about 1.86 million such errors. Preliminary analysis showed that roughly 96.6% of them—about 1.79 million—were caused by clients cancelling requests.

The cause was in how errors were handled between services.

When a client closed the connection during a call to the bidding service, that service returned gRPC Canceled. The order service converted it into a generic 500 server error and wrote it to the logs.

Yet other parts of the service already handled these situations correctly: if the client closed the connection, the server did not try to send a response or log an error.

I checked whether these cancellations were caused by slow responses from the bidding service. They weren't: calls completed in milliseconds. For example, the median duration of an unsuccessful bid retrieval call was 7.5 ms, compared with 5.6 ms for a successful one.

In other words, slow responses were not the problem. The client had already stopped waiting, while the server continued to count the cancelled request as its own failure.

As a result, monitoring showed errors where the server had not actually failed. Including related scenarios, client cancellations generated several million ERROR-level log entries in a week. This noise made it harder to distinguish real failures from expected system behavior.

Based on the investigation, I proposed changing the error handling:

1. If the client cancels a request during a call to the bidding service, don't return a 500 error or write an ERROR log entry.
2. If the cancellation originates in the bidding service itself while the client is still waiting for a response, keep treating it as a server error.
3. Check how the change affects event consumers, and notify the owners of dashboards and alerts about the expected drop in error counts.

According to a preliminary estimate, this change could eliminate about 96% of the 500 errors in question. That's the expected effect calculated from logs, not the result of a deployed fix. The findings also need to be reviewed with the bidding team.

This example shows why QA engineers should analyze production data. A high number of errors in monitoring doesn't necessarily mean that users experienced the same number of failures. Before assigning priority and proposing a fix, you need to understand what exactly the system measures and how it classifies errors.

Another Example: Errors When Changing a Ride's Status​

While analyzing another scenario, I found that about 3% of requests to the ride-status transition endpoint failed over the course of a day. The logs showed that 90% of those errors occurred when the driver requested a status the ride was already in. And 55% of the errors involved a transition to “completed” for a ride that was already completed.

Timing data and traces pointed to repeated taps, stale screen state, and bursts of requests from an older ride screen. The next day, I repeated the analysis and got the same 90%.

We split the work into two tasks:

1. Return success if the ride is already in the requested status.
2. Add logging where we still lacked data—on the passenger side.

How to Verify an Agent's Findings​

Before putting metrics in a ticket, I check the time period the data covers, which clusters are included, what counts as an error, and what denominator is used to calculate each percentage. Then I check whether a spike can be explained by one instance or a rare scenario and, where possible, repeat the analysis over a different period.

It's also important to distinguish three things:

1. An observation from the logs;
2. A hypothesis about the cause;
3. The expected effect of a fix.

They shouldn't be presented together as one proven fact.

Level 3. Prepare a Fix, Pass Code Review, and Ship It​

When the cause is clear, the risks are limited, and the change is small enough, a QA engineer can prepare the patch by himself instead of handing the task to a developer. Although AI helps write code and tests, this in no way means that a patch can be shipped without the usual engineering checks.

At inDrive, I started with simple changes reviewed by developers. Over time, I began taking on more substantial tasks. Now my process includes code analysis, testing, code review, the standard release process, and verifying the impact after release.

Example: An Idempotency Key Was Reserved Too Early​

Let's return to the bug from the beginning of the article. Some endpoints reserved the request key before validating its contents. If validation failed or subsequent processing ended unsuccessfully, the key stayed reserved. The user's next attempt received a response saying an operation had been repeated, even though it hadn't happened in the first place.

In simplified form, the original sequence looked like this:

reserve(key)          // the key is reserved before validation
validate(request)
process(request)

After the fix:

validate(request)
reserve(key)          // only a valid request reserves the key
try:
    process(request)
onFailure:
    release(key)      // the request can be retried after failure

For an operation that really has already completed, the conflict response remains. But the response now also includes an explicit reason that can be tracked in monitoring.

After the release, I checked the metrics. Duplicate-key errors on these endpoints were down by about 30%; the decrease held every day after the release and across every cluster with traffic. Median response time, the 99th percentile, and other errors showed no noticeable change. The additional load on the shared cache—about 0.01%—had been assessed before the merge and considered acceptable.

That result matters more than the fact that AI helped write a few lines of code: the fix reached production, and we could verify its expected effect.

A More Complex Example: A Bug in a Shared Authentication Library​

One service was crashing at startup with an unclear error. At first, it looked like a broken response parser. By tracing execution with AI, I followed the issue beyond our service into the shared authentication library. When the auth service returned a plain-text error page, the client tried to parse it as JSON and lost the actual response status code.

While investigating the source of that error further, I found a more serious problem: a background job that was supposed to refresh keys regularly ran only once and then stopped. After the next key rotation, services using an older version of the library could keep using stale keys, causing token validation to fail.

We reproduced the scenario. The fix was reviewed by the owners of the shared library, and then a new version was released. This case shows why it's important to investigate a root cause all the way through, including beyond the service where the error first appeared.

What I Don't Delegate to AI​

The agent is good at reading unfamiliar code, tracing connections between services, and drafting log queries, patches, and tests. But it can also confidently explain a cause that doesn't actually exist.

That's why I keep four decisions for myself:

1. What has been confirmed by evidence;
2. How broad the change should be;
3. Which tests are needed;
4. Whether the patch is ready to ship and what to check after release.

I also follow specific rules:

• In each ticket, I state what was reproduced and what was found only by reading code.
• If a conclusion turns out to be wrong, I add an explicit correction to the ticket history rather than silently replacing the text.
• Before merging, I measure the load on shared systems.
• I run automated tests in a test environment and separately note flaky tests and tests that were already failing.
• I don't turn a small patch into a large refactor.
• I send changes to shared code to the owners of dependent components.

These aren't special rules for AI. They're ordinary engineering discipline, and they become essential when a QA engineer at inDrive starts changing production code.

Where to Start​

1. You don't need a big task for your first experiment. Choose a bug you filed this week and try only Level 1: use an agent to trace the implementation flow, verify the likely cause, and add code references, a proposed fix, risks, and acceptance criteria to the bug report.
2. Once you can verify the findings, add Level 2: take one metric from logs or monitoring and use it to confirm the scope of the problem. Don't put a percentage in the ticket until you've checked exactly what it measures.
3. Move to Level 3 with a small, straightforward change—for example, removing a feature toggle that has been fully rolled out and is no longer needed. Prepare the patch and tests, pass code review by the code owners, and follow the usual release process. After release, make sure the system behaves as expected.

For me, the biggest change in QA work isn't that an agent can write code. I no longer have to stop at reporting a bug: I can find the cause, confirm it with data, prepare a fix, and—when the risk is acceptable—take it all the way to production. Responsibility for quality still stays with the engineer.

Acknowledgements​

My growth over the past few months has been possible thanks to the support of my colleagues: they review my code, ask questions, and help me grow and push me further. Thank you to the Driver Experience team and to all the colleagues at inDrive who share their practices and support this process.