R
What 263 real-world signals, an 11.3% return, and one brutal drawdown taught me about the difference between building a machine and believing one.
1. When the Machine Started Making Money
For years, I had assumed that the difficult part would be building a machine-learning system capable of finding an edge in a betting market. It took a profitable live experiment to make me realize that I had underestimated the harder problem: deciding what evidence would justify believing that the edge was real.
On March 26, 2026, I started putting my tennis model through the test I had been postponing for a long time. Instead of running another historical simulation, I began recording the bets I actually placed while matches were being played. The system had to make its decision first. Reality was allowed to answer afterward.
Five months later, I had 263 ATP betting signals in the ledger. Of those, 113 had won and 150 had lost. I had staked 4,850 units and made a profit of 546.7 units, for a return on investment of 11.3%.
It was exactly the sort of number that invites a triumphant headline.
I Built an AI That Beats the Tennis Betting Market.
There was only one problem.
I didn't believe that headline.
This wasn't false modesty. I wanted the system to work. I had spent years building it, collecting data, engineering features, training models and trying to understand a very particular moment in a tennis match. If anything, I had every psychological incentive to believe the number.
That was precisely what worried me.
The project is called Tennis OS, and the idea behind it is surprisingly simple. In a best-of-three tennis match, there is a moment I find particularly interesting: one player has won the first set, the other has won the second, and the match is tied at 1–1.
The third set begins at 0–0, but the match does not really begin again. Two sets of information have accumulated. We have seen who is serving comfortably, who is struggling on return, who is winning more points, whether the pre-match favorite actually looks like the better player and, importantly, how the betting market has reacted to all of it.
The score resets. The information does not.
At that moment, the market and my system are effectively being asked the same question: who wins the deciding set?
The market answers through a price. Tennis OS answers through a probability.
For every match that reaches this state, the system takes a snapshot of what could genuinely have been known at that moment. It combines pre-match expectations, what happened during the first two sets and the live market price, then passes that information through two machine-learning models — a neural network and a random forest. The purpose is not to predict tomorrow's match or the winner of the tournament. It is much narrower: estimate the probability that each player wins Set 3.
That probability becomes interesting only when it is compared with the price.
Suppose the market offers decimal odds of 3.00 on a player, roughly implying a winning probability of 33% before accounting for the bookmaker's margin. Now suppose Tennis OS estimates that player's chance at 42%. The system is not claiming that the player will probably win; in fact, it still expects him to lose more often than not. It is claiming something subtler: the price may be wrong.
That distinction eventually changed the way I thought about the entire project.
After those first 263 signals, Tennis OS had been wrong about the outcome more often than it had been right. Its winning rate was only 43%. Yet it had made money because betting is not a contest to see who can predict the largest number of winners. It is a contest between probability and price.
A model can correctly predict winners 60% of the time and still be economically useless if the prices require it to be right 65% of the time. Another model can lose most of its bets and still make money if the winners occur often enough relative to the odds being offered.
So prediction wasn't really the product I had built.
The disagreement between the machine and the market was the product.
And that created a much more difficult problem. If my system repeatedly said the market was wrong, how could I tell whether it had discovered something the market was missing — or whether my machine was simply wrong in a more complicated way?
That question would eventually become much more important than the 11.3%.
2. The Trouble With Beautiful Backtests
Before there were real bets, there were years of historical ones.
Backtesting is one of the most seductive parts of building a system like this because it feels like science. You construct a hypothesis, run it against thousands of historical matches and measure what would have happened. If the result is poor, you investigate. Perhaps another feature would help. Perhaps the threshold is wrong. Perhaps a certain price range behaves differently.
Then you run the test again.
Sometimes the result improves.
So you investigate a little more.
This is perfectly normal research, right up until you notice an uncomfortable fact: every time you look at the historical result and use what you saw to make another decision, the past has taught you something.
Give me several thousand tennis matches, enough variables and enough freedom to experiment, and I can search for an extraordinary number of patterns. Perhaps underdogs behave differently after winning the second set. Perhaps favorites in a certain odds range are systematically overpriced. Perhaps the effect exists only on hard courts. Perhaps it disappears above a certain probability threshold.
The possibilities multiply quickly, particularly once machine learning enters the picture.
And if you ask historical data enough questions, eventually it will tell you something interesting.
The difficulty is knowing whether you discovered a property of tennis markets or simply discovered the particular past you happened to possess.
A conventional train/test split helps, but I gradually realized that even this doesn't solve the whole problem. Imagine that I preserve a test dataset and never train my model on it. I run the finished system against that dataset, inspect the results, notice that one subgroup performs badly, modify the rules and test again.
Technically, the model was never trained on the test data.
Intellectually, however, I was.
The test had taught me what to change.
This is where an apparently rigorous machine-learning project can become surprisingly slippery. A smooth historical equity curve may look convincing. A double-digit backtested ROI may look even better. Once those numbers appear, it becomes remarkably easy to construct a plausible story explaining why they exist.
Humans are very good at explanations, especially after we already know the answer.
I was no exception.
Over the years, Tennis OS accumulated its own little graveyard: hypotheses that looked promising and then collapsed, attractive subgroups whose profitability depended on a few unusual winners, filters that seemed sensible until they encountered different data. Some ideas survived. Many did not.
Eventually I began to appreciate the graveyard almost as much as the successes. A research process that is capable only of confirming its own ideas isn't really testing them.
But I still had a fundamental problem. However careful I became with historical data, I had already seen too much of it. I knew too much about the world in which I was trying to prove that my system worked.
What I needed was not another clever model.
I needed ignorance.
I needed matches whose outcomes I genuinely did not know, data that did not yet exist, and decisions that had to be made before reality revealed whether they were right.
There is a wonderfully inconvenient way to obtain such a dataset.
You wait.
So on March 26, I stopped asking only what Tennis OS would have done and started recording what I actually bet.
The future had one enormous methodological advantage over my backtest.
I couldn't optimize it.
Not yet.
3. Watching 65% of the Profit Disappear
For a while, reality was remarkably cooperative.
The ledger grew, the bets accumulated and the profit climbed. By June 24, Tennis OS was sitting at +476.5 units. After years of historical experiments, seeing a live record move in the same direction was exhilarating. The system was encountering matches it had never seen, making decisions before the outcome was known, and apparently continuing to find value.
I could feel my attitude changing.
Perhaps this really was working.
That thought was more dangerous than I understood at the time.
Once you begin to believe in something you have built, every new result arrives with emotional baggage. Wins feel like confirmation. Losses invite explanations. A player may have been injured. The market moved unusually quickly. The match was strange. Perhaps this tournament behaves differently. Perhaps that particular odds range should never have been included.
None of those explanations is necessarily foolish. Some may even be correct.
But there is always another explanation available after the event, and a sufficiently motivated researcher can become extraordinarily creative when protecting a hypothesis he wants to survive.
Then July arrived.
By July 22, the +476.5 units had fallen to +166.4. In less than a month, 310.1 units had disappeared — roughly 65% of the profit at the previous peak.
Watching that happen was probably more educational than watching the profit accumulate.
A historical drawdown is an object on a chart. You inspect it, measure it and perhaps add another statistic to a report. A live drawdown feels completely different because you are standing inside it without knowing where it ends.
And, more importantly, you can do something about it.
I remember looking at the deteriorating results and seeing no shortage of possible interventions. I could tighten an expected-value threshold, change an odds corridor, inspect the losing matches for a common characteristic, remove a weak subgroup or introduce another defensive filter. With enough analysis, I could almost certainly have found a plausible reason why at least some of those losses should not happen again.
The engineer in me wanted to diagnose the problem.
The researcher in me was beginning to understand that diagnosing it too quickly could destroy the experiment.
Suppose I examined the July losses, discovered that a particular type of signal had performed badly, removed that signal type and then recalculated the historical performance. The new system might genuinely be better. But July could no longer be evidence that it was better. I would have used July to design it.
I would, in effect, have shown the system the exam questions after seeing where it had made mistakes.
That realization forced me to separate two activities that had previously been much too easy to mix together: production and research.
Production meant the system whose decisions were currently being judged. Research could continue beside it. I could investigate the losses, develop new features, train new models and test new hypotheses. What research could not do was travel backward in time and quietly repair the record of the system already under evaluation.
The rule that emerged was almost embarrassingly simple:
Never use new information to make an old decision look better.
So the July losses stayed.
No threshold was moved around them while the drawdown remained open. No losing subgroup was quietly redefined out of the experiment. The ugly part of the equity curve remained exactly where reality had put it.
And something else happened during this period that made me reconsider what I meant by evidence.
When the real-money experiment began in March, my records captured the bets I actually placed: player, stake, price and eventual result. What they did not yet capture with the same rigor was the complete machine-generated decision trail behind every bet. Fully automated, timestamped decision logging arrived only on July 12.
That distinction matters. Of the 263 signals behind the headline result, 54% were generated before the stronger logging system existed — and those earlier signals account for 74% of the total profit.
It would be convenient to describe all 263 observations as if they had identical evidential quality.
They don't.
The bets were real. The early audit trail was less complete.
Initially, I found that frustrating. Eventually I came to see it as part of the result. If the purpose of the experiment was to determine whether I could trust the model, then weaknesses in the experiment itself had to be treated as evidence too.
I couldn't improve the past by describing it more confidently than it deserved.
4. The Experiment Was Testing Me
Tennis OS eventually recovered.
By August 28, cumulative profit had climbed to +546.7 units, above the previous peak, and the overall return stood at 11.3%.
The easy interpretation was that the model had survived its test.
I wasn't prepared to make that claim.
Two hundred and sixty-three bets are enough to become interesting. They are not enough to make variance disappear. Nor does a profitable sequence tell me why the profit occurred. Perhaps the system really is identifying probabilities the market systematically misprices. Perhaps the effect belongs mainly to one narrow class of matches. Perhaps a small number of large winners are carrying more of the result than I would like. Perhaps market conditions changed during the experiment.
There is another possibility that every model builder has to keep alive:
Perhaps I have simply been lucky.
The irritating thing about luck is that it doesn't always look like luck while you are experiencing it.
This was when I began to realize that the hardest engineering problem in Tennis OS might not be the neural network or the random forest. It might be designing a process capable of constraining the person who built them.
Me.
I know which outcome I want. I know how satisfying it would be to conclude that several years of work had uncovered a persistent inefficiency in a betting market. I also know enough about the system to produce intelligent explanations for almost any individual failure.
That makes me part of the threat model.
Once I accepted that, things that had previously felt like administrative overhead — version boundaries, timestamped logs, frozen decision rules, research separation — started to look different. They were not bureaucracy surrounding the model.
They were part of the model's credibility.
The system itself did not remain literally frozen for five months. Nor should it have. The predictive model was deliberately retrained in August through a documented process, while the established betting decision logic was preserved across the change. New ideas could also continue to be developed, but increasingly they entered what I call shadow mode: they could observe matches, calculate what they would have done and accumulate evidence, but they could not change the live production decision.
I have come to like this arrangement.
It allows curiosity without giving curiosity permission to rewrite history.
A promising new idea does not have to sit in a drawer for six months. It can run alongside production immediately. But if it looks brilliant after twenty matches, that does not entitle it to retroactively claim that it would have avoided last month's losses.
It has to earn its future.
This also changed my relationship with losing bets. Earlier in the project, a loss naturally triggered the question: What should I change?
Now there is another question that has to come first:
Am I looking at new information — or am I looking at noise that happens to hurt?
Sometimes the right response to new evidence is to change the system.
Sometimes the right response is to do absolutely nothing.
Knowing the difference requires something machine-learning projects don't often advertise in their architecture diagrams: patience.
The model was no longer allowed to learn opportunistically from every loss.
I was.
5. How Much Evidence Is Enough?
That brings me back to the 11.3%.
It is a real result. It came from real bets placed on future outcomes, not from a portfolio reconstructed afterward. The system suffered a substantial drawdown and recovered. Its established decision logic was not conveniently rewritten around that drawdown.
All of that makes the result more interesting to me than any historical backtest I ever produced.
But interesting is not the same as proven.
So the next stage of Tennis OS is deliberately less glamorous than the headline number suggests. Instead of asking how to squeeze more profit from the current strategy, I am trying to find better ways to make it fail.
One of those tests concerns calibration. If the system repeatedly assigns players a 60% probability of winning, do approximately 60% of them actually win? A model can generate a profitable run while still producing probabilities that are systematically too confident or too conservative.
Another concerns the disagreement that started the whole project: machine versus market. When Tennis OS and the live market arrive at materially different probabilities after two sets, is that disagreement actually informative? Does the model know something the market is underweighting, does the market know something the model cannot see, or are both stories simply being imposed on random variation?
There is even disagreement inside Tennis OS itself. Its neural network and random forest do not always see a match the same way. Perhaps those disagreements contain useful information about uncertainty. Perhaps they are warning signals. Perhaps they mean nothing at all.
The important point is that these questions are being investigated without being allowed to repair the record that generated them.
If one of them survives, it can become part of a future system.
If it fails, it joins the graveyard.
That is no longer something I regard as wasted work.
The purpose of the experiment is not to protect Tennis OS from being wrong. It is to make being wrong visible as early and as honestly as possible.
Somewhere during these five months, I found a phrase that captures the relationship I now want to have with the project:
FIGHT FOR THE MISSION. BUT NEVER FIGHT THE EVIDENCE.
The first half matters to me. I want this thing to work. Building it, testing it and trying to understand why it behaves as it does has become a serious intellectual project.
But the second half matters more.
If another 500 prospective signals turn +11.3% into zero, then zero is not a betrayal of the project. It is the result.
If calibration fails, it fails.
If the apparent market inefficiency disappears under stronger evidence, then the correct response is not to search frantically for the subset in which it survives. The correct response is to learn something and move on.
The mission can survive a dead hypothesis.
In fact, it has to.
When I started building Tennis OS, I thought the central question was whether machine learning could predict the deciding set of a tennis match.
Later, I thought the more interesting question was whether a machine could find probabilities that the live market had priced incorrectly.
After five months of putting real decisions in front of reality, I think both questions were incomplete.
The harder question is this:
How much evidence should it take before I allow myself to believe a machine that is telling me exactly what I hoped it would say?
I still don't know.
And for the first time in this project, not knowing feels less like a problem to solve than the reason to keep going.
1. When the Machine Started Making Money
For years, I had assumed that the difficult part would be building a machine-learning system capable of finding an edge in a betting market. It took a profitable live experiment to make me realize that I had underestimated the harder problem: deciding what evidence would justify believing that the edge was real.
On March 26, 2026, I started putting my tennis model through the test I had been postponing for a long time. Instead of running another historical simulation, I began recording the bets I actually placed while matches were being played. The system had to make its decision first. Reality was allowed to answer afterward.
Five months later, I had 263 ATP betting signals in the ledger. Of those, 113 had won and 150 had lost. I had staked 4,850 units and made a profit of 546.7 units, for a return on investment of 11.3%.
It was exactly the sort of number that invites a triumphant headline.
I Built an AI That Beats the Tennis Betting Market.
There was only one problem.
I didn't believe that headline.
This wasn't false modesty. I wanted the system to work. I had spent years building it, collecting data, engineering features, training models and trying to understand a very particular moment in a tennis match. If anything, I had every psychological incentive to believe the number.
That was precisely what worried me.
The project is called Tennis OS, and the idea behind it is surprisingly simple. In a best-of-three tennis match, there is a moment I find particularly interesting: one player has won the first set, the other has won the second, and the match is tied at 1–1.
The third set begins at 0–0, but the match does not really begin again. Two sets of information have accumulated. We have seen who is serving comfortably, who is struggling on return, who is winning more points, whether the pre-match favorite actually looks like the better player and, importantly, how the betting market has reacted to all of it.
The score resets. The information does not.
At that moment, the market and my system are effectively being asked the same question: who wins the deciding set?
The market answers through a price. Tennis OS answers through a probability.
For every match that reaches this state, the system takes a snapshot of what could genuinely have been known at that moment. It combines pre-match expectations, what happened during the first two sets and the live market price, then passes that information through two machine-learning models — a neural network and a random forest. The purpose is not to predict tomorrow's match or the winner of the tournament. It is much narrower: estimate the probability that each player wins Set 3.
That probability becomes interesting only when it is compared with the price.
Suppose the market offers decimal odds of 3.00 on a player, roughly implying a winning probability of 33% before accounting for the bookmaker's margin. Now suppose Tennis OS estimates that player's chance at 42%. The system is not claiming that the player will probably win; in fact, it still expects him to lose more often than not. It is claiming something subtler: the price may be wrong.
That distinction eventually changed the way I thought about the entire project.
After those first 263 signals, Tennis OS had been wrong about the outcome more often than it had been right. Its winning rate was only 43%. Yet it had made money because betting is not a contest to see who can predict the largest number of winners. It is a contest between probability and price.
A model can correctly predict winners 60% of the time and still be economically useless if the prices require it to be right 65% of the time. Another model can lose most of its bets and still make money if the winners occur often enough relative to the odds being offered.
So prediction wasn't really the product I had built.
The disagreement between the machine and the market was the product.
And that created a much more difficult problem. If my system repeatedly said the market was wrong, how could I tell whether it had discovered something the market was missing — or whether my machine was simply wrong in a more complicated way?
That question would eventually become much more important than the 11.3%.
2. The Trouble With Beautiful Backtests
Before there were real bets, there were years of historical ones.
Backtesting is one of the most seductive parts of building a system like this because it feels like science. You construct a hypothesis, run it against thousands of historical matches and measure what would have happened. If the result is poor, you investigate. Perhaps another feature would help. Perhaps the threshold is wrong. Perhaps a certain price range behaves differently.
Then you run the test again.
Sometimes the result improves.
So you investigate a little more.
This is perfectly normal research, right up until you notice an uncomfortable fact: every time you look at the historical result and use what you saw to make another decision, the past has taught you something.
Give me several thousand tennis matches, enough variables and enough freedom to experiment, and I can search for an extraordinary number of patterns. Perhaps underdogs behave differently after winning the second set. Perhaps favorites in a certain odds range are systematically overpriced. Perhaps the effect exists only on hard courts. Perhaps it disappears above a certain probability threshold.
The possibilities multiply quickly, particularly once machine learning enters the picture.
And if you ask historical data enough questions, eventually it will tell you something interesting.
The difficulty is knowing whether you discovered a property of tennis markets or simply discovered the particular past you happened to possess.
A conventional train/test split helps, but I gradually realized that even this doesn't solve the whole problem. Imagine that I preserve a test dataset and never train my model on it. I run the finished system against that dataset, inspect the results, notice that one subgroup performs badly, modify the rules and test again.
Technically, the model was never trained on the test data.
Intellectually, however, I was.
The test had taught me what to change.
This is where an apparently rigorous machine-learning project can become surprisingly slippery. A smooth historical equity curve may look convincing. A double-digit backtested ROI may look even better. Once those numbers appear, it becomes remarkably easy to construct a plausible story explaining why they exist.
Humans are very good at explanations, especially after we already know the answer.
I was no exception.
Over the years, Tennis OS accumulated its own little graveyard: hypotheses that looked promising and then collapsed, attractive subgroups whose profitability depended on a few unusual winners, filters that seemed sensible until they encountered different data. Some ideas survived. Many did not.
Eventually I began to appreciate the graveyard almost as much as the successes. A research process that is capable only of confirming its own ideas isn't really testing them.
But I still had a fundamental problem. However careful I became with historical data, I had already seen too much of it. I knew too much about the world in which I was trying to prove that my system worked.
What I needed was not another clever model.
I needed ignorance.
I needed matches whose outcomes I genuinely did not know, data that did not yet exist, and decisions that had to be made before reality revealed whether they were right.
There is a wonderfully inconvenient way to obtain such a dataset.
You wait.
So on March 26, I stopped asking only what Tennis OS would have done and started recording what I actually bet.
The future had one enormous methodological advantage over my backtest.
I couldn't optimize it.
Not yet.
3. Watching 65% of the Profit Disappear
For a while, reality was remarkably cooperative.
The ledger grew, the bets accumulated and the profit climbed. By June 24, Tennis OS was sitting at +476.5 units. After years of historical experiments, seeing a live record move in the same direction was exhilarating. The system was encountering matches it had never seen, making decisions before the outcome was known, and apparently continuing to find value.
I could feel my attitude changing.
Perhaps this really was working.
That thought was more dangerous than I understood at the time.
Once you begin to believe in something you have built, every new result arrives with emotional baggage. Wins feel like confirmation. Losses invite explanations. A player may have been injured. The market moved unusually quickly. The match was strange. Perhaps this tournament behaves differently. Perhaps that particular odds range should never have been included.
None of those explanations is necessarily foolish. Some may even be correct.
But there is always another explanation available after the event, and a sufficiently motivated researcher can become extraordinarily creative when protecting a hypothesis he wants to survive.
Then July arrived.
By July 22, the +476.5 units had fallen to +166.4. In less than a month, 310.1 units had disappeared — roughly 65% of the profit at the previous peak.
Watching that happen was probably more educational than watching the profit accumulate.
A historical drawdown is an object on a chart. You inspect it, measure it and perhaps add another statistic to a report. A live drawdown feels completely different because you are standing inside it without knowing where it ends.
And, more importantly, you can do something about it.
I remember looking at the deteriorating results and seeing no shortage of possible interventions. I could tighten an expected-value threshold, change an odds corridor, inspect the losing matches for a common characteristic, remove a weak subgroup or introduce another defensive filter. With enough analysis, I could almost certainly have found a plausible reason why at least some of those losses should not happen again.
The engineer in me wanted to diagnose the problem.
The researcher in me was beginning to understand that diagnosing it too quickly could destroy the experiment.
Suppose I examined the July losses, discovered that a particular type of signal had performed badly, removed that signal type and then recalculated the historical performance. The new system might genuinely be better. But July could no longer be evidence that it was better. I would have used July to design it.
I would, in effect, have shown the system the exam questions after seeing where it had made mistakes.
That realization forced me to separate two activities that had previously been much too easy to mix together: production and research.
Production meant the system whose decisions were currently being judged. Research could continue beside it. I could investigate the losses, develop new features, train new models and test new hypotheses. What research could not do was travel backward in time and quietly repair the record of the system already under evaluation.
The rule that emerged was almost embarrassingly simple:
Never use new information to make an old decision look better.
So the July losses stayed.
No threshold was moved around them while the drawdown remained open. No losing subgroup was quietly redefined out of the experiment. The ugly part of the equity curve remained exactly where reality had put it.
And something else happened during this period that made me reconsider what I meant by evidence.
When the real-money experiment began in March, my records captured the bets I actually placed: player, stake, price and eventual result. What they did not yet capture with the same rigor was the complete machine-generated decision trail behind every bet. Fully automated, timestamped decision logging arrived only on July 12.
That distinction matters. Of the 263 signals behind the headline result, 54% were generated before the stronger logging system existed — and those earlier signals account for 74% of the total profit.
It would be convenient to describe all 263 observations as if they had identical evidential quality.
They don't.
The bets were real. The early audit trail was less complete.
Initially, I found that frustrating. Eventually I came to see it as part of the result. If the purpose of the experiment was to determine whether I could trust the model, then weaknesses in the experiment itself had to be treated as evidence too.
I couldn't improve the past by describing it more confidently than it deserved.
4. The Experiment Was Testing Me
Tennis OS eventually recovered.
By August 28, cumulative profit had climbed to +546.7 units, above the previous peak, and the overall return stood at 11.3%.
The easy interpretation was that the model had survived its test.
I wasn't prepared to make that claim.
Two hundred and sixty-three bets are enough to become interesting. They are not enough to make variance disappear. Nor does a profitable sequence tell me why the profit occurred. Perhaps the system really is identifying probabilities the market systematically misprices. Perhaps the effect belongs mainly to one narrow class of matches. Perhaps a small number of large winners are carrying more of the result than I would like. Perhaps market conditions changed during the experiment.
There is another possibility that every model builder has to keep alive:
Perhaps I have simply been lucky.
The irritating thing about luck is that it doesn't always look like luck while you are experiencing it.
This was when I began to realize that the hardest engineering problem in Tennis OS might not be the neural network or the random forest. It might be designing a process capable of constraining the person who built them.
Me.
I know which outcome I want. I know how satisfying it would be to conclude that several years of work had uncovered a persistent inefficiency in a betting market. I also know enough about the system to produce intelligent explanations for almost any individual failure.
That makes me part of the threat model.
Once I accepted that, things that had previously felt like administrative overhead — version boundaries, timestamped logs, frozen decision rules, research separation — started to look different. They were not bureaucracy surrounding the model.
They were part of the model's credibility.
The system itself did not remain literally frozen for five months. Nor should it have. The predictive model was deliberately retrained in August through a documented process, while the established betting decision logic was preserved across the change. New ideas could also continue to be developed, but increasingly they entered what I call shadow mode: they could observe matches, calculate what they would have done and accumulate evidence, but they could not change the live production decision.
I have come to like this arrangement.
It allows curiosity without giving curiosity permission to rewrite history.
A promising new idea does not have to sit in a drawer for six months. It can run alongside production immediately. But if it looks brilliant after twenty matches, that does not entitle it to retroactively claim that it would have avoided last month's losses.
It has to earn its future.
This also changed my relationship with losing bets. Earlier in the project, a loss naturally triggered the question: What should I change?
Now there is another question that has to come first:
Am I looking at new information — or am I looking at noise that happens to hurt?
Sometimes the right response to new evidence is to change the system.
Sometimes the right response is to do absolutely nothing.
Knowing the difference requires something machine-learning projects don't often advertise in their architecture diagrams: patience.
The model was no longer allowed to learn opportunistically from every loss.
I was.
5. How Much Evidence Is Enough?
That brings me back to the 11.3%.
It is a real result. It came from real bets placed on future outcomes, not from a portfolio reconstructed afterward. The system suffered a substantial drawdown and recovered. Its established decision logic was not conveniently rewritten around that drawdown.
All of that makes the result more interesting to me than any historical backtest I ever produced.
But interesting is not the same as proven.
So the next stage of Tennis OS is deliberately less glamorous than the headline number suggests. Instead of asking how to squeeze more profit from the current strategy, I am trying to find better ways to make it fail.
One of those tests concerns calibration. If the system repeatedly assigns players a 60% probability of winning, do approximately 60% of them actually win? A model can generate a profitable run while still producing probabilities that are systematically too confident or too conservative.
Another concerns the disagreement that started the whole project: machine versus market. When Tennis OS and the live market arrive at materially different probabilities after two sets, is that disagreement actually informative? Does the model know something the market is underweighting, does the market know something the model cannot see, or are both stories simply being imposed on random variation?
There is even disagreement inside Tennis OS itself. Its neural network and random forest do not always see a match the same way. Perhaps those disagreements contain useful information about uncertainty. Perhaps they are warning signals. Perhaps they mean nothing at all.
The important point is that these questions are being investigated without being allowed to repair the record that generated them.
If one of them survives, it can become part of a future system.
If it fails, it joins the graveyard.
That is no longer something I regard as wasted work.
The purpose of the experiment is not to protect Tennis OS from being wrong. It is to make being wrong visible as early and as honestly as possible.
Somewhere during these five months, I found a phrase that captures the relationship I now want to have with the project:
FIGHT FOR THE MISSION. BUT NEVER FIGHT THE EVIDENCE.
The first half matters to me. I want this thing to work. Building it, testing it and trying to understand why it behaves as it does has become a serious intellectual project.
But the second half matters more.
If another 500 prospective signals turn +11.3% into zero, then zero is not a betrayal of the project. It is the result.
If calibration fails, it fails.
If the apparent market inefficiency disappears under stronger evidence, then the correct response is not to search frantically for the subset in which it survives. The correct response is to learn something and move on.
The mission can survive a dead hypothesis.
In fact, it has to.
When I started building Tennis OS, I thought the central question was whether machine learning could predict the deciding set of a tennis match.
Later, I thought the more interesting question was whether a machine could find probabilities that the live market had priced incorrectly.
After five months of putting real decisions in front of reality, I think both questions were incomplete.
The harder question is this:
How much evidence should it take before I allow myself to believe a machine that is telling me exactly what I hoped it would say?
I still don't know.
And for the first time in this project, not knowing feels less like a problem to solve than the reason to keep going.