H
For a long time I thought the hard part of a data project was getting the right answer. This one taught me that getting the answer was only half the job. The harder part was being able to explain where it came from.
It started with a pretty ordinary problem. A small business has a fixed marketing budget and several channels competing for it. Someone has to decide how to split the money. Not rank the channels. Actually decide. How much goes here. How much goes there. Whether one channel gets very little, or nothing at all.
At first I thought this would be a scoring problem. I would calculate which channels performed best, rank them, and allocate more money to the stronger ones. That version looked neat. It was also almost useless.
The problem is that budget allocation is not really a ranking problem. Every pound given to one channel is a pound that cannot go somewhere else. Then the real-world rules start appearing. Keep at least some spend on TikTok. Do not put too much of the total budget into one platform. Make sure a particular business objective receives a minimum amount. Leave part of the budget in reserve.
Once those rules appear, a list saying Facebook is first and Google is second does not tell you what to do. The decision is connected. So I stopped treating the channels independently and rebuilt the problem as a constrained optimisation problem. That changed everything.
The basic structure became something like this:
I built the optimisation layer in Python using PuLP and the CBC solver. Each platform and objective becomes part of one allocation problem. The model is trying to maximise the value produced by the available budget while still respecting the rules the user has given it.
In simplified form, the logic is:
The actual model also uses marginal yield brackets so that it does not assume the next pound spent is always worth exactly as much as the previous one. But the important change was simpler than that. The rules were no longer something applied after the recommendation. They were part of the decision itself.
Once I had the optimisation working, I expected the useful output to be the allocation. It turned out the solver was giving me something else that was much more interesting. It could tell me which constraints were actually shaping the result.
A minimum spend rule can exist without affecting the final answer. If the optimiser would have allocated more than that amount anyway, the rule changes nothing. But another constraint may sit directly against the optimum. Change that rule and the recommendation changes too. That is a binding constraint.
And because I kept the core problem as a continuous Linear Program, I could also read its dual value, usually called a shadow price. A shadow price gives the marginal value of relaxing a constraint. Put less formally, it answers a question I think managers ask all the time without having a number for it: What is this rule costing me?
That became much more useful than simply saying a constraint had been applied.
At one point there was a tempting way to stop the optimiser from concentrating too much money on one platform. Add an on/off decision. A channel is active or inactive. The model can then decide which channels should receive money. That pushes the formulation towards mixed-integer optimisation.
There is nothing wrong with that approach when the real decision is genuinely discrete. If a campaign has a fixed activation cost or a channel really must be either used or not used, an integer variable makes sense. But that was not the decision I was trying to model. More importantly for this project, I would lose the clean dual interpretation I was using to explain the effect of the constraints.
I realised I was about to make the optimiser more complicated while removing one of the most useful things it could tell the user. So I kept the allocation continuous. That decision ended up shaping the whole tool.
The moment I remember most was not a complicated test. I gave the model a £20,000 budget and added a minimum-spend rule to a weaker channel. It was the kind of rule I could easily imagine setting myself. Keep something there. Stay diversified. Do not put everything into the current winner.
The optimiser followed the rule. But the constraint report showed a negative shadow price next to it. The number was small, but what it meant was not. My own rule was reducing the objective.
The software was effectively saying: You can have this extra diversification if you want it. Here is what you are giving up to get it.
That was different from an optimiser telling me that my rule was wrong. It was not overruling me. It was making the trade-off visible before I made the decision. That is when the project stopped being mainly about optimisation for me. It became a project about how software should disagree with its user.
Shadow prices are useful if you already understand what they mean. Most people allocating a marketing budget do not want a lesson in Linear Programming before they can use a recommendation. So the final stage of the system became a separate rule-based interpretation layer.
After the solver finishes, the tool looks at the allocation and asks different questions. Is the budget heavily concentrated? Is the recommendation stable when the assumptions change? Which constraints are binding? What are their shadow prices? Would a more diversified plan look very different?
The system keeps the optimum as a performance-first plan, which I call Plan A. It can then produce a risk-managed Plan B by limiting excessive concentration and redistributing the released budget across the remaining options using their observed productivity.
Plan B is not another optimiser pretending to know the user's risk preference. It is a comparison. The interesting output is the distance between the two. If the diversified plan gives up almost nothing, that is useful information. If diversification has a large efficiency cost, that is useful information too.
Again, I am not trying to make the decision disappear. I am trying to make the decision easier to inspect.
This became important as the project grew. The model uses observed productivity ratios from historical data. It can run different planning scenarios and forecast the corresponding KPI outcomes. It does not prove that increasing spend on a channel will causally create the same return in the future.
That distinction matters. It would have been very easy to make the tool sound more intelligent by hiding that limitation. I think doing the opposite makes it more useful. A decision-support system should expose where its confidence ends. Otherwise explainability becomes another interface feature rather than something real.
I eventually released the project as an open-source Python tool called CLARO, short for Constrained Linear Allocation and Resource Optimiser. But the part of it I care about most is not the optimiser. Optimisation itself is not new. The interesting question for me became what happens immediately after the optimisation finishes.
A system can return:
and technically have done its job. But the person receiving those numbers still has several questions. Why this allocation? Which of my rules changed it? What happens if I relax one? How sensitive is this recommendation? What would I sacrifice if I wanted something safer?
If the software has information that can answer those questions and chooses not to show it, I think it is throwing away part of the result. That was probably the biggest lesson from building. I started by trying to make a better allocation. I ended up caring much more about making the allocation accountable.
More decisions are being handed to automated systems. Budgets get allocated, applications get ranked, tasks get prioritised and recommendations arrive faster than ever. The systems are getting very good at producing answers. That does not automatically make them good at supporting decisions.
For me, the standard has changed. When software tells me what to do, I want to be able to ask what shaped the answer, what assumptions it depends on, and what my own rules cost me. Asking a system to show its work is not a sign that the user is not technical enough. I think it is the technical standard.
Getting the right answer matters. But an answer nobody can question is not much to build on.
It started with a pretty ordinary problem. A small business has a fixed marketing budget and several channels competing for it. Someone has to decide how to split the money. Not rank the channels. Actually decide. How much goes here. How much goes there. Whether one channel gets very little, or nothing at all.
At first I thought this would be a scoring problem. I would calculate which channels performed best, rank them, and allocate more money to the stronger ones. That version looked neat. It was also almost useless.
A ranked list is not a budget decision
The problem is that budget allocation is not really a ranking problem. Every pound given to one channel is a pound that cannot go somewhere else. Then the real-world rules start appearing. Keep at least some spend on TikTok. Do not put too much of the total budget into one platform. Make sure a particular business objective receives a minimum amount. Leave part of the budget in reserve.
Once those rules appear, a list saying Facebook is first and Google is second does not tell you what to do. The decision is connected. So I stopped treating the channels independently and rebuilt the problem as a constrained optimisation problem. That changed everything.
The basic structure became something like this:
Code:
historical performance
↓
productivity ratios
↓
business priorities
↓
budget + policy constraints
↓
linear programming solver
↓
recommended allocation
↓
KPI forecast
↓
interpretation of the resultI built the optimisation layer in Python using PuLP and the CBC solver. Each platform and objective becomes part of one allocation problem. The model is trying to maximise the value produced by the available budget while still respecting the rules the user has given it.
In simplified form, the logic is:
Code:
maximise:
allocation × observed productivity × business priority
subject to:
total allocation <= available budget
platform allocation >= required platform minimums
objective allocation >= required objective minimumsThe actual model also uses marginal yield brackets so that it does not assume the next pound spent is always worth exactly as much as the previous one. But the important change was simpler than that. The rules were no longer something applied after the recommendation. They were part of the decision itself.
Then I realised the solver knew something I was not showing
Once I had the optimisation working, I expected the useful output to be the allocation. It turned out the solver was giving me something else that was much more interesting. It could tell me which constraints were actually shaping the result.
A minimum spend rule can exist without affecting the final answer. If the optimiser would have allocated more than that amount anyway, the rule changes nothing. But another constraint may sit directly against the optimum. Change that rule and the recommendation changes too. That is a binding constraint.
And because I kept the core problem as a continuous Linear Program, I could also read its dual value, usually called a shadow price. A shadow price gives the marginal value of relaxing a constraint. Put less formally, it answers a question I think managers ask all the time without having a number for it: What is this rule costing me?
That became much more useful than simply saying a constraint had been applied.
The obvious technical shortcut would have removed that explanation
At one point there was a tempting way to stop the optimiser from concentrating too much money on one platform. Add an on/off decision. A channel is active or inactive. The model can then decide which channels should receive money. That pushes the formulation towards mixed-integer optimisation.
There is nothing wrong with that approach when the real decision is genuinely discrete. If a campaign has a fixed activation cost or a channel really must be either used or not used, an integer variable makes sense. But that was not the decision I was trying to model. More importantly for this project, I would lose the clean dual interpretation I was using to explain the effect of the constraints.
I realised I was about to make the optimiser more complicated while removing one of the most useful things it could tell the user. So I kept the allocation continuous. That decision ended up shaping the whole tool.
A £20,000 example changed how I thought about the project
The moment I remember most was not a complicated test. I gave the model a £20,000 budget and added a minimum-spend rule to a weaker channel. It was the kind of rule I could easily imagine setting myself. Keep something there. Stay diversified. Do not put everything into the current winner.
The optimiser followed the rule. But the constraint report showed a negative shadow price next to it. The number was small, but what it meant was not. My own rule was reducing the objective.
The software was effectively saying: You can have this extra diversification if you want it. Here is what you are giving up to get it.
That was different from an optimiser telling me that my rule was wrong. It was not overruling me. It was making the trade-off visible before I made the decision. That is when the project stopped being mainly about optimisation for me. It became a project about how software should disagree with its user.
I did not want the explanation to stop at shadow prices
Shadow prices are useful if you already understand what they mean. Most people allocating a marketing budget do not want a lesson in Linear Programming before they can use a recommendation. So the final stage of the system became a separate rule-based interpretation layer.
After the solver finishes, the tool looks at the allocation and asks different questions. Is the budget heavily concentrated? Is the recommendation stable when the assumptions change? Which constraints are binding? What are their shadow prices? Would a more diversified plan look very different?
The system keeps the optimum as a performance-first plan, which I call Plan A. It can then produce a risk-managed Plan B by limiting excessive concentration and redistributing the released budget across the remaining options using their observed productivity.
Plan B is not another optimiser pretending to know the user's risk preference. It is a comparison. The interesting output is the distance between the two. If the diversified plan gives up almost nothing, that is useful information. If diversification has a large efficiency cost, that is useful information too.
Again, I am not trying to make the decision disappear. I am trying to make the decision easier to inspect.
I also had to decide what the model should not claim
This became important as the project grew. The model uses observed productivity ratios from historical data. It can run different planning scenarios and forecast the corresponding KPI outcomes. It does not prove that increasing spend on a channel will causally create the same return in the future.
That distinction matters. It would have been very easy to make the tool sound more intelligent by hiding that limitation. I think doing the opposite makes it more useful. A decision-support system should expose where its confidence ends. Otherwise explainability becomes another interface feature rather than something real.
The answer was never the whole product
I eventually released the project as an open-source Python tool called CLARO, short for Constrained Linear Allocation and Resource Optimiser. But the part of it I care about most is not the optimiser. Optimisation itself is not new. The interesting question for me became what happens immediately after the optimisation finishes.
A system can return:
Code:
Google £8,400
Facebook £5,100
Instagram £3,700
TikTok £2,800and technically have done its job. But the person receiving those numbers still has several questions. Why this allocation? Which of my rules changed it? What happens if I relax one? How sensitive is this recommendation? What would I sacrifice if I wanted something safer?
If the software has information that can answer those questions and chooses not to show it, I think it is throwing away part of the result. That was probably the biggest lesson from building. I started by trying to make a better allocation. I ended up caring much more about making the allocation accountable.
More decisions are being handed to automated systems. Budgets get allocated, applications get ranked, tasks get prioritised and recommendations arrive faster than ever. The systems are getting very good at producing answers. That does not automatically make them good at supporting decisions.
For me, the standard has changed. When software tells me what to do, I want to be able to ask what shaped the answer, what assumptions it depends on, and what my own rules cost me. Asking a system to show its work is not a sign that the user is not technical enough. I think it is the technical standard.
Getting the right answer matters. But an answer nobody can question is not much to build on.