S
I have spent a good chunk of the last few years building and defending medallion lakehouses. Bronze, silver, gold, the whole ritual. And honestly, it works. It is a genuinely good pattern for turning messy operational exhaust into something a BI team can trust. But somewhere around the third time I watched an AI agent try to "just query the warehouse" and quietly hallucinate a metric that did not exist, I started to suspect the lakehouse, as we currently build it, was not designed for what we are now asking it to serve: autonomous agents that plan, call tools, write back results, and sometimes act without anyone checking each step.
A traditional lakehouse assumes a human sits on the other end of every request. Someone opens a BI tool, someone schedules a dbt job, someone decides which report needs refreshing this morning. Even "real-time" lakehouses, the ones streaming through Kafka into Delta tables, still assume a person eventually interprets the output.
Here is roughly what that looks like in practice.
Notice the shape. It is linear. Sources flow one direction, get progressively cleaner, and land somewhere a human can look at them. Governance sits beside the pipeline, mostly as a catalog and an access policy, not as something actively consulted mid-flight. And critically, nothing in this diagram decides anything. It moves data. Decisions happen upstream, in someone's head, before the pipeline even runs.
That is fine when the consumer is a person with judgment and context. It gets shaky fast when the consumer is an agent that needs to decide, on its own, whether a number is trustworthy enough to act on.
I will admit I underestimated this at first. My instinct was: give the agent SQL access, point it at the gold layer, done. That sort of works for demos. It falls apart the moment two tables define "active customer" differently, which, in my experience, is basically every enterprise warehouse I have touched. An agent without a shared semantic layer will confidently average two incompatible definitions together and report the result with total conviction. Arguably worse than a human getting it wrong, since a human at least hesitates.
So the platform needs a few things a classic lakehouse usually treats as optional extras:
A semantic layer that is actually enforced, not just documented in a wiki nobody reads. Tools like the dbt Semantic Layer point in the right direction here, though I would not call the tooling mature yet.
A context store, typically vector-based, so agents can retrieve documentation, lineage, and prior decisions rather than guessing from column names alone.
A tool layer that mediates access. Agents should call defined actions, not run arbitrary queries against raw storage. This is partly a governance point and partly just safety; you do not want an autonomous process with unrestricted write access to production tables, full stop.
And a feedback loop. This is the one people skip. If an agent flags an anomaly or corrects a value, that correction needs to flow back into the platform, not evaporate into a chat log.
Put together, it starts to look less like a pipeline and more like a small nervous system.
I want to be careful not to oversell this. Autonomous agents making decisions on enterprise data raise real governance questions, and I do not think they are fully solved anywhere I have seen, including places with strong engineering cultures. Who approves an agent-initiated schema change? What happens when two specialist agents, say a cost agent and a data-quality agent, disagree about whether a pipeline should run? These are not hypothetical. They surface the moment you move past a proof of concept.
My working answer, for now, is the human-in-the-loop checkpoint you see at the bottom of the diagram. Not as a rubber stamp, but as an actual audit point where irreversible or high-cost actions pause for approval. It is not elegant. It slows things down on purpose. I think that is the right trade-off until the industry has more track record with agents that act rather than just recommend.
There is also a cost argument worth being honest about. Vector stores, additional orchestration layers, and always-on agent loops are not free, and I have seen teams add "agentic" infrastructure to platforms that genuinely did not need it yet. If your organization's actual pain point is stale dashboards, you probably do not need an agent mesh. You need better scheduling and a semantic layer. Save the agentic layer for problems that are actually about autonomous decision-making, not for problems that are really just batch latency in disguise.
None of this replaces the lakehouse. The bronze, silver, gold structure is still doing useful work underneath everything I described. What changes is what sits on top of it, and how much trust the platform is willing to extend to a process that never sleeps and never says "let me check with my manager." Building that layer thoughtfully, with real governance and a healthy amount of skepticism toward vendor claims, seems to me like the actual engineering problem of the next few years, more than any specific agent framework will be.
I am still figuring a lot of this out as I build it, and I would genuinely like to hear how other data teams are handling the semantic layer and governance pieces in practice.
The lakehouse was built for a different question
A traditional lakehouse assumes a human sits on the other end of every request. Someone opens a BI tool, someone schedules a dbt job, someone decides which report needs refreshing this morning. Even "real-time" lakehouses, the ones streaming through Kafka into Delta tables, still assume a person eventually interprets the output.
Here is roughly what that looks like in practice.
Notice the shape. It is linear. Sources flow one direction, get progressively cleaner, and land somewhere a human can look at them. Governance sits beside the pipeline, mostly as a catalog and an access policy, not as something actively consulted mid-flight. And critically, nothing in this diagram decides anything. It moves data. Decisions happen upstream, in someone's head, before the pipeline even runs.
That is fine when the consumer is a person with judgment and context. It gets shaky fast when the consumer is an agent that needs to decide, on its own, whether a number is trustworthy enough to act on.
What agents actually need that the lakehouse doesn't give them
I will admit I underestimated this at first. My instinct was: give the agent SQL access, point it at the gold layer, done. That sort of works for demos. It falls apart the moment two tables define "active customer" differently, which, in my experience, is basically every enterprise warehouse I have touched. An agent without a shared semantic layer will confidently average two incompatible definitions together and report the result with total conviction. Arguably worse than a human getting it wrong, since a human at least hesitates.
So the platform needs a few things a classic lakehouse usually treats as optional extras:
A semantic layer that is actually enforced, not just documented in a wiki nobody reads. Tools like the dbt Semantic Layer point in the right direction here, though I would not call the tooling mature yet.
A context store, typically vector-based, so agents can retrieve documentation, lineage, and prior decisions rather than guessing from column names alone.
A tool layer that mediates access. Agents should call defined actions, not run arbitrary queries against raw storage. This is partly a governance point and partly just safety; you do not want an autonomous process with unrestricted write access to production tables, full stop.
And a feedback loop. This is the one people skip. If an agent flags an anomaly or corrects a value, that correction needs to flow back into the platform, not evaporate into a chat log.
Put together, it starts to look less like a pipeline and more like a small nervous system.
The part that makes people nervous, and should
I want to be careful not to oversell this. Autonomous agents making decisions on enterprise data raise real governance questions, and I do not think they are fully solved anywhere I have seen, including places with strong engineering cultures. Who approves an agent-initiated schema change? What happens when two specialist agents, say a cost agent and a data-quality agent, disagree about whether a pipeline should run? These are not hypothetical. They surface the moment you move past a proof of concept.
My working answer, for now, is the human-in-the-loop checkpoint you see at the bottom of the diagram. Not as a rubber stamp, but as an actual audit point where irreversible or high-cost actions pause for approval. It is not elegant. It slows things down on purpose. I think that is the right trade-off until the industry has more track record with agents that act rather than just recommend.
There is also a cost argument worth being honest about. Vector stores, additional orchestration layers, and always-on agent loops are not free, and I have seen teams add "agentic" infrastructure to platforms that genuinely did not need it yet. If your organization's actual pain point is stale dashboards, you probably do not need an agent mesh. You need better scheduling and a semantic layer. Save the agentic layer for problems that are actually about autonomous decision-making, not for problems that are really just batch latency in disguise.
Where this leaves the lakehouse
None of this replaces the lakehouse. The bronze, silver, gold structure is still doing useful work underneath everything I described. What changes is what sits on top of it, and how much trust the platform is willing to extend to a process that never sleeps and never says "let me check with my manager." Building that layer thoughtfully, with real governance and a healthy amount of skepticism toward vendor claims, seems to me like the actual engineering problem of the next few years, more than any specific agent framework will be.
I am still figuring a lot of this out as I build it, and I would genuinely like to hear how other data teams are handling the semantic layer and governance pieces in practice.