I Built an End-to-End Enterprise Intelligence Simulation to Answer One Question

[Rick Bahague]

Most enterprise AI architecture diagrams look the same. An AI agent sits at the top, utilizing "Ontology", "Knowledge Graph", "Certified Data" and "Vector Index", and connections promise that every answer will be grounded, governed and traceable.

I wanted to find out whether these integrations actually work. So I simulated the full environment using open source tools just answering one question, end to end: "Which overloaded transformers have the highest customer impact risk?"

The domain is for an electricity distribution grid, and every architectural simulation decision served that one question.

Overview of the system​

The design splits "enterprise intelligence" into two tracks.

1. Semantic resolution - works out what the user means or intent. An OWL (Web Ontology Language) ontology, loaded with RDFLib, maps terms included on the question to a governed concept. A YAML metric registry holds the certified metric definition. A Neo4j graph captures the grid topology including transformers, substations and service areas.
2. Evidence retrieval - finds out what is true. A certified data service (FastAPI and DuckDB over simulated CSVs) supplies figures to the response. I registered pre-computed model scores in MLflow. A Qdrant index runs hybrid dense and BM25 (keyword ranking) search over 12 simulated policy documents.

The agent calls seven tools:

1. resolve_business_request turns plain language into ontology concepts.
2. get_metric_definition returns a certified definition and threshold.
3. query_certified_metric queries the certified data.
4. get_entity_relationships walks the graph.
5. get_governed_model_output returns the approved risk scores.
6. search_certified_context retrieves policy text.
7. get_evidence_lineage returns the session's audit trail.

I confined responses of these tools to hold it together with every tool returning the same evidence contract.

json
{
"value": "15 row(s) for metric='transformer_loading'",
"source": "certified_transformers_v1.0.0",
"timestamp": "2026-10-08T12:00:00+00:00",
"version": "1.0.0",
"lineage": "metric=transformer_loading, filters={'load_pct__gt': 50}",
"confidence": 0.95,
"classification": "INTERNAL"
}

The contract carries value, source, timestamp, version, lineage, confidence and classification. If a number in the final response lacks traceability, it should not be part of the response.

The whole stack runs as seven Docker Compose services: Neo4j, Qdrant, Redis, MLflow, the certified data service, the tool server and a Streamlit UI.

Simulated data in a real architecture​

Everything that orchestrates the docker containers, network and app are real code. I created mock data to simulate including 50 transformers (15 of them above 50% load, peaking at 70.5%), 10 service areas, impact measures, risk scores and 12 policy documents.

A seeded script generates all the mock data set, to ensure the every run sees the same configuration and situation. That is important, because I could check the agent's answers against the source CSVs, figure by figure, instead of trusting that they sounded plausible.

On the first run, the agent, runs a standard tool_use loop with adaptive thinking. I expected it to write an orchestrator for the ten-step sequence. But it did not.

Given the tools and a governance system prompt, the model planned the sequence itself and fanned out in 27 tools calls. All successful and audited, and the answer was correct.

If a rule or metric can be computed, compute it as part of the data set, instead of asking the model to figure it out​

The model fails at grouping. I added a group by / min_count option to the certified query, so the data service does the grouping properly for questions requiring aggregations in the response.

Governance at work​

Governed AI is easy to say. In this simulation, it means specific, testable behaviour:

1. Fail-closed authorisation - list the allowed roles for each tool, with no wildcards. A request with no role is a guest, and guests can do nothing.
2. An audit line for every call, including denied and failed ones.
3. Schema validation before any tool runs. The tool server rejects unknown fields.
4. Column allowlists and bound parameters. The model never writes SQL. This was my first known constraint for the simulation.
5. Circuit breakers with degraded mode. When I stopped Qdrant mid-demo, the agent opened its answer with a warning, dropped the policy actions it could no longer cite, and stated confidence 0.3 instead of guessing.

I want my agent that says "this part of my evidence is missing" than to be always confident.

System Flow​

A user types the question in the chat UI and watches the tool calls stream in.

The response has:

• a one-line answer;
• a ranked table with columns relevant to questions
• two to four policy actions, each citing its source document;
• a confidence statement.

The evidence panel shows the contract behind every number. The audit page shows every call the session made.
The integration test checks the live answer for figures, ranking, citations, audit completeness, lineage, cascade groups, impact bands and latency.

Conclusion​

If your experimenting enterprise data retrieval with agents, I recommend picking one question that is frequently ask and build it all the way through. That one question will force the ontology, registry, graph, warehouse, vector index, model registry, agent, auth, audit and UI to work together.

Thanks to the people behind the open-source projects that carried this build: RDFLib, Neo4j, Qdrant, DuckDB, FastAPI, MLflow, Redis and Streamlit.