S
Sagar Bonthu
Guest
A company reports quarterly earnings per share of $2.10. Wall Street's consensus estimate was $2.05, so the headline says the company beat expectations by five cents.
That subtraction is only correct if the two numbers measure the same thing.
This is easy to miss. One analyst may forecast adjusted earnings while the company reports both official (GAAP) and adjusted figures. Two analysts who both write "adjusted EPS" may be leaving out different expenses. A third may have published an estimate before the company changed its guidance and never updated it.
Working with financial data has made me cautious about treating the label on a number as its full definition. Before I let a system average or compare values, I want it to know what those values actually represent. That is the engineering problem hiding behind a simple earnings headline, and in this article I will walk through how I think about solving it, including where a language model helps and where I would never let it make the call.
One note before I start. The examples here are generalized from public methodologies and common data system design patterns. They do not describe any financial data provider's internal implementation.
Suppose four analysts cover the same company and publish estimates for the same quarter:
If I look only at the numbers, these are four observations of the same thing, and averaging them is trivial. Once I read the notes beside them, the problem changes. A consensus system first has to decide whether the estimates are comparable at all. That depends on details such as the fiscal period, the accounting basis, what was excluded, the currency, the unit, when the estimate was last revised, and whether anything important happened after it was published.
This is a recognized problem and not just my own caution. S&P Global's published methodology for Capital IQ Estimates addresses it directly. Estimates prepared on different bases are not necessarily combined into one consensus. Its example has ten analysts covering a company: eight include stock-based compensation and two do not. The two estimates on the minority basis are left out. S&P also notes that a larger number of contributors does not automatically make a consensus more accurate. [1]
So before anything is averaged, the system has to decide which observations belong together.
A study in the Journal of Accounting Research shows how much these decisions matter. The researchers compared earnings information from Bloomberg, Capital IQ, FactSet, I/B/E/S, and Zacks. Across 94,030 company-quarters covered by all five providers, the earnings surprise differed between providers in 58% of cases. In roughly 9% of company-quarters, at least one provider showed a miss while another showed a beat. The differences were larger where providers were more likely to differ in their contributor base or accounting methodology. [2]
In other words, the same company in the same quarter can beat and miss at the same time, depending on whose data you read. The disagreement starts long before the final subtraction.
Cleaning up the analyst side does not finish the job. Suppose the company reports:
If the consensus represents adjusted earnings, comparing it with $1.82 produces a misleading miss. If the analysts were forecasting GAAP earnings, using $2.10 creates the opposite error.
Even the word "adjusted" does not settle it. A company's reconciliation may remove restructuring costs, acquisition expenses, impairments, tax effects, or other items. The reported result has to be read on the same basis the analysts used. LSEG says this explicitly in its I/B/E/S Actuals documentation: reported actuals are standardized to the basis used by the contributing analysts. [3]
That leaves two matching jobs:
For the remainder of this article I focus on the analyst side, because free-form research notes make the problem easier to see. The same principle applies to reported actuals. Once both sides line up, "actual minus consensus" finally means something.
A large part of this work belongs in ordinary, predictable software. Checking the period, converting units, adjusting for stock splits, and applying freshness rules all behave the same way every time. If one value is in millions and another is in billions, I want plain code doing that conversion, not a model.
The hard cases appear when an important distinction exists only in language. Here is one analyst's note:
"Q2 adjusted EPS estimate: $2.09. Excludes restructuring and acquisition integration charges. Stock-based compensation remains included in operating expenses."
A simple lookup table sees the label "adjusted EPS" and files the estimate under adjusted EPS. Now take a second analyst:
"Q2 adjusted EPS estimate: $2.07. Excludes restructuring, acquisition integration charges and stock-based compensation."
The same lookup sends both estimates to the same place, even though they treat stock-based compensation in opposite ways.
I could keep adding aliases and special cases. That works for wording the system has already seen, and it falls behind as soon as new phrasing or new combinations of adjustments show up. The missing information is about meaning: what does "adjusted" mean in this particular note? Recovering meaning from prose is exactly where a language model becomes useful.
Here is the rule I hold to: the model does not decide whether an estimate belongs in the consensus. That decision depends on policies and checks that live outside the source text.
So I split the information in two.
Facts the system already knows stay out of the model's hands. These include the estimate's value, its currency, the company, the fiscal period the system expects, and when the estimate was last revised. I deliberately do not show the model the expected period, because I want to use it later as an independent cross-check.
The model gets only the relevant passage from the analyst's note, with each sentence numbered, plus the identifier and version of the source document. (Finding that passage inside a long research report is a separate retrieval problem that I am leaving out here.)
From that passage, the model fills in a fixed form. For the first analyst's note, the completed form would say:
Three details make this form trustworthy enough to build on:
The completed form is then combined with the known facts to create a candidate estimate.
A candidate still has to earn its place in the consensus.
The policy that decides comparability is written independently of the model. It covers how contributors are grouped, which accounting basis the consensus uses, how fresh an estimate must be, and other methodology choices. I version that policy separately, so a change to a prompt or a model can never quietly redefine who gets into the consensus.
Each candidate passes through these checkpoints in order:
The three outcomes mean different things. Eligible means the estimate is valid and comparable with the consensus basis. Exclude means the estimate may be perfectly valid by itself but should not take part in this particular consensus. Review means the evidence is inconsistent or too thin for an automatic decision.
This separation matters because it stops the system from treating every failed check as bad data. An estimate can be legitimate and still be the wrong fit for one aggregation.
I also do not route cases based on the model saying it is "95% confident." A model's self-reported confidence does not establish that it is right. The signals I trust are ones I can observe: whether the period from the text agrees with independent metadata, whether the cited evidence exists, whether the accounting treatment fits the required basis, and whether anything else conflicts with the interpretation.
Those checks also make failures easy to diagnose. An engineer can see at a glance whether a case stopped because of unclear meaning, missing evidence, stale data, or a comparability rule.
Go back to the second analyst, the one who excludes stock-based compensation. Suppose the model reads the note perfectly: EPS, second quarter of 2026, adjusted basis, with restructuring, acquisition integration, and stock-based compensation all excluded.
Nothing is wrong with that estimate, and nothing is wrong with the model's reading of it.
Now assume the consensus policy follows the majority treatment, where stock-based compensation stays included. The validator compares the candidate against that policy and returns Exclude.
This is the case I find most useful when explaining the design. The reading is correct and the estimate is still not eligible. Understanding a source and deciding eligibility are two different jobs, and they should live in two different places.
The same thinking applies to conflicting evidence. If the note says Q2 and independent metadata says Q3, I do not ask the model to pick whichever answer seems more convincing. That case goes to review. An unresolved contradiction is information too.
Human review is valuable when the machine-readable evidence cannot support a safe decision. In practice that means cases like these:
Simple cases should flow through without touching any of these. For ambiguous ones, review stops an uncertain reading from quietly turning into trusted data.
If you use a routing score, build it from things you can measure against a test set: agreement with metadata, whether evidence resolves, consistency of mappings, and past validation results. Do not build it from how certain the model says it feels.
Adding a language model is not free. It brings latency, cost, versioning concerns, and one more component to re-evaluate whenever a prompt or model changes. The fixed form also becomes part of the system's interface, so changes to its fields or allowed values deserve the same care as changes to any production API.
I think the design earns its complexity when sources vary so much that fixed mappings become brittle: analyst notes in many styles, shifting non-GAAP terminology, adjustments described in prose, or new financial concepts that appear before anyone has mapped them by hand.
Arithmetic, currency handling, and date calculations gain nothing from being routed through a model. Keep them in plain code.
Keeping the model's job narrow also limits the damage a model change can do. A new model can be tested against the same form and the same test cases while the policies for periods, freshness, and eligibility stay untouched. That makes the AI component easier to replace, easier to test, and easier to distrust when you need to.
This pattern is not specific to earnings. It applies anywhere you aggregate numbers whose meaning is buried in text. Here is what I would do first:
A five-cent earnings beat looks like the result of one subtraction. Before that subtraction can be trusted, the analyst estimates have to sit on a comparable basis, and the company's reported result has to represent the same financial concept.
Traditional software already handles much of this well. The difficult cases are the ones where the distinction lives inside human language: an adjustment note, an unfamiliar metric name, or a sentence explaining what was included or left out. In the design I have described, the language model works as a careful reader. It turns that language into a structured, source-backed interpretation that the surrounding system can test.
The failure case is what makes the boundary worth having. A model can read the measure correctly and the policy can still exclude the estimate. A plausible reading has to survive the evidence and policy checks around it before it becomes trusted data.
Once that work is done, deciding whether $2.10 beat $2.05 really is the easy part.
That subtraction is only correct if the two numbers measure the same thing.
This is easy to miss. One analyst may forecast adjusted earnings while the company reports both official (GAAP) and adjusted figures. Two analysts who both write "adjusted EPS" may be leaving out different expenses. A third may have published an estimate before the company changed its guidance and never updated it.
Working with financial data has made me cautious about treating the label on a number as its full definition. Before I let a system average or compare values, I want it to know what those values actually represent. That is the engineering problem hiding behind a simple earnings headline, and in this article I will walk through how I think about solving it, including where a language model helps and where I would never let it make the call.
One note before I start. The examples here are generalized from public methodologies and common data system design patterns. They do not describe any financial data provider's internal implementation.
Four Forecasts, Four Different Meanings
Suppose four analysts cover the same company and publish estimates for the same quarter:
- Analyst A says $2.04 and leaves out restructuring charges.
- Analyst B says $2.09 and leaves out restructuring and acquisition integration charges.
- Analyst C says $2.07 and uses a different definition of adjusted earnings.
- Analyst D says $1.91 and has not revised the estimate since the company issued new guidance.
If I look only at the numbers, these are four observations of the same thing, and averaging them is trivial. Once I read the notes beside them, the problem changes. A consensus system first has to decide whether the estimates are comparable at all. That depends on details such as the fiscal period, the accounting basis, what was excluded, the currency, the unit, when the estimate was last revised, and whether anything important happened after it was published.
This is a recognized problem and not just my own caution. S&P Global's published methodology for Capital IQ Estimates addresses it directly. Estimates prepared on different bases are not necessarily combined into one consensus. Its example has ten analysts covering a company: eight include stock-based compensation and two do not. The two estimates on the minority basis are left out. S&P also notes that a larger number of contributors does not automatically make a consensus more accurate. [1]
So before anything is averaged, the system has to decide which observations belong together.
A study in the Journal of Accounting Research shows how much these decisions matter. The researchers compared earnings information from Bloomberg, Capital IQ, FactSet, I/B/E/S, and Zacks. Across 94,030 company-quarters covered by all five providers, the earnings surprise differed between providers in 58% of cases. In roughly 9% of company-quarters, at least one provider showed a miss while another showed a beat. The differences were larger where providers were more likely to differ in their contributor base or accounting methodology. [2]
In other words, the same company in the same quarter can beat and miss at the same time, depending on whose data you read. The disagreement starts long before the final subtraction.
The Reported Number Has the Same Problem
Cleaning up the analyst side does not finish the job. Suppose the company reports:
- GAAP diluted EPS of $1.82
- Adjusted diluted EPS of $2.10
If the consensus represents adjusted earnings, comparing it with $1.82 produces a misleading miss. If the analysts were forecasting GAAP earnings, using $2.10 creates the opposite error.
Even the word "adjusted" does not settle it. A company's reconciliation may remove restructuring costs, acquisition expenses, impairments, tax effects, or other items. The reported result has to be read on the same basis the analysts used. LSEG says this explicitly in its I/B/E/S Actuals documentation: reported actuals are standardized to the basis used by the contributing analysts. [3]
That leaves two matching jobs:
- Work out which analyst forecasts represent a comparable expectation.
- Find the company-reported result that corresponds to that expectation.
For the remainder of this article I focus on the analyst side, because free-form research notes make the problem easier to see. The same principle applies to reported actuals. Once both sides line up, "actual minus consensus" finally means something.
Where Fixed Rules Start to Break
A large part of this work belongs in ordinary, predictable software. Checking the period, converting units, adjusting for stock splits, and applying freshness rules all behave the same way every time. If one value is in millions and another is in billions, I want plain code doing that conversion, not a model.
The hard cases appear when an important distinction exists only in language. Here is one analyst's note:
"Q2 adjusted EPS estimate: $2.09. Excludes restructuring and acquisition integration charges. Stock-based compensation remains included in operating expenses."
A simple lookup table sees the label "adjusted EPS" and files the estimate under adjusted EPS. Now take a second analyst:
"Q2 adjusted EPS estimate: $2.07. Excludes restructuring, acquisition integration charges and stock-based compensation."
The same lookup sends both estimates to the same place, even though they treat stock-based compensation in opposite ways.
I could keep adding aliases and special cases. That works for wording the system has already seen, and it falls behind as soon as new phrasing or new combinations of adjustments show up. The missing information is about meaning: what does "adjusted" mean in this particular note? Recovering meaning from prose is exactly where a language model becomes useful.
Give the Model a Narrow Job
Here is the rule I hold to: the model does not decide whether an estimate belongs in the consensus. That decision depends on policies and checks that live outside the source text.
So I split the information in two.
Facts the system already knows stay out of the model's hands. These include the estimate's value, its currency, the company, the fiscal period the system expects, and when the estimate was last revised. I deliberately do not show the model the expected period, because I want to use it later as an independent cross-check.
The model gets only the relevant passage from the analyst's note, with each sentence numbered, plus the identifier and version of the source document. (Finding that passage inside a long research report is a separate retrieval problem that I am leaving out here.)
From that passage, the model fills in a fixed form. For the first analyst's note, the completed form would say:
- Measure: EPS
- Period stated in the text: second quarter of 2026
- Basis: adjusted
- Items excluded: restructuring, acquisition integration
- Items included: stock-based compensation
- Evidence: the period and basis come from sentence 1, the exclusions from sentence 2, the inclusion from sentence 3
Three details make this form trustworthy enough to build on:
- It is a strict form and not free text. Required fields are checked, known concepts come from a fixed list of allowed values, and a malformed answer is stopped before it goes any further.
- Every answer points to the sentence that supports it. A model's explanation does not count as proof. The pointers must lead back to an unchangeable version of the source, so another component can confirm that the sentence exists and says what the model claims.
- The model reads the period from the text by itself, without being told what the system expects. If the two disagree later, that disagreement is a useful signal.
The completed form is then combined with the known facts to create a candidate estimate.
Run Every Candidate Through Checkpoints
A candidate still has to earn its place in the consensus.
The policy that decides comparability is written independently of the model. It covers how contributors are grouped, which accounting basis the consensus uses, how fresh an estimate must be, and other methodology choices. I version that policy separately, so a change to a prompt or a model can never quietly redefine who gets into the consensus.
Each candidate passes through these checkpoints in order:
- Is the form valid? If not, send it to review.
- Do the evidence pointers lead to real sentences in the source version? If not, send it to review.
- Does the period the model read from the text match the period the system expected? If not, send it to review.
- Has a significant event, such as new company guidance, happened since the estimate was last revised? If so, exclude it as stale.
- Does the accounting treatment match the basis the consensus policy requires? If not, exclude it.
- If every check passes, the estimate is eligible.
The three outcomes mean different things. Eligible means the estimate is valid and comparable with the consensus basis. Exclude means the estimate may be perfectly valid by itself but should not take part in this particular consensus. Review means the evidence is inconsistent or too thin for an automatic decision.
This separation matters because it stops the system from treating every failed check as bad data. An estimate can be legitimate and still be the wrong fit for one aggregation.
I also do not route cases based on the model saying it is "95% confident." A model's self-reported confidence does not establish that it is right. The signals I trust are ones I can observe: whether the period from the text agrees with independent metadata, whether the cited evidence exists, whether the accounting treatment fits the required basis, and whether anything else conflicts with the interpretation.
Those checks also make failures easy to diagnose. An engineer can see at a glance whether a case stopped because of unclear meaning, missing evidence, stale data, or a comparability rule.
A Correct Reading Can Still Be Excluded
Go back to the second analyst, the one who excludes stock-based compensation. Suppose the model reads the note perfectly: EPS, second quarter of 2026, adjusted basis, with restructuring, acquisition integration, and stock-based compensation all excluded.
Nothing is wrong with that estimate, and nothing is wrong with the model's reading of it.
Now assume the consensus policy follows the majority treatment, where stock-based compensation stays included. The validator compares the candidate against that policy and returns Exclude.
This is the case I find most useful when explaining the design. The reading is correct and the estimate is still not eligible. Understanding a source and deciding eligibility are two different jobs, and they should live in two different places.
The same thinking applies to conflicting evidence. If the note says Q2 and independent metadata says Q3, I do not ask the model to pick whichever answer seems more convincing. That case goes to review. An unresolved contradiction is information too.
What Should Actually Go to a Human
Human review is valuable when the machine-readable evidence cannot support a safe decision. In practice that means cases like these:
- Two plausible concepts remain after interpretation and the system cannot choose between them.
- The period read from the text disagrees with the structured metadata.
- An adjustment cannot be classified reliably.
- The cited evidence cannot be traced back to the source.
- A newly introduced financial measure has no established mapping yet.
Simple cases should flow through without touching any of these. For ambiguous ones, review stops an uncertain reading from quietly turning into trusted data.
If you use a routing score, build it from things you can measure against a test set: agreement with metadata, whether evidence resolves, consistency of mappings, and past validation results. Do not build it from how certain the model says it feels.
When This Design Is Worth the Cost
Adding a language model is not free. It brings latency, cost, versioning concerns, and one more component to re-evaluate whenever a prompt or model changes. The fixed form also becomes part of the system's interface, so changes to its fields or allowed values deserve the same care as changes to any production API.
I think the design earns its complexity when sources vary so much that fixed mappings become brittle: analyst notes in many styles, shifting non-GAAP terminology, adjustments described in prose, or new financial concepts that appear before anyone has mapped them by hand.
Arithmetic, currency handling, and date calculations gain nothing from being routed through a model. Keep them in plain code.
Keeping the model's job narrow also limits the damage a model change can do. A new model can be tested against the same form and the same test cases while the policies for periods, freshness, and eligibility stay untouched. That makes the AI component easier to replace, easier to test, and easier to distrust when you need to.
If You Are Building Something Like This
This pattern is not specific to earnings. It applies anywhere you aggregate numbers whose meaning is buried in text. Here is what I would do first:
- Before averaging anything, list the attributes that make two values comparable: period, basis, exclusions, currency, unit, and freshness.
- Keep every predictable operation in ordinary code. Use a model only where meaning has to be recovered from language.
- Make the model fill in a strict form with fixed allowed values, and reject anything malformed.
- Require a pointer to the source sentence for every field, and have a separate component verify that the pointer is real.
- Hold back at least one fact you already know, such as the expected period, so you can cross-check the model's reading against it.
- Write the eligibility policy outside the model and version it separately.
- Use three outcomes, not two. "Valid but does not belong here" is different from "bad data," and both are different from "needs a human."
- Route on signals you can observe, never on the model's stated confidence.
- Send contradictions to review. Do not ask the model to settle them.
Conclusion
A five-cent earnings beat looks like the result of one subtraction. Before that subtraction can be trusted, the analyst estimates have to sit on a comparable basis, and the company's reported result has to represent the same financial concept.
Traditional software already handles much of this well. The difficult cases are the ones where the distinction lives inside human language: an adjustment note, an unfamiliar metric name, or a sentence explaining what was included or left out. In the design I have described, the language model works as a careful reader. It turns that language into a structured, source-backed interpretation that the surrounding system can test.
The failure case is what makes the boundary worth having. A model can read the measure correctly and the policy can still exclude the estimate. A plausible reading has to survive the evidence and policy checks around it before it becomes trusted data.
Once that work is done, deciding whether $2.10 beat $2.05 really is the easy part.