The demo goes great.
The model answers correctly. The latency is acceptable. The stakeholder nods. You leave the call with momentum.
Three months later, the pilot is struggling. Users are not trusting the outputs. There are edge cases nobody caught. The data pipeline is brittle. The customer is frustrated.
This is not a model problem. The model is probably fine. This is the deployment gap — the distance between a system that works in a controlled demo and a system that works in a real customer environment.
Here are the four layers where it breaks, and what the forward deployed engineer does about each one.
Layer 1: The data assumption
What breaks: The demo used curated, clean data. Production has the real data.
Every AI demo makes assumptions about the input. The data is structured a certain way, the fields are populated, the format is consistent. In production, the customer's actual data is messier than what was assumed — often dramatically so.
Common failure modes:
- Fields that were assumed to be filled are sometimes empty
- Dates in three different formats depending on which team entered them
- Historical records from a legacy system that use a completely different schema
- Free-text fields where users have been creative in ways no prompt anticipated
The fix is a data audit before you finalize any architecture. Not after the demo — before the design. Ask to see a sample of 500 real records from the actual system. Look at the edge cases. Count the nulls. Find the format inconsistencies.
The answer to "why does the AI work in the demo but not in production" is, more often than not, that the demo data was not the real data.
Layer 2: The integration reality
What breaks: The demo ran in isolation. Production requires talking to existing systems.
In a demo, the AI system is the only thing running. In production, it has to integrate with:
- The CRM or ERP that the customer has been using for ten years
- The authentication system that has its own quirks
- The data warehouse that has a rate limit nobody mentioned
- The internal API that returns different responses depending on environment
- The workflow tool that triggers the AI call at a time and with a payload nobody anticipated
Every integration adds surface area for failure. Every external system is a dependency that can go down, change its API, or return something unexpected.
The forward deployed engineer maps the integration surface before design. Draw out every external system the AI needs to touch. For each one: what is the SLA? What happens when it is slow or down? What happens when it returns an unexpected response?
The integration layer is where most AI demos die. Design it explicitly, not as an afterthought.
Layer 3: The trust gap
What breaks: The demo was evaluated by the technical team. Production is used by the actual users.
Users of AI systems behave differently from evaluators of AI systems. The evaluator is looking for the system to succeed. The user is busy, skeptical, and will find every edge case.
The trust gap manifests as:
- Users who spot one wrong output and never fully trust the system again
- Users who over-trust the system in ways that cause new problems
- Users who find workarounds that bypass the AI entirely
- Users who escalate to management any time the output looks odd
The fix is trust scaffolding: mechanisms that help users develop calibrated trust in the system over time. This includes:
- Showing the evidence behind AI outputs, not just the outputs
- Making it easy to provide feedback or flag errors
- Designing the UI to support verification, not just consumption
- Starting with lower-stakes use cases where errors are recoverable
Trust is not something users give to AI systems automatically. It is something they build through repeated experience. The FDE who designs for trust from the start has a dramatically higher chance of getting a production system that users actually use.
Layer 4: The ownership vacuum
What breaks: The demo had clear owners. Production has a gap nobody claimed.
This is the most underrated failure mode.
In the demo phase, there is a clear owner: the implementation team. When something goes wrong, someone is accountable.
In production, accountability often fragments:
- The vendor owns the model
- The customer owns the environment and the data
- The customer's IT team owns the infrastructure
- Nobody owns the space in between — the integration, the edge cases, the incident response when something goes wrong at 2am
Without a named owner for the production system, incidents escalate slowly, bugs linger, and the customer's trust erodes gradually until someone makes the decision to shut the project down.
The FDE closes this gap explicitly. Before go-live: who owns incident response? Who is the escalation path? What is the rollback procedure if the system needs to be turned off quickly? How are errors surfaced and tracked?
This is not bureaucracy. It is engineering. The system that has no incident response plan does not have a production plan.
What this means for how you build
The deployment gap is predictable. It has the same layers every time. The FDE who knows this designs for it upfront:
- Data audit before architecture — see the real data before you commit to a design
- Integration map before build — draw every external system and its failure modes
- Trust scaffolding from day one — design for user trust, not just technical correctness
- Ownership documentation before go-live — name every owner before you hand off
None of this is glamorous. None of it shows up in the demo. All of it determines whether the project survives contact with the real world.
The engineers who understand this are the ones companies hire to deploy AI into customers. The demo is easy. The deployment is the job.
This is part of The Deployment Gap series — essays on the distance between AI that works and AI that delivers value in production.