The brief was straightforward. A payroll SaaS company wanted a ChatGPT-style chatbot to reduce support headcount by 30%. The CEO had seen a competitor's LinkedIn post. The board was asking questions. The timeline was tight.
This is exactly the kind of brief that causes FDE engagements to go wrong.
Not because the ask is unreasonable. Because the ask is the first layer. The job is to find what is underneath it.
What the data said
The company gave us a data room: ticket exports, interview transcripts, cost sheets, SLA breach records, CSAT surveys, integration schemas, a vendor post-mortem. Messy, assembled in a hurry, with timestamps in mixed timezones and category labels in five different spellings of the same thing.
Three numbers stood out immediately.
Ticket volume in August rose 0.98%. SLA credit payouts in August rose 112%. The CEO said volume had "gone up" and that explained the credits. The data said otherwise.
The second number: 884 of 1,367 tickets in the extract had no category label. That is 64.7% uncategorised. The "How-To is the biggest category" claim that drove the entire chatbot framing was based on the 35% of tickets that had any label at all.
The third number: 22.3% of tickets had no first-reply timestamp. The reported SLA compliance rate of 93% excluded those tickets entirely. The real compliance rate was unknown.
The actual problem
The company's largest Enterprise customer, Kestrel Logistics, accounted for 24 SLA breaches in Q3 and had an average CSAT score of 2.38 out of 5. Every senior stakeholder said the same thing: agents are slow, it is a headcount problem, a bot would fix it.
The Engineering Manager told a different story. Bug PNJ-4412: a connector idempotency issue causing duplicate attendance entries. Kestrel's supervisors had double-entered shifts. The connector dropped the idempotency key on retries. Engineering had a fix in testing. Estimated ship date: two to three weeks.
No chatbot resolves a connector bug. Kestrel's pain was going to go away on its own.
Meanwhile, the company had changed PF (provident fund) contribution ceilings on 1 August and WPS (wage protection system) layout rules on 15 July. The L2 lead found out about the PF change "around the first, second of August. Not before." The knowledge base article may not have been updated. The vendor chatbot pilot six months earlier had given customers answers quoting the previous financial year's TDS slabs because the KB sync had no automated refresh.
A chatbot grounded on the current KB, without a live feed from the payroll engine's statutory rates endpoint, would automate the propagation of wrong regulatory numbers at scale.
What problem discovery actually looks like
It is mostly reading. Not skimming. Reading the postmortem and noticing that the vendor called 38% deflection a success while the company's own review found 41% of those "deflected" sessions were users abandoning after a wrong or empty answer. Reading the transcript and noticing that the Head of Support could not recall when a specific KB article was last reviewed when asked directly. Reading the cost sheet and noticing that August credits doubled while August volume was nearly flat.
Then it is linking. Agent IDs in the ticket history match agent IDs in the roster: yes, they join. SLA breach ticket IDs match ticket export IDs: 113 of 128 match, enough to trace. Customer IDs are consistent across all four files. None of this was guaranteed. In many real engagements it is not true.
Then it is holding the tension between what people say and what the data shows. The CFO said the 30% headcount target was the CEO's, announced in an all-hands before she was consulted. The CEO thought the CFO wanted it. Both were making decisions based on a number neither of them owned.
What an FDE takes from this
Three things.
First: the first answer is almost never the right answer. The competitor's chatbot claim, the 30% headcount target, the "it is a volume problem" explanation. All of them had an alternative story waiting in the data.
Second: before you build anything, build a claim ledger. Every assertion a stakeholder makes is a hypothesis. Each one is either confirmed, contradicted, or unresolved by the evidence. You ship when the hypothesis is confirmed, not when the stakeholder says it confidently.
Third: the most important question in problem discovery is not "what should we build?" It is "what would make this problem go away without building anything?" In this case, the answer was partly a two-week engineering fix. Knowing that changes what you recommend.
The chatbot question was real. The company had a knowledge management problem that an AI system could genuinely help with, once the KB staleness issue was addressed and the statutory rates endpoint was made available to a support tool. But that is a very different thing from "plug GPT into our help center and cut twelve people."
The job of an FDE is to find that difference before it costs anyone eighteen months and a second failed pilot.