The situation
A production e-commerce system with around 2,000 active users was integrated with a CRM. Customers could log in and view their invoices. For months, everything worked.
Then the CRM changed how it handled filtering internally. The integration didn’t change — but the behaviour did. Some customers started seeing invoices that did not belong to them.
No crashes. No errors. No alerts. By the time we got involved, the issue had already been present for four weeks. From the system’s perspective, everything was fine. From a GDPR perspective, it was already an incident.
The symptom
Nothing looked broken. The portal fetched invoices from the CRM using a filtered query. The CRM returned data. The portal displayed it. Logs showed no errors.
But the data was wrong — and worse, it looked completely valid.
Investigation
The first step was not to fix anything. It was to understand the system.
- What systems are involved?
- Where does the data come from?
- What guarantees exist between them?
The portal relied on a filtered query sent to the CRM API. For a long time, that query returned exactly what was expected. Then it didn’t.
Root cause
The CRM had changed how filtering worked internally. The same query still returned results — but those results were no longer strictly limited to the correct customer.
The integration trusted the CRM response. There was no validation layer on the portal side. The assumption was:
That assumption became false.
Why this kind of bug is dangerous
Most teams expect failures to look like failures:
- exceptions
- failed requests
- system crashes
But integration failures often don’t behave like that. The system keeps working. Only the meaning of the data changes. In this case, incorrect invoices were shown to real customers — for weeks — without triggering a single alert. That’s far more dangerous.
The fix
The fix was not to tweak the query. It was to remove the assumption.
- The portal now enforces customer-level constraints explicitly
- The CRM response is treated as untrusted input
- Edge cases were tested to prevent cross-customer leakage
Conclusion
The issue was resolved. But more importantly, the system became predictable again.
If your system depends on external services:
- never assume their behaviour is stable
- never trust their filtering blindly
- always validate critical constraints on your side
Because when integrations fail silently, you don’t get an error. You get incorrect reality.