Start with the difference
When something goes wrong with an integration, the natural reaction is to look at the integration.
That's fair. After all, that's the piece of the process with “integration” in its name.
But one of the first questions I've learned to ask is much simpler:
What has changed?
It won't solve every problem. Sometimes the integration really is broken. But when something that has been working suddenly produces an unexpected result, understanding what changed is usually a good place to start.
And the answer isn't necessarily in the integration.
The integration may be doing exactly what it was told to do
Consider an employee who suddenly gets assigned the wrong pay rule.
The first suspicion might be that the integration assigned the wrong pay rule.
But what changed?
Maybe the employee moved to another location. Maybe the HR system hasn't caught up with the move yet. Maybe the HR data is simply wrong. Or perhaps the source data is correct, but a recent configuration change has altered how that employee's information is interpreted.
The integration may have processed everything correctly.
This is one of the more frustrating — and useful — lessons in integration work:
A bad result doesn't necessarily mean bad integration logic.
Integration failure and transaction failure aren't quite the same thing
I tend to think about these as two different problems.
An integration failure is more likely to be infrastructure or process related.
The SFTP endpoint isn't accessible. A connection fails. A file wasn't generated. A certificate expired. Authentication stopped working. A process can't reach a required service.
Something in the machinery isn't working.
A transactional failure is different.
The integration runs. The connection works. The file is generated. The process completes. But an employee has the wrong pay rule. Or a location produces an unexpected value. Or a particular group of employees suddenly gets a different payroll result.
The integration may have successfully completed while producing an unexpected result.
The difficult problems are often the ones where the data is almost right
In my experience, transactional problems are often harder to find than actual integration failures because the data isn't obviously wrong.
It's subtly wrong.
An employee has the correct job, but the wrong location. The location has a valid code, but it's associated with a different configuration than expected. A source field contains a perfectly reasonable value, but that value no longer matches an assumption somewhere downstream.
And sometime even worse.... Nothing crashes.
No connector error. No exception. No failed process. Nothign to sink your teeth into. The integration simply does what it was designed to do with the data it received.
So, what changed?
When something suddenly looks wrong, I like to compare it with something that works. And where ever possible, I want to follow the transaction through the process.
What is different about this employee? What is different about this location?
Was there a recent change to the source data? A configuration change? A new pay rule? A change to how an existing pay rule calculates? A deployment? A change in the business process?
What is different about this transaction?
Sometimes the answer is obvious. Sometimes it takes a little digging. But establishing the difference between a working transaction and the one that didn't gives you something concrete to investigate.
Don't ignore the infrastructure
Of course, not every problem is a data problem.
Sometimes the file really isn't there yet because the upstream process is taking longer than expected.
Sometimes the SFTP server is inaccessible. Sometimes a connection has failed. Sometimes a credential or certificate has expired.
Those are genuine integration or infrastructure failures, and they need to be treated as such.
The point isn't to assume the integration is innocent. It's to avoid assuming it's guilty before we know where the problem actually occurred.
Sometimes the best fix is not to change the integration
This is probably the biggest lesson for me.
When an integration appears to be producing an unexpected result, don't immediately change the integration.
First ask: What has changed?
Sometimes the answer will lead directly to an integration defect. Sometimes it will lead to infrastructure. And quite often, especially with transactional issues, it will lead back to the data or configuration.
That's not always the most exciting answer. But it's a much better outcome than rewriting perfectly good integration logic to compensate for a problem somewhere else.
The integration may not be broken. The transaction may simply be different.