API Integration Strategy for Growing Businesses

Point-to-point works, until it does not

Connecting two systems directly is the right answer for two systems. It is simple, there is nothing extra to run, and anyone can understand it by reading the code at either end. Adding a middleware layer for a single integration is over-engineering.

The problem is not any individual connection. It is the accumulation. Each new system is connected to several existing ones, each connection has its own authentication, data mapping, retry behaviour and failure mode, and no two were written the same way because they were built a year apart by different people.

The point to reconsider is not a specific number of systems, it is the first time a change to one system requires you to modify several integrations, and nobody is certain which ones.

Connections grow faster than systems

The arithmetic is unforgiving. Fully connecting n systems takes up to n(n-1)/2 connections: three systems need three, five need ten, eight need twenty-eight. Real estates are never fully connected, but the growth curve is the same shape, and it is why integration debt accumulates quietly and then suddenly dominates.

A central layer changes the curve. Each system integrates once with the hub rather than separately with every peer, so connections grow in line with systems rather than with pairs of systems. That is the entire argument for middleware, and it is sufficient on its own.

The cost is a component you now operate, monitor and upgrade, and a single point whose failure affects everything. That trade is worth making later than vendors suggest and earlier than most teams actually make it.

Agree the source of truth per entity

The most common cause of integration pain is not technical. It is that two systems both believe they own the same entity. The CRM and the ERP each hold a customer record, both allow edits, and nobody decided which one wins.

Decide ownership per entity, write it down, and enforce it. Customers are mastered in one system; products in one system; orders in one system. Other systems hold copies and treat them as read-only, or send changes through the owner rather than editing their local copy.

Ownership can be split by field where it has to be — sales owns the contact details, finance owns the credit terms — but that must be a deliberate design with the boundary documented, not an accident discovered when two values diverge.

Idempotency, retries and dead letters

Networks fail halfway. A message is delivered and the acknowledgement is lost, so the sender retries and the receiver processes it twice. Without protection, that is a duplicate order or a double payment, and it happens in every integration eventually.

Idempotency is the protection: every message carries a unique identifier, and the receiving end records what it has already processed and ignores repeats. This is not difficult to build, and retrofitting it after the first duplicate is considerably more work than including it at the start.

Retries need backoff and a limit. Immediate retries against a struggling system make the outage worse; exponential backoff with jitter gives it room to recover. After a defined number of attempts, the message goes to a dead-letter queue rather than being dropped or retried forever.

A dead-letter queue is only useful if someone looks at it. It needs an owner, an alert when it is not empty, and a documented way to reprocess a message once the underlying problem is fixed.

Working versus silently broken

Integrations do not usually fail loudly. They fail by stopping — a credential expires, an endpoint changes, a scheduled job quietly dies — and nobody notices until someone asks why last week's orders are missing.

Monitor for absence, not only for errors. If a feed normally carries several hundred records a day, an alert should fire when it carries none, because zero errors and zero records looks identical to healthy on most dashboards.

Track queue depth and processing lag as well. A growing backlog is the earliest warning that a downstream system is struggling, and it arrives well before anything starts failing outright.

Give every integration an owner. Unowned integrations are the ones that break silently, because the alert, if there is one, goes to a distribution list nobody reads.

Reconciliation as a standing process

Even well-built integrations drift. A record is deleted at one end, a manual correction is applied directly in a database, a failed message is never reprocessed. Over months, the two systems diverge in small ways that nobody has noticed.

Run reconciliation on a schedule: compare record counts and key totals between systems and report the differences. Daily for financial data, weekly for most other things. The point is that drift is found by a routine process rather than by a customer.

Decide in advance what happens when a discrepancy appears — who investigates, how a correction is applied, and whether anything is automatic. Automatic correction is appropriate for some categories and dangerous for others, and that distinction is worth making while nothing is on fire.