What is schema drift in an integration?

Schema drift is a change in the structure of data an integration receives or produces. A field can appear, disappear or change type. Your integration needs a policy for what it accepts, what it rejects and what requires review.

Use a customer export as an example. Yesterday its customer reference was a text value. Today an upstream change sends a nested object instead. A parser may reject it immediately. Another implementation may convert it into text and keep running, leaving a destination value nobody intended.

The second outcome can be harder to notice because the workflow still appears active.

Schema drift and ordinary data changes

A new customer name is an ordinary value change. A name field becoming a collection of localized values is a structural change. The receiving logic needs to understand which one occurred.

Microsoft's schema-drift documentation for Azure Data Factory describes fields and types being added, removed or changed, along with the tradeoff between flexible handling and an explicit known schema. The right acceptance policy depends on what your destination does with the data.

For our customer export, a new unused field may be harmless. A changed customer identifier can affect which business record gets updated. Treating both changes identically would ignore the consequence of the field.

Start with the contract you need

List the fields that establish identity, authorization scope and business meaning. Define their expected type and whether they can be missing.

Then decide which extra fields the flow can ignore. Accepting an additional description field may be reasonable if it cannot affect the destination operation.

This gives flexibility a boundary. The integration does not have to fail on every upstream addition, but it should not quietly reinterpret a critical identifier to keep the run green.

Keep the contract close to the mapping. A field can be structurally valid while carrying a business meaning the destination should not accept.

Conversions need visible rules

A number represented as text can look easy to fix. The conversion still needs a rule for empty values, separators and leading characters.

An item code is a useful example. Converting it to a number can remove zeroes at the beginning that belong to the identifier. The result might be valid as a number and wrong as an item reference.

The integration should preserve the meaning the business agreed. A permissive conversion that makes the payload pass is not enough evidence that the destination record is correct.

When a conversion is intentional, record it in the mapping and test representative cases. That gives a later investigator a reason for the behavior rather than a mysterious helper function.

Version changes and account customization

An API version is one source of structural change. Custom fields and account configuration can create differences even when the public API version stays the same.

Check the actual account and action used by your flow. A sample response from a generic guide does not establish the complete shape of a particular customer's records.

For a multi-customer product, this means the test plan needs to include the account-specific differences that matter. An optional field in one account can become operationally important in another process.

Do not turn that into an excuse to accept arbitrary input. Establish which variation is supported and what happens when a record falls outside it.

Make the response fit the consequence

For a harmless extra field, ignoring it may be the right choice. For a changed financial amount or identifier, stopping the affected operation for review may be safer than guessing.

Document the behavior so the operator can distinguish an unsupported change from a temporary application failure. They call for different next steps.

Preserve enough evidence to investigate without scattering private payloads through general logs. The team needs to identify the record, the contract mismatch and the relevant version or configuration.

After the business accepts a change, update the mapping and regression checks together. Otherwise the new behavior can become another undocumented exception.

Test the changes that can stay quiet

Include a missing optional field, a missing required identifier and a value with a different type. Add a structurally valid value with changed business meaning.

For the customer example, check whether the destination still updates the intended customer and whether the integration stops when it cannot establish that identity.

The quiet version has shown up on our own projects. NetSuite items arrived without the unit of measure the flow needed, and a team member cleared the resulting errors by hand for hours every week. Nothing about the run looked dramatic. The missing field was the problem.

A test that checks only for an error misses the quiet failure. Inspect the destination result and the evidence available to the operator.

List the fields your flow cannot afford to misread. Give each one an accept, reject or review rule and test the quiet changes first.

Read the docs