What is a sync watermark?

A sync watermark is a saved position used to decide which data an integration should consider next. It can be a timestamp, a sequence value or another position your source defines. Its meaning matters more than its format: the next run uses it to decide which work has already been covered.

Suppose you copy changed customer records into another system. After each completed slice, you keep a position for the next run. If that position moves past work that did not finish, a later run can skip records the destination never received.

This is a progress contract, so write down what progress means.

A watermark and an order date

An order date describes the business event. A change timestamp describes a modification, if the source maintains it that way. They answer different questions.

Filtering by order date may find new orders while missing an older order whose address changed today. For an ongoing update flow, check which source field or cursor represents the changes you need.

Microsoft's incremental-copy tutorial for Azure Data Factory illustrates reading an old watermark, establishing a new boundary and saving progress for a later run. That is a reference pattern, not a claim about every source API or MindCloud action.

Define the slice before moving it

For the customer example, decide the lower and upper boundaries of the current slice. Keep those boundaries stable while processing the slice so the operator can explain which changes it was meant to include.

If you keep asking for everything after the latest observed time, a changing source can make the scope hard to reconstruct. A fixed slice gives the run a reviewable target.

Record how boundary equality works. A source that returns timestamps with limited precision can place several records at the same time. An implementation that assumes every timestamp identifies one change can miss a record at the boundary.

The right treatment depends on the source. It might require a secondary identifier, a documented cursor or an intentional overlap with safe repeated processing. Check the actual contract rather than choose an approach from the field name alone.

Our published NetSuite readiness checklist turns this into a test. Run enough data for several pages, include records that share a timestamp, change records during the run and restart it midway. The pass mark is that every intended record is accounted for, and the checkpoints and overlap handling leave no gaps and no uncontrolled duplicate effects.

Advance after the agreed work finishes

The saved position should reflect the completion rule the integration promises. Reading a page, writing a destination record and confirming the whole slice are different events.

Imagine that the source page was read successfully but the destination rejected a customer. If the saved watermark now says the entire slice is complete, the next run may never reconsider that customer.

You need a defined relationship between progress and unresolved work. One design might hold the boundary until the slice is handled. Another might retain a separate durable record of unfinished items. Neither should quietly lose the rejected record.

Make that relationship visible in the operating guide. The person responding to an incident should know whether the next run will revisit an exception or whether it requires another recovery path.

Late records need a deliberate policy

A source can make a record visible later than the timestamp it carries. It can also change how timestamps behave when imports or corrections occur. Those source details affect whether a simple increasing filter is enough.

Test a late-visible record and an older record changed after the previous run. Check whether both become eligible under the chosen rule.

If the source cannot provide a reliable change position, do not invent one by assuming a business date is equivalent. You may need a bounded re-read, periodic reconciliation or another supported change mechanism.

The important part is that the limitation is known. A timestamp field can look precise while failing to represent the event the integration needs to observe.

Keep progress scoped to the work

A position for one customer account does not automatically describe another account. A position for one object type may not describe another type with a different update model.

Include the relevant account and resource in the progress design. If a connection or scope changes, decide whether the saved position remains valid.

Also keep backfill progress distinct from ordinary processing when they cover different ranges or business rules. Reusing a position without establishing its meaning can make both jobs harder to recover.

Write the completion rule for one flow before the next run depends on it. Then check the cursor behavior for your source in the docs.

Read the docs