17 November 2026 · 4 min read · EximFetch team

Keeping customs master data in sync: updatedSince, change logs and webhooks

Code lists and rate tables change on their own schedules, and a stale copy files wrong documents silently. A pattern for keeping a local copy current: one bulk load, incremental pulls, a change log that includes removals, and alerts instead of polling.

Every filing system has master tables: customs locations, port and airline codes, ITC-HS lines with their policy, currency codes, exchange rates, drawback serials. They were loaded from a government file at some point and they have been drifting since. A location that closed last year still validates; a line that became Restricted last month still files as Free. Nobody notices until a Shipping Bill is rejected or, worse, accepted and queried later.

This article describes a sync pattern that keeps a local copy current with four mechanisms, using the EximFetch API as the example, because that is what it was designed for.

Why "re-download everything nightly" is not enough

Re-pulling a whole dataset every night and replacing the table works for a 600-row location list. It does not work for 12,000 tariff lines with policy text, or 155,000 PIN codes, and it has a worse problem than size: it tells you nothing. The table changed, but which rows? Was a location removed? Did a policy flip from Free to Restricted on a code you ship every week? The application needs the delta, with reasons, not a fresh copy.

Mechanism 1: one bulk load

Start with a complete copy. On the Premium plan /export/{dataset} streams a whole dataset as CSV, JSON or NDJSON in one call that counts as one request:

curl -L "https://api.eximfetch.com/export/icegate-locations?format=ndjson" \
  -H "Authorization: Bearer $EXIMFETCH_API_KEY" -o icegate-locations.ndjson

On other plans, page through the endpoint with limit=500. Either way, record the time you finished.

Mechanism 2: incremental pulls with updatedSince

Every record on every dataset carries two timestamps: fetchedAt, when the source was last read, and updatedAt, when the record's content last changed. The second is the one that matters. A dataset can be refreshed daily and change once a year; updatedAt only moves when a field actually differs.

Every endpoint accepts updatedSince=<ISO date-time> and returns only records whose content changed after that instant:

curl "https://api.eximfetch.com/icegate-locations?updatedSince=2026-11-01T00:00:00Z" \
  -H "Authorization: Bearer $EXIMFETCH_API_KEY"

Store the time of the last pull, ask again on a schedule, upsert what comes back. A quiet dataset returns an empty page and costs one call. For most tables, once a day is plenty; the exchange-rate tables are the exception, and they have their own alert.

Mechanism 3: the change log, for removals

updatedSince has a blind spot: a record that was removed is not returned, because it is not there to return. For master data, removals matter as much as additions; a closed customs location that stays in your dropdown is a rejected document.

/changes is one row per record added, changed or removed by any refresh, newest first, with the record key and, for additions and changes, the new values:

curl "https://api.eximfetch.com/changes?dataset=icegate-locations&since=2026-11-01T00:00:00Z" \
  -H "Authorization: Bearer $EXIMFETCH_API_KEY"

Filter with change=removed to get only deletions, or dataset= for several datasets at once. For the ITC-HS policy there is a dedicated feed, /policy-changes, that lists lines whose import or export policy was added, changed or removed, filterable by code prefix, so a compliance desk can watch only the chapters it trades.

The same change log drives the public status page, which shows for every dataset when it was last refreshed, when it last changed, and whether the government source is currently failing.

Mechanism 4: webhooks instead of polling

Polling is fine for daily master data and wrong for anything time-sensitive. Under Settings, Alerts, you set one HTTPS URL and choose events:

  • a new CBIC exchange-rate notification, with the rates in the payload;
  • a dataset refresh that changed records, with the counts;
  • Shipping Bill events, if you track bills.

Every delivery is signed. The X-EximFetch-Signature header carries a timestamp and an HMAC-SHA256 of the timestamp and the raw body under your signing secret; verify it and reject anything older than a few minutes. Failed deliveries are retried at 1, 5 and 30 minutes and then hourly for a day, and an email fallback can run alongside. The API docs have the payload and verification code in Node and Python.

Putting it together

Data Load Keep current Alert
Locations, port and airline codes, currencies Bulk once updatedSince daily, /changes for removals Dataset refresh
ITC-HS with policy Bulk once /policy-changes daily on your chapters Dataset refresh
Customs exchange rates History once updatedSince after each notification Exchange-rate notification
Drawback, tariff, anti-dumping, FTA rates Bulk once updatedSince weekly Dataset refresh
Tariff values, export duties Small; pull whole updatedSince daily Dataset refresh

Two small clients, for Python and Node, wrap the paging, updatedSince, the change log and the webhook signature check; they are on the docs page.

Rules for software teams

  1. Upsert by the record key, never by position. The key is in every change-log row.
  2. Process removals. Mark the row inactive rather than deleting it, so historical documents still resolve.
  3. Store updatedAt and the source with every row. Audits ask where a code came from and when it changed.
  4. Verify webhook signatures and deduplicate on the delivery id; a delivery can arrive twice.

Summary

Keep a local copy current with one bulk load, daily or weekly pulls with updatedSince, the change log for removals and policy flips, and signed webhooks for the time-sensitive events. Every record carries when its content last changed and where it came from, so the sync is a delta with reasons, not a nightly re-download.

Published 17 November 2026, updated 17 November 2026.