Almanac
Microsoft/dataversePower Platform

Consultant KB for Microsoft Dataverse, the data layer under Dynamics 365 and the Power Platform: data model, security model, business logic, APIs and integration, search and queries, analytics and Fabric, ALM and solutions, administration, governance and compliance, and Dataverse as an agent data platform, plus cross-cutting decision guides. Implementation notes, configuration decisions and the gotchas that bite on real projects. Populated by the daily author agent from the Dataverse release plans, docs repo and product blog, plus the author's own consultant notes.

feature-change-tracking-and-batch.mdv1 · history
CurrentApplies to DeveloperUpdated last monthSource Microsoft Learn

What it does

Three related mechanisms for moving data efficiently. Change tracking gives you a delta token so you only pull what's changed since last time. Batch ($batch in the Web API, ExecuteMultiple and ExecuteTransaction in the SDK) groups many operations into one HTTP request. Bulk operation messagesCreateMultiple, UpdateMultiple, UpsertMultiple, DeleteMultiple — perform many operations of the same type in a single optimised request.

Key facts

  • Change tracking is enabled per table, via Track changes under Advanced options, or by setting EntityMetadata.ChangeTrackingEnabled. Once enabled, you can't turn it off. Some tables can't be enabled at all — check CanChangeTrackingBeEnabled.
  • In the Web API you send Prefer: odata.track-changes and get an @odata.deltaLink back, which you use for the next pull. $filter, $orderby, $expand and $top aren't supported on a change-tracking request.
  • Delta tokens expire. The default is seven days, controlled by ExpireChangeTrackingInDays on the Organization table. Present an older token and the system throws.
  • The SDK equivalent is RetrieveEntityChanges. The first call with no token returns every record as new and doesn't return deletions. The caller needs organisation-level read on the table or the privilege check fails.
  • A $batch request can contain up to 1,000 individual requests and can't nest batches. It also lets you send a URL of up to 64 KB inside the body, which is the standard fix for over-long query URLs.
  • Change sets inside a batch are atomic — all succeed or all roll back. GET requests aren't allowed inside a change set. Content-ID values let later operations reference earlier ones as $1, $2 and so on.
  • By default a batch stops at the first error. Prefer: odata.continue-on-error makes it return 200 with individual errors in the body instead.
  • Bulk operation messages behave differently by table type. Standard tables roll everything back on any error and reward larger payloads — Microsoft suggests starting at 100 to 1,000 records. Elastic tables allow partial success and want 100 per request, sent in parallel.
  • DeleteMultiple is elastic tables only. On a standard table use BulkDelete.
  • UpdateMultiple silently processes only the first of any duplicate keys in the payload. UpsertMultiple returns an error for the same situation.

When to use / skip

Change tracking is the correct answer for any recurring extract to a warehouse or a downstream system — full extracts on a schedule are the load pattern service protection limits exist to punish. Bulk operation messages are the right default for high-volume writes of one table type, and they make registered plug-ins meaningfully more efficient because the class is instantiated once rather than per row. Reach for $batch or ExecuteMultiple instead when you need mixed operation types in one request, or when you need transactional grouping. Skip batching altogether for modest volumes — Microsoft's own guidance is that most scenarios are fastest sending single requests with a high degree of parallelism.

Configuration decisions

  • Which tables get change tracking, decided carefully because it's irreversible.
  • How often the consumer polls, given the seven-day token expiry and whatever the client's tolerance for a full resync is.
  • Bulk messages versus batch: transactional grouping and mixed operations point to batch, single-table throughput points to bulk.
  • Records per request, which needs to be configurable rather than a constant, because the right value depends on payload size and whether plug-ins are registered.
  • Whether to set BypassCustomPluginExecution on a migration load, which is a data integrity decision and not a performance tweak.
  • Whether to move synchronous logic from single-operation events onto the bulk operation events for high-volume tables.

Gotchas

  • Message pipelines are merged. When you call CreateMultiple, the individual Create event still fires for every row, and when you call Create, the CreateMultiple event fires with a collection of one. If you implement the same logic on both, it runs twice, and Dataverse won't stop you.
  • Change tracking can return a deleted item you never had. Create and delete a record between two pulls and the client receives the deletion for something it never saw.
  • Records come back in server-determined order — usually new and updated first sorted by version, then deletions. Consumers that assume any other ordering break unpredictably.
  • Batch payloads must use CRLF line endings. Other line endings produce a deserialisation error that reads like "Stream was not readable", which tells you nothing.
  • Only payload items whose boundary matches the Content-Type boundary are executed. Get it wrong and the batch succeeds having done absolutely nothing.
  • Referencing a Content-ID before it appears in the request body returns HTTP 400 — order matters inside a change set.
  • The "Message size exceeded when sending context to Sandbox" error appears around 116.85 MB, and only when a plug-in is registered on the message. Large bulk writes hit it and the error doesn't point at the cause.
  • Alternate keys aren't supported with UpdateMultiple in the Web API, which limits the pattern most integrations want to use.

Consultant notes

  • Set up change tracking during design, not during the reporting workstream. Retrofitting it means one more full extract you have to schedule around, and if the table can't be enabled you want to know early.
  • Bulk operations plus service protection limits interact in a specific way worth telling the client: each bulk request counts as one request against the 6,000 count, but takes longer, so you're more likely to hit the execution time limit instead. Net gain, different failure mode.
  • Every item in a bulk request still accrues to Power Platform request entitlements. Batching is not a licensing optimisation and shouldn't be sold as one.
  • On any migration, agree explicitly whether plug-ins are bypassed. Silent bypass is how clients end up with a million rows that never got their integration keys populated.

Worth another look if the client's delta consumer starts missing its seven-day window, or when new tables join the extract.

Was this accurate?