Skip to content

Incremental backup (delta sync) ​

The first backup of a mailbox, drive, or site reads everything. Every later backup reads only the messages and files that were created, changed, or deleted since the previous run. A nightly run therefore costs a small share of the Microsoft 365 requests, bandwidth, and time of the first one.

bash
atlas outlook backup -m user@example.com -s "$TENANT"          # incremental, from the saved sync checkpoints
atlas outlook backup -m user@example.com -s "$TENANT" --full   # ignore the checkpoints and read everything

How incremental backup works ​

At the end of each run, Atlas saves a sync checkpoint (a Microsoft Graph delta link) for every mail folder, drive, library, or calendar it finished. The next run sends the checkpoint back, and Microsoft 365 returns only the changes after it.

A checkpoint is a Microsoft Graph URL that carries an opaque sync token:

https://graph.microsoft.com/v1.0/users/.../messages/delta?$deltatoken=aGR2b...

Checkpoints are kept in the owner's delta cursor object, for example _meta/outlook-cursors/{owner_id}.json for a mailbox. Outlook also stores them in each snapshot manifest. See Storage layout for every workload's path.

Checkpoints are sensitive

A checkpoint holds tenant-scoped sync state. Anyone who obtains one could learn which items in the mailbox changed. Atlas stores checkpoints only inside cursors and manifests encrypted with the tenant data key, never in plaintext. Do not copy them into logs or tickets.

First run and later runs (Outlook)
  1. First run. For each folder, Atlas requests /users/{id}/mailFolders/{id}/messages/delta, including the message body. Microsoft Graph pages through every message. The final @odata.deltaLink is saved in the mailbox's cursor, _meta/outlook-cursors/{owner_id}.json, and in the snapshot manifest when the run writes one.
  2. Later runs. Atlas sends the saved link, and Microsoft Graph returns only the changes since the last sync.

When Atlas reads everything again ​

Atlas falls back to a full backup of a folder, drive, or library whenever its checkpoint can no longer be trusted. You never need to detect these cases yourself.

SituationWhat Atlas does
You pass --fullIgnores every saved checkpoint. Use it for a periodic audit, or when you suspect a checkpoint is damaged
The checkpoint expired or was rejectedMicrosoft Graph returns syncStateNotFound, resyncRequired, or syncStateInvalid. Atlas reads the folder in full and logs a warning
The previous run stored nothing (stale-delta safeguard)A saved Outlook checkpoint returns no items, but the previous manifest has no entries, so the earlier run was interrupted before it stored anything. Atlas discards the checkpoint and reads the folder in full
The snapshot the checkpoint belongs to is goneThe snapshot was deleted or quarantined, or the whole backup of the owner was removed. Resuming would skip every change it held, so Atlas reads every folder, drive, or library in full
First backup of a mailbox whose snapshots use mutable message IDsMixing ID formats breaks correlation, so this one run is full and records id_format: immutable in its manifest. It happens once per mailbox. See Immutable message IDs

Checkpoints expire after a period of inactivity, which breaks the incremental chain. When Outlook rejects a checkpoint after some of its pages were already stored, the full read skips those messages and keeps the removals they carried.

A full read is scoped to the folder, drive, or library that triggered it. Checkpoints are kept per drive, so a site with several libraries, or a user with several drives, keeps working checkpoints for the rest. Files in the drive read in full are recorded again as created. Files elsewhere keep their history, and an edit there is still reported as updated.

How a checkpoint is matched to its snapshot

Outlook, Contacts, and Calendar cursors record the snapshot that was newest when they were saved. When that snapshot is no longer the newest readable manifest, Atlas ignores the checkpoints and reads every folder in full.

OneDrive and SharePoint cursors record the snapshot they were saved with: the one the run wrote, or the newest one when the run changed nothing. A backup resumes the cursor only while that snapshot's manifest still exists. The check is one listing of the manifest prefix and does not read the manifests, because a drive has no pointer to its newest one. Once the manifest is gone, the run reads every drive or library in full, records a deletion for each file the remaining snapshots list that no longer exists, rebuilds the version watermarks from the version index, and warns. An owner or site with no snapshot at all, such as an empty drive, records that, and its cursor is resumed until a manifest appears.

Cursors written before 7.0 record no snapshot. They are trusted while any manifest exists, until the next backup rewrites them.

What a full backup costs ​

A full run, whether you asked for it with --full or Atlas fell back to one, costs what a first backup costs in Microsoft 365 requests and time. For Outlook, Microsoft Graph lists every message again at one delta page per 10 messages by default (--page-size), and Atlas fetches each message's MIME with one /$value request plus one reaction $batch per page. The content must be read before it can be hashed.

Storage barely grows. Each unique message or file is stored once, so content already in the bucket costs one HEAD request and no upload. The run always writes a snapshot, and for Outlook that manifest lists every message of the selected folders, so it is as large as the first backup's.

A full read records no deletions for Outlook, because Microsoft 365 reports what exists, not what was deleted. See Deleted and moved messages.

After a snapshot delete ​

delete -s also deletes the owner's delta cursor. What the next backup reads depends on the workload:

WorkloadNext backup after delete -s
Calendar, ContactsReads every calendar or folder in full, whichever snapshot was deleted. Each manifest is complete, so deleting an older snapshot loses nothing
OutlookResumes from the checkpoints in the remaining newest snapshot, so it reads only the changes since that snapshot, including those the deleted one held. Only the newest snapshot can be deleted, so those checkpoints are always the right base
OneDrive, SharePointReads every drive or library in full. Only the newest snapshot can be deleted, because newer ones are built on it. Every existing file gets an entry in the new snapshot, stored content is not uploaded again, and files the remaining snapshots list that are gone are recorded as deleted. See delete -s

delete -m, and delete without -s on OneDrive and SharePoint, remove the cursor with the rest of the owner's data, so the next backup is a full backup on every workload.

Deleted and moved messages (Outlook) ​

bash
atlas outlook backup -m user@example.com          # records messages deleted since the last run
atlas outlook restore -s <snapshot-id> -T user@example.com

An Outlook snapshot holds only the messages its run stored. restore -s, save -s, and verify -s therefore read the snapshot together with every older snapshot of the mailbox.

Microsoft 365 also reports messages deleted from, or moved out of, each folder. Atlas records them in the manifest as removed_object_ids, and a removal hides that message in every older snapshot of the chain. A message deleted before a snapshot is therefore not restored, saved, or verified from it. A run whose only change is a deletion still writes a snapshot, because the removal exists nowhere else once the checkpoint moves past it.

A moved message keeps its ID, because Atlas uses immutable message IDs. Microsoft 365 reports the move as a removal from the old folder and a new message in the new one. A manifest's own entries win over its own removals, so the message stays restorable at its new location.

When removals are recorded ​

Removals are only safe when the run saw both ends of a move. Atlas records them only when every selected folder finished:

RunRemovalsCheckpoints
Every selected folder finished, no --folder, no folder that can receive mail excludedRecordedAdvance
Interrupted, or a folder failedNoneEvery folder that reported a removal keeps its previous checkpoint, so the next complete run fetches and records the removals
A folder was deleted while the run listed foldersNone. The folder is skipped with a warning and recorded as deleted-during-listing, including Recoverable Items subfolders, because mail may have moved into itThe vanished folder keeps its previous checkpoint and stored messages. The next run no longer lists it and records removals as usual
--folder or --exclude-junkNone, and the removals it saw are droppedAdvance anyway, because a mailbox always backed up this way would otherwise never move forward

Excluding hidden system folders such as Conversation Action Settings, or the non-mail Recoverable Items subfolders, does not block removals, because mail cannot be moved into them. An unrecognised Recoverable Items subfolder does block them, because Atlas cannot tell what it holds.

A message moved into Junk Email during a run that records no removals therefore stays restorable from its last captured copy instead of disappearing from the chain. Messages fetched again after a withheld checkpoint deduplicate against the objects already stored.

Fix a folder that fails on every run

A folder that fails on every run holds back the checkpoints of every folder with pending removals until it succeeds. Removals stay unrecorded until then, so fix a persistent folder error rather than leaving it.

Deletions that are never recorded ​

These deleted messages still come back from the chain:

CaseWhy
A filtered or excluded run--folder or --exclude-junk. Checkpoints advance, so the removals it saw are not reported again
A full read--full, an expired checkpoint, or the stale-delta safeguard reads the folder again. Microsoft 365 reports what exists, not what was deleted. Removals read before Microsoft Graph rejected an expired checkpoint are still recorded
Snapshots written before Atlas recorded removalsThe field is absent, so nothing is hidden

Commit order for OneDrive and SharePoint ​

A OneDrive or SharePoint run writes three objects, always in this order:

  1. The snapshot manifest, if the run produced entries.
  2. The run's version index.
  3. The cursor with the new checkpoints.

A run with no entries skips step 1. Outlook follows the same rule: the manifest first, then the cursor. An Outlook run that captured nothing and found no deletions writes only the cursor.

This order makes a failed run safe to rerun. The checkpoint never moves past content that no snapshot references, which would let the next run report success while that content stayed unreachable. The worst case is repeated work: a crash after the manifest but before the cursor makes the next run read the same interval again and write a second snapshot. Content already stored deduplicates, so it costs only Microsoft 365 reads.

A run covering several drives or libraries commits all their checkpoints together, after the manifest. A finished drive never has its progress saved ahead of the snapshot that references it.

File version capture (OneDrive and SharePoint) ​

OneDrive and SharePoint backups keep earlier versions of each file as well as the current copy. For every file reported as changed, Atlas lists its versions with GET /drives/{drive-id}/items/{item-id}/versions and stores each version it has not captured yet. The current version is skipped, because the snapshot entry already holds it.

Atlas keeps a watermark per file in the cursor, so it knows which versions it already holds without downloading them. Versions are processed oldest first, and the watermark stops at the first version the run could not capture:

Outcome for a versionWatermarkNext run
Stored, or already stored as a deduplicated objectAdvances past itSkips it
404 or 410 from Microsoft GraphAdvances past itSkips it. The library's version policy removed it and it can never be retrieved
Any other error, such as 403 or 500Stops before itRetries it and every newer version

Nothing is dropped silently. A version with a missing or unparseable lastModifiedDateTime is never treated as captured: Atlas fetches it again when the file changes, and deduplication prevents a second stored copy.

Steady-state backups read only the cursor, which they read anyway, and do not read the version index.

Watermark format
json
{
  "owner_id": "2f1c9a04-7b3d-4e11-9c2a-6d5e8f0a1b2c",
  "delta_link_by_drive": { "b!x9Kp...": "https://graph.microsoft.com/v1.0/..." },
  "version_watermark_by_file_id": {
    "01ABCDEF...": {
      "last_modified_at": "2026-08-24T09:12:44Z",
      "version_ids": ["7.0", "8.0"]
    }
  }
}
  • last_modified_at skips every version older than the timestamp.
  • version_ids skips only the versions already captured at exactly that timestamp.

Microsoft Graph reports lastModifiedDateTime to the second, so two versions can share a timestamp. A timestamp alone could skip a version created later in the same second, so Atlas also keeps the IDs captured at the newest timestamp and drops older IDs. Version IDs are compared for equality only. Atlas never parses or sorts them, because Microsoft Graph does not guarantee their format or order.

A watermark stores one timestamp and the IDs at that timestamp, not the full history.

--full, and cursors without watermarks ​

--full does not reset watermarks. It reads every file again, but the version history Atlas already holds stays valid. To fetch the version history again on purpose, delete the version index and the cursor for that owner or site.

A cursor written before watermarks existed has no watermark map. The first run that reads it rebuilds the map from the version index, logs Seeding version dedup watermarks, and writes exact state to the cursor. A cursor with timestamp-only values upgrades each file when that file next changes, and versions that share the timestamp are captured once more during the conversion.

Immutable message IDs ​

Atlas sends Prefer: IdType="ImmutableId" on every Outlook request: delta pages, message reads, and attachment fetches. With immutable IDs a message moved between folders keeps its identity, so only its changed folder metadata is stored again.

The default Outlook ID changes when a message moves. A move into an archive folder would then look like a deletion and a new message: Atlas would download content it already holds, and older snapshot entries would point at IDs Microsoft 365 no longer resolves. Microsoft requires the header on every request that handles an ID, because mixing the two formats breaks correlation.

Immutable IDs are stable within one mailbox, not across mailboxes. They also change when an item moves into an In-Place Archive or is exported and imported again. See Obtain immutable identifiers for Outlook resources.

Original MIME retrieval ​

For each new or changed message, Atlas stores the original RFC 5322 MIME bytes as the message's encrypted object:

GET /users/{id}/messages/{id}/$value

The original bytes keep what the JSON form of a message does not expose: the Received: chain, Authentication-Results (DKIM, SPF, and ARC), In-Reply-To and References threading, and S/MIME payloads. See Backup fidelity.

MIME is the only format for new snapshots. There is no --fidelity option.

Request cost ​

MessageRequests beyond the delta page
No attachmentsOne /$value request
With attachmentsOne /$value request. Attachments are inside the MIME and cost nothing extra

Incremental runs fetch MIME only for new and changed messages. The first backup of a large mailbox is the expensive case, and the retry policy absorbs the throttling it causes.

When a message cannot be stored as MIME ​

SituationWhat Atlas does
Microsoft 365 cannot produce MIME for the messageStores the legacy JSON form and records the format in the snapshot entry. The run succeeds
The MIME request still fails after retries, for example a 503Stores the JSON form so the message is kept, reports it as an error of its folder, keeps the folder's previous checkpoint, and exits 2. The next run fetches the folder's changes again, captures the MIME, and its entry replaces the JSON one
A JSON entry's attachments could not be fetched or storedHandled the same way: the next run fetches the message and its attachments again

Entries with payload_format: "mime" hold original bytes. Entries without the field hold legacy JSON. A snapshot can mix both, and save, read, restore, and verify handle both. A message that fails on every run holds its folder's checkpoint back until it succeeds, and the changes fetched again each time deduplicate against the stored objects.

Reactions, flag, and pin ​

Atlas captures each message's follow-up flag, Outlook reactions (like, heart, celebrate, and so on), and pin, and stores them in the message's snapshot entry:

PropertySnapshot entry fieldStored when
Follow-up flag status, with start, due, and completed datesflagThe message is flagged. A MIME entry needs it because MIME carries at most an X-Message-Flag header
Reactionsreaction_propertiesThe message has reactions
Pinpin_propertiesThe pin property 0x0F01 is present, which it is not on a message that was never pinned

Adding a reaction or a flag updates the message's lastModifiedDateTime, so the next incremental run picks it up as a changed message.

The flag arrives with the delta page. Reactions and the pin need one extra read per message:

  • A throttled or failed read is retried for that message alone.
  • A read that still fails backs the message up without its reactions and pin. The run summary prints a warning per folder with the count. The checkpoint moves on, so those properties are read again only when the message next changes. The flag is not affected.
  • A permission or licensing failure stops the mailbox, as it does for MIME.
  • If the configured mailbox connector cannot read these properties at all, Atlas logs that once and captures none.
How reactions and the pin are read

Reactions and the pin live in MAPI properties, not in the MIME and not in the delta payload. Outlook pins a message by setting its renew times, SystemTime 0x0F02 and SystemTime 0x0F01, to a far-future date. Microsoft Graph rejects the $expand that would read them on the delta query with 400 BadRequest, so Atlas reads them for each delta page in one $batch of up to 20 requests:

GET /users/{id}/messages/{id}?$select=id&$expand=singleValueExtendedProperties($filter=...)

The requests in a batch are chained with dependsOn, so Exchange runs them one at a time and the batch never exceeds the four concurrent requests a mailbox allows. Every message still counts once against the per-mailbox budget, because Microsoft Graph evaluates each request in a batch on its own. The batch saves round trips: at the default page size of 10, a first backup of N messages makes N /$value requests plus N/10 batch round trips. At --page-size 100 it is N/20.

Every message in a page is read, including messages without reactions and messages whose content deduplicates, because reactions and the pin belong to the snapshot entry rather than to the stored object.

Retry and error handling ​

Atlas retries 429 Too Many Requests (honouring Retry-After) and 500, 502, 503, 504 on its own, up to 12 times. 501 and every other 4xx are not retried, because the same request returns the same error. Retries and fail-safes lists every parameter. For incremental backup, two outcomes matter:

ResponseEffect on the checkpoint
syncStateNotFound, resyncRequired, syncStateInvalidThe checkpoint is no longer valid. Atlas reads the folder in full
Retries exhausted on a folderAtlas marks the folder as failed and continues with the remaining folders. The failed folder keeps its previous checkpoint

Stopping a run ​

ActionEffect on checkpoints
First Ctrl+CCompleted folders and their checkpoints are saved to a partial manifest, and the command exits 2. The dashboard marks interrupted folders. The next run resumes from those checkpoints, and an interrupted folder is read again from its previous one
Second Ctrl+CExits at once with 130. No manifest is written, so no checkpoint moves. Objects already uploaded stay in storage and deduplicate on the next run. The lease is not released, so a backup of the same mailbox started within 5 minutes exits 12

A partial manifest from the first Ctrl+C is a valid snapshot. Microsoft Graph has no cancel, so a request it already accepted still completes on its side, and Atlas drops the result. Ctrl+C and signals covers the other commands.

See also ​