Docs / Obsidian Mail Server / People

Migration from Exchange and IMAP

Coming in the next release: Obsidian Mail Server 0.9.0 does not include migration from Exchange or IMAP yet.

Mailboxes can come from Microsoft Exchange (everything below) or from any IMAP server, such as Gmail, cPanel and other hosting, or Dovecot (From an IMAP server).

People > Migration moves mailboxes from your own Microsoft Exchange server (2013, 2016 or 2019) to this server. Exchange Online (Microsoft 365) is not supported yet: it only accepts modern (OAuth) sign-in. It works like Exchange's own migration batches: everything is copied once, then whatever changes keeps being copied, while people go on working in Outlook as before. When you are ready you complete the batch and point mail here.

What comes over:

• Mail: every mail folder (Inbox, Sent Items, Drafts, Deleted Items, Junk Email, Archive and every folder people made, with their subfolders), with each message's read state, flag, categories, replied/forwarded mark and received date.
• Calendars: the calendar and any other calendars people made, with recurring meetings and their changed or cancelled occurrences, attendees and their responses, the organizer, reminders and time zones. Meetings keep their identity, so an update or cancellation the organizer sends later still finds the meeting.
• Contacts and tasks, in their folders.
• The online archive (In-Place Archive), into the person's archive here (switched on when needed).
• Settings: automatic replies (as plain text), the category list with its colors, and inbox rules (see below).

What does not: notes, the journal, contact groups (distribution lists in a contacts folder), calendars Exchange makes by itself (birthdays), attachments on calendar items, and folder permissions and delegates (set those up again here under Shared access). The mailbox page shows each folder, whether it was copied and why not, and each item left out.

Before you start

1. The people must exist here. A batch copies into existing mailboxes. Create them under Users & groups, or connect Active Directory first, which creates everyone in one go.
2. The domain must be an accepted domain here (Domains), but do not change your MX record yet.
3. A service account on Exchange that may open every mailbox. In the Exchange Management Shell:

   New-ManagementRoleAssignment -Name "Obsidian migration" -Role ApplicationImpersonation -User svc-migrate

   This gives the account access to mailboxes through Exchange Web Services only, not the right to sign in to Windows or change anything in Exchange. Remove it again when the move is done: Remove-ManagementRoleAssignment "Obsidian migration". Without such an account, each mailbox can sign in with its own password instead (choose Each mailbox signs in with its own password, and give the passwords when you create the batch).

Connect

Migration > Connect to Exchange asks for:

• EWS address: usually https://mail.example.com/EWS/Exchange.asmx (the same name Outlook on the web uses). Giving just https://mail.example.com adds the rest.
• Service account: DOMAIN\svc-migrate or [email protected], and its password (stored encrypted, never shown again).
• Sign-in method: Windows (NTLM), Exchange's default. Choose Basic only if the Exchange server allows it.
• Mailboxes at a time: how many mailboxes copy at once. Exchange slows down accounts that ask too much (throttling); this server waits when told to, but 4 is gentle on a single Exchange server.

Test signs in and opens one mailbox. Exchange usually has a certificate from the company's own CA, which this server does not know: then the test shows the certificate. Trust this certificate accepts exactly that one (test again when it is renewed); Trust this CA with the company CA certificate pasted accepts its renewals too.

Common test results:

Batches

A batch is a group of mailboxes that move together, for example a department. New batch asks for:

• Mailboxes: one per line, the address here. If the address on Exchange is different, add it after a comma: [email protected], [email protected]. With per-mailbox sign-in, the Exchange password comes third.
• Sync interval: after the first copy, how often changes are copied (every hour is usual).
• Leave out Deleted Items / Junk Email / the online archive: those are not copied.
• Do not copy settings: automatic replies, categories and inbox rules stay as they are here.

Settings are copied after the first full copy of the mail and once more in the final sync. Inbox rules are copied only when every condition means the same here: a rule copied without one of its conditions (say "flagged for follow-up") would act on more mail than it did, so such rules are listed as not copied, with the reason. Actions with no equivalent here (marking importance, text-message alerts, replying with a template) are left out of a copied rule, and the mailbox page says so. A rule that moves mail to a folder needs that folder to be copied too.

Starting the batch copies each mailbox. A big mailbox takes a while, mostly depending on how fast Exchange answers. The batch page shows each mailbox's progress (items copied of the items Exchange reports), and clicking a mailbox shows its folders, which are copied and which not and why, and any message that could not be copied.

While a batch is syncing:

• New mail, deleted mail, moved mail, read/unread, flags and categories in Exchange are all copied on the next sync. A message moved between folders is copied again into its new folder and removed from the old one.
• Folders made, renamed, moved or deleted in Exchange are mirrored too.
• Do not use the mailboxes here yet: anything changed here can be overwritten by the next sync, and mail sent from here is not in Exchange.
• Stop pauses the batch (a running sync stops after its current step); Resume carries on where it stopped.
• A mailbox that fails (a wrong password, Exchange unreachable) is tried again every 15 minutes; the reason is shown.

Cutover

When every mailbox is In sync:

1. Complete the batch. Each mailbox gets one last sync; then the batch shows Completed and stops syncing.
2. Point mail here: change the MX record of the domain to this server (see Domains and DNS), and the autodiscover record, so Outlook and phones find this server.
3. Mail that still reaches Exchange during the DNS change (records are cached for up to their TTL) is not copied after completion. Lower the MX record's TTL a day before, or complete the batch an hour after switching MX.
4. Outlook and phones: give Outlook for Windows a new profile for the same address, and set phones up again (see Connecting phones, Macs and apps). The mail is already here when they connect.
5. Remove the ApplicationImpersonation assignment on Exchange, and later the batch here (the copied mail stays).

From an IMAP server

Choose IMAP as the kind of server in Connect to the old mail server. IMAP carries mail only, so calendars, contacts, rules and automatic replies stay behind: export those from the old service by hand if people need them.

• IMAP server: imaps://imap.example.com (port 993, encrypted from the start; the usual choice) or imap://imap.example.com (port 143, encrypted with STARTTLS). Unencrypted IMAP is never used.
• How to sign in: usually each mailbox signs in with its own password: give each mailbox's sign-in name and password in the batch ([email protected], [email protected], her-password). Servers with a master user (Dovecot, Cyrus) can sign in for every mailbox with one administrator account instead: the master user signs in for the mailbox with SASL PLAIN (its authorization identity).
• Gmail and Google Workspace: server imaps://imap.gmail.com. Each person turns IMAP on in Gmail's settings and creates an app password (it needs 2-step verification); use that, not their normal password. Gmail labels are folders, so a message with two labels is copied into both. All Mail, Starred and Important are views of the same messages and are left out, otherwise every message would arrive twice.
• cPanel and most hosting: server imaps://mail.<your domain> and each mailbox's full address and password.

What comes over: every folder, with its subfolders. Inbox, Sent, Drafts, Trash, Junk and Archive land on the folders here when the server marks them (most do). Each message comes over byte for byte, with its received date, its read, replied, forwarded and flagged state, and its keywords (labels and tags from Thunderbird or Apple Mail). Later syncs copy new messages, read and flag changes, and deletions. A folder renamed on the old server arrives as a new folder, and its old copy here is removed. When a server renumbers a folder (rare, after a repair), its messages are matched by their Message-ID and are not copied twice.

PST import and export

People > Import & export brings an Outlook data file (.pst) into a mailbox, or saves a mailbox as one, like Exchange's mailbox import and export requests. Both run in the background; the page shows each request's progress, and you can leave it meanwhile.

Import. Choose the mailbox and the file. The upload goes in pieces, so a large file (tens of gigabytes) gets through proxies and resumes if the connection drops. The file is checked before anything is imported.

• Without a target folder, the PST's folders merge into the mailbox's own: Inbox into Inbox, Sent Items into Sent Items, Calendar into Calendar, Contacts and Tasks likewise; other folders are created where they were (or reused when a folder of that name is already there).
• With Below folder (Imported/Old laptop), everything goes below that folder instead, and nothing mixes with the mailbox's own folders.
• Mail keeps its read state, flag, categories, received date, attachments and attached messages; appointments, contacts and tasks become real calendar items, contacts and tasks. Senders and recipients from an Exchange mailbox's PST (shown there only as Exchange addresses) are matched to people here by name; others keep their name with Exchange's encapsulated address (IMCEAEX-...).
• Folders Outlook hides (Recipient Cache and the like), the Outbox, Sync Issues and search folders are left out. Include Deleted Items is on unless you clear it.
• A restart, or Retry after a failure, continues where the import stopped; nothing is imported twice. Running the same PST in again does import it twice.

Export. Choose the mailbox; when the request is Completed, Download the file and open it in Outlook (File > Open & Export > Open Outlook Data File). It holds every folder of the mailbox with its mail, calendar, contacts and tasks, as Outlook sees them on this server. Include hidden folder data adds the views, settings and rules Outlook keeps in folders. Exports and downloads are recorded in the change history, as eDiscovery exports are.

Mailbox bundles move a mailbox to another Obsidian server with nothing lost. Choose Mailbox bundle (.omsbox) as the export format; on the other server, create the person and import the file into their mailbox the same way as a PST. A bundle holds the mailbox exactly as this server keeps it: every message's original content byte for byte, with its read state, flags, categories and received time; calendar items, contacts and tasks as they are; the hidden folder data Outlook keeps; the online archive (switched on at the other end when needed); and the settings: categories with their colors, the automatic reply and inbox rules (a rule that moves mail to a folder points at that folder there). Bundles from a newer server version are refused until the receiving server is updated.

Files stay on the server (under the data directory, pst/) until you Remove the request. A request runs on the server that received the upload; with several servers, open the admin center on that one to download an export.

Outlook 97-2002 (ANSI) PST files can be imported too. Files protected with a PST password import as well (the password was never encryption); files protected by Windows Information Protection cannot be read.

From the command line

oms migration endpoint set example "Exchange 2019" --url https://mail.example.com/EWS/Exchange.asmx --user EXAMPLE\svc-migrate
oms migration endpoint test example "Exchange 2019" [email protected]
oms migration batch new example "Wave 1" --endpoint "Exchange 2019" --mailboxes [email protected],[email protected] --start
oms migration batch show example "Wave 1"
oms migration batch complete example "Wave 1"

An IMAP source: oms migration endpoint set example "Old hosting" --kind imap --url imaps://mail.example.com --access user.

--csv file instead of --mailboxes reads lines of mailbox,address on the old server,login,password. The password of endpoint set is asked for (or read from standard input), never put on the command line.

PST files:

oms pst check /tmp/anna.pst                                  # is the file sound, and what is in it (like ScanPST)
oms pst import [email protected] /tmp/anna.pst --folder "Imported/Old laptop"
oms pst export [email protected]                              # add --bundle for a mailbox bundle
oms pst list example                                         # progress; a completed export's file path
oms pst cancel|retry|remove <id>

pst import reads the file where it is (the service needs read access to it) and never moves or deletes it.