Fixing mail.app’s IMAP date problem

Using imapsync to migrate mail from one IMAP server to another, I hit a problem. Google led me to the following description of the same issue:

The IMAP Date problem is the result of how mail.app figures out the Date Received time for an email. Rather than using the Date: header in the email it uses the time the file was written to the file system. This becomes a problem when files are copied to a new location on the server and the creation time of the file is changed.

Not mail.app's problem!

The solution is to use imapsync with the flag --syncinternaldates

I am now using

/usr/bin/imapsync --syncinternaldates \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5

to finish the transfer of about 4GB of mail history.

To start, try it out on a small folder --folder Test

First test with --dry :

/usr/bin/imapsync --syncinternaldates  --folder Test \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5 --delete --dry

If everything looks OK then run

/usr/bin/imapsync --syncinternaldates  --folder Test \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5 --delete

Expunge from the source by hand (in Mail.app Mailbox > Erase Deleted Messages).

Using the --expunge flag upsets other users of the IMAP server:

Currently it is processing folder which is 100MB in size, it is deleting (and then expunging) one message at a time. The way that is implemented on the server, is that the message is deleted, and then the whole file is written out again with that one message removed. This is leading to a lot of disk IO.

That's the short story. The actual story was more involved and also included using my gmail IMAP account as a temporary buffer to recover from some false starts. (Free 6.5GB quota - thank you Google!)