Bookmark this site!

2008-04-28

Fixing mail.app’s IMAP date problem

Using imapsync to migrate mail from one IMAP server to another, I hit a problem. Google led me to the following description of the same issue:

The IMAP Date problem is the result of how mail.app figures out the Date Received time for an email. Rather than using the Date: header in the email it uses the time the file was written to the file system. This becomes a problem when files are copied to a new location on the server and the creation time of the file is changed.

Not mail.app's problem!

The solution is to use imapsync with the flag --syncinternaldates

I am now using

/usr/bin/imapsync --syncinternaldates \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5
to finish the transfer of about 4GB of mail history.

To start, try it out on a small folder --folder Test

First test with --dry :

/usr/bin/imapsync --syncinternaldates  --folder Test \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5 --delete --dry
If everything looks OK then run
/usr/bin/imapsync --syncinternaldates  --folder Test \
     --host1 mail.inf.ed.ac.uk --user1 mfourman --ssl1 \
     --host2 imap.staffmail.ed.ac.uk --user2 mfourman -ssl2 \
     --noauthmd5 --delete
Expunge from the source by hand (in Mail.app Mailbox > Erase Deleted Messages).

Using the --expunge flag upsets other users of the IMAP server:

Currently it is processing folder which is 100MB in size, it is deleting (and then expunging) one message at a time. The way that is implemented on the server, is that the message is deleted, and then the whole file is written out again with that one message removed. This is leading to a lot of disk IO.

That's the short story. The actual story was more involved and also included using my gmail IMAP account as a temporary buffer to recover from some false starts. (Free 6.5GB quota - thank you Google!)

2 comments:

Unknown said...

My mail app does not open (The application Mail quit unexpectedly) Mac OS X and other applications are not affected. Click relaunch to launch the application again. Tried many times, but keeps on closing. Any suggestion!!!!

Mau

Michael Fourman said...

I've found that this can happen if your local copy of your email gets corrupted.

I leave mail on the server as well as keeping a local copy. Deleting the offending folder — e.g. ~/Library/Mail/IMAP-username@gmail.com — and then letting Mail rebuild it, works for me.