Mail Folder Gets Corrupted with Thunderbird Daily
I am running Thunderbird Daily. I had a problem recently where a number of my mail folders got corrupted; mail messages were missing. So, what I did was to restore from a good backup before the corruption the folder and folder.msf files to a new directory. One corrupt folder is my Sent folder. I then copied the restored Sent and Sent.msf to my TB Local Folders directory as Sent-restored and sent-restored.msf (while TB was not running). I started TB and looked at the sent-restored folder. The messages are there, and they are intact. I then select ~2400 mail messages from Sent-restored and copy into Sent. When I look at the Sent directory, I see that all of these messages have been copied, and the directory seems OK - all the correct dates, recipients, and subjects. But ALL of the messages I copied now have the same text - the text of one (I believe) of the messages.
I have tried this twice, with the same results. What can I do to debug/correct this? Thanks.
Modified by Wayne Mery
All Replies (20)
Is Sent an IMAP or POP folder? If it's IMAP, uploading 2400 messages at once is likely to fail, apart from issues present in Daily or caused by external factors such as AV scanning. Try copying a few dozen at a time and see if the process completes. Also, when restoring an mbox file to Mail/Local Folders, omit the msf file and have TB recreate it when it's restarted.
My connection to the mail server is POP3. But what I am doing does NOT contact the mail server. I am copying mail messages from one mail folder in TB to another mail folder. As I do not know the internals of the mail folder and the .msf [index] file, I decided that I would let TB copy the messages from one folder to another. I am assuming that the Sent folder is no different than any other mail folder in its internal format; it is just a collection of mail messages with an index.
If the Sent folder is corrupted, adding messages from Sent-restored isn't going to change that. The corrupted Sent and Sent.msf file should be deleted, then when TB restarts, a new Sent folder for the POP account will be created when you send the first message, and the new Sent should be not corrupted. Then, copy the messages from Sent-restored to Sent.
What I meant by corrupted is this - a large number of messages mysteriously disappeared from some of my mail folders. The mail folders (and .msf files) were "intact"; I had no trouble accessing any message that was still in the folder. So I thought that if I copied the missing mails from another "intact" mail folder, this would fix the problem. If I were to go and delete these 2400 mail messages I added, then my Sent folder would still be "intact" - still accessible but missing mail messages.
If I delete the Sent and Send.msf files, then, obviously for this file, TB will create a new Sent folder and include any new mail that I send. But it will not have my previously sent mail, and I will have to copy those messages from another folder. And it will not have mail that I have sent in the days since I took the backup from which I restored the Sent and Sent.msf files. Assuming I do that and TB creates a new Sent folder, how is my recovery process different from what I have done twice?
I'm lost here. If you need to retain the messages in the currently corrupted Sent folder before deleting the mbox and msf file, first copy them to another folder. But don't try to copy them back after the new Sent folder is created. That is the advice given in the cited kb article.
I do not understand your reply. First, the article you reference refers to compacting folders, and I am not compacting folders; TB may do it for me when it determines that a folder needs compacting. If I have two mail folders that appear to be non-corrupt - I can view any mail message in the folder, and it appears intact - then I should be able to copy/move messages from one folder to another. I have done this in the past when I have saved a message or a small group of messages into the wrong folder. What I am trying to do, essentially, is the same thing. except with a larger number of mail messages.
I just deleted 2,490 messages from my Sent folder. Then I did an Exit. Then I restarted TB Daily.my sent folder has intact messages
03/15/2020 08:26AM 08/12/2021 01:42PM
All of the messages between these two timestamps are gone, as expected. From this I conclude that the Sent and Sent.msf files are not corrupted. The Sent and Sent.msf files are from a backup, and I know that during the backup process I am not sending mail, so these two files should be un-corrupted. I then looked at the Sent-restored folder, and a few messages that I checked seemed intact. Then I copied the first of the missing messages from Sent-restored to Sent, and it is intact in the Sent folder.
Then I copied the next ten messages from Sent-restored to Sent. The first message copied correctly, and the other nine were corrupt - they had the same text. The text is a reply to a message 8/5/2011, which I determined is the FIRST message in my Sent folder. But what is displayed in the TB "index" is the date-time-sender-subject of the message that was moved.
Is the non-.msf file a real mbox file? It looks like one. What happens if I have no .msf file? Will TB re-create it?
I have no idea why, when I copy (or move) more than one message from a seemingly intact Sent-restored folder to the seemingly intact Sent folder, TB copies (or moves) the first message correctly but for the other messages in the collection, the date-time-sender-subject appears normal, but the body of the mail is the body of the first mail in the Sent folder.
I have no idea what may be happening, and I am at a loss as to how to accomplish what I want to do - copy a large number of messages from Sent-restored to Sent.
You're making the assumption that some folders are not corrupted, when the evidence suggests they are. Or, the msf index files are corrupted and need to be rebuilt (TB recreates them automatically on restart), which is usually the case when the Subject and message body is mismatched. No matter which condition applies, or what is the underlying cause, you are better off rebuilding folders and not copying mail from suspect folders back into new, uncorrupted folders. Since the account is POP, all messages are on the local computer anyway, so it doesn't really matter if they aren't all in one Sent folder.
If I shutdown TB, then move the Sent.msf file elsewhere (to archive it), and then restart TB, will this cause the Sent..msf file to be re-created? Is this what I have to do?
I was an e-mail administrator before I retired, and once I had to help a user who had a 2+Gb mbox file due to a mail loop. He wanted me to extract a few mail messages into a new mbox file. It took me a few hours using grep, head. and tail.
If you delete the msf file, while TB is closed, it will be automatically rebuilt when TB restarts.
I Exited TB, then renamed the Sent.msf file, then restarted TB Daily. TB said that it rebuilt the index, and the Sent folder looks OK. But the sent.msf file now has zero length. When will TB re-write the Sent.msf file? Thanks.
I just checked, and Sent.msf has been rewritten to disk. So I copied one mail message from Sent-restored to Sent, and it copied fine. I copied 10 more messages from Sent-restored to Sent, and the first copied correctly and the other nine had the text of the first mail in my Sent folder. I checked the messages in the Sent-restored folder before I copied them, and all were intact.
Did this happen with the original Sent folder or the new one created after the Sent mbox was deleted? You have to delete the mbox and msf file, then send a message to create a new Sent mbox and msf files.
I CANNOT delete the Sent file, as it contains mail that I have sent. The "problem" with this file is that messages I had sent between 03/13/2020 and 08/12/2021 are missing (and I have no idea how that happened). Also, the Sent file that I have restored is current up to the time of that backup. It does not contain all of the messages I have sent since that backup copy.
If TB has recreated the Sent.msf file without complaint, does this imply that the Sent file is a proper mbox file and has no problems/corruption? If I take your suggestion, then what do I do? Take the new, empty, Sent folder and copy all of the messages from the Sent-restored file? What about all of the messages that I have sent since then? Or do I
1) Exit TB, 2) move the current Sent and Sent.msf files as backup, 3) move the Sent-restored and Sent-restored.msf files into new Sent and Sent.msf files, 4) Start TB, 5) Copy newly-sent messages from the saved Sent folder into the Sent folder.
Exactly what file(s) do you think are corrupted? There are only four files involved here:
Sent Sent.msf Sent-restored Sent-restored.msf
When I look at these two folders in TB, I do not see any problems.
No, rebuilding the msf file doesn't mean the mbox file is repaired.
Copy the messages from the corrupted Sent folder to another folder, then delete the Sent mbox and msf file. That way, you aren't losing any messages already in Sent, but are replacing it with a new, uncorrupted Sent folder that stores sent messages going forward. Don't bother copying from Sent-restored to the new Sent, in case that introduces another source of possibly corrupted mail.
I want ONE file that has my sent e-mail messages. With what you suggest, I will have
1) a new Sent folder that will contain all messages I send from today onward
2) a "Sent1" folder that contains all of the messages I have sent previously (except for those from 2020 and 2001 that are missing
3) a Sent-restored file that contains the missing sent e-mails (and many more).
If I want to search for mails that I have sent, then I will have to search through three folders instead of one. And one folder will have more than just the missing messages. I assume that if I then take the Sent-restored folder and copy the missing messages elsewhere, I will have the same problem with the "elsewhere" folder in which I place the messages.
If I copy one message from Sent-restored to Sent, and there is no corruption, why would there be corruption if I try to copy more than one message?
Is there anything special with the Sent folder other than the fact that it is not in the Local Folders sub-directory (it is in the pop.att.yahoo.com sub-directory)? I just checked, and missing mails I have copied from two other restored Local Folders mail folders were copied/moved intact.
Maybe I will have to resort to using grep -n, head, and tail to copy the missing mails into my existing Sent mbox file.
I realize you want a single Sent folder, but since one or more of your existing folders might be corrupted, I can only suggest a way to separate the possibly corrupted from the new and not corrupted folders. Beyond that, I have no further advice to offer, as you seem intent on ignoring anything I provide.
As I asked above - Which of the four folders in question do you think is/are corrupted?
Here is what I did : 1) Exit TB daily 2) Copy messages at the end of Sent to Sent-newmsgs (via grep -n and tail +n) 2) Rename Sent and Sent.msf 3) Copy sent-restorted to Sent
Copy sent-restored.msf to Sent.msf
4) Restart TB daily 5) Copy one message from Sent-newmsgs to Sent ==> OK 6) Copy 8 messages from Sent-newmsgs to Sent ==> BAD
Delete the 8 bad copied messages. Are they bad because sent-msf is corrupt?
7) Exit TB
cat Sent-newmsgs >> Sent erase Sent.msf
8) Restart TB daily 9) Delete the one now-duplicate message in Sent
Now I have one intact Sent folder. I have no idea if any of the mbox files I am using is corrupted; the indices and mail messages that TB displays seem to be OK.
There are currently issues with daily builds that need fixes, and are not likely helped by temporary measures. I suggest you visit https://thunderbird.topicbox.com/groups/daily
I did see one item that might be related to my problem.
I think that what I should do is to install the latest non-daily and test it so see if it has the problem. I no longer need daily because the fix for a problem I reported has already made it into the base product from daily. Do I just
1) Exit TB Daily 2) Download and install the latest TB 3) Start TB and respond to the prompts to use the proper profile.
Is that all I need to do? Thanks.