Saturday 5 November 2016

StalledDueToTarget_MdbCapacityExceeded

Intro

The Mailbox Migration Performance Analysis blog posted by the Exchange team back in 2014 provides a great deal of details about monitoring and measuring migration performance using the various states and the time a job spent in each state.

What it doesn't touch on, however, is what appears to be a rare condition: sometimes the Exchange Online mailbox database designated to store a particular on-boarded mailbox apparently runs out of space. At least that's what an educated guess made me to conclude.

Some Context

In one of my recent Office 365 migration projects one particular mailbox was stuck for hours in the StalledDueToTarget_MdbCapacityExceeded state.

Get-MoveRequest <Identity> showed that the request has been queued. Nice detail but in this case it wasn't very informative.

Displaying some more details, including the status report, returned no useful information either. In fact the report may be a bit misleading in that it may suggest that the mailbox is too big and it doesn't fit within the quota. Well, the figures indicated that both the mailbox and the archive were well below the 50Gb mailbox and unlimited archive quota.

Get-MoveRequest <Identity> | Get-MoveRequestStatistics -IncludeReport | fl DisplayName, StatusDetail, TotalMailboxSize, TotalArchiveSize,Report


I have never come across this particular condition and I turned to my omniscient technical advisor, GG (Google the Great). No luck there either. Not one hit. Am I the only one in the universe who's seen this so far? I started to speculate.

The error suggests that the mailbox in question has been designated to be stored on a specific mailbox database, however the database had no capacity to accommodate the mailbox.

Assuming that I am correct, I have no way of knowing how this happened. I can think of two possible scenarios:

1. I can imagine that under some rare conditions the database selection algorithm may over-provision mailbox databases. The system detects it down the line, but it fails to re-allocate the mailbox to another eligible database. and the mailbox move gets stuck and never completes. I suspect this is the most likely cause.

2. The maximum size of the target mailbox database has been administratively reduced *after* the move request has been queued. Unlikely but not impossible.

I may never find out, but it is irrelevant anyway.

After the troubled mailbox has been sitting in this state for over 5 hours, I said to myself that I have to do something about it. Maybe removing the mailbox from the batch and re-migrating it separately will put the mailbox into a roomier database. And it did!

The Fix

This worked for me:

1. Stop the migration batch. Mailboxes cannot be removed from a batch that is still running.

Stop-MigrationBatch "Batch Name" -Confirm:$false


2. Give it a couple of minutes and confirm that the batch has stopped. If it is still "stopping" then give it some more time.

Get-MigrationBatch "Batch-Name"


3. Find the user's primary SMTP address by running one of the following commands on the on-premise system:

((Get-Mailbox -Identity UserName).PrimarySmtpAddress).ToString()

or

Get-Mailbox -Identity UserName | fl PrimarySmtpAddress

4. Remove the mailbox from the batch. Substitute Primary_SMTP_Address with the one obtained above.

Remove-MigrationUser <Primary_SMTP_Address>


5. Restart the migration batch.

Start-MigrationBatch "Batch Name"


6. Now that the user has been removed from the migration batch, create a new batch and add the mailbox. This time Exchange Online will place the mailbox into a database with sufficient space and the mailbox will be moved successfully - or at least that's what happened in my case.

That's it. Enjoy your move!

9 comments:

  1. G'day Zoltan,

    I've just seen the same error in my migration of a mailbox that has been waiting on 95% for finalisation for a couple of days. The EXO DB probably had plenty of space originally for the 15GB mailbox but must have since reached quota so I can't finalise. Maybe it is moving the mailbox data to another DB automagically but I kind of doubt it, I'm thinking I'll have to follow your advice and kill the mailbox from the batch and kick it off again. Just letting you know that you are not alone, even if Google says you are!

    -- Hayden Greaves

    ReplyDelete
    Replies
    1. Hayden, glad to see that my experience helps others out there :-)

      The more such experiences, the more likely MS will fix it.

      Delete
  2. How about seeing this status in on-premises Exchange 2016?

    ReplyDelete
    Replies
    1. I've only seen this issue once, so I cannot comment. Haven't heard of on-prem occurrences.

      Delete
  3. I'm getting the same error with a 2010 to 2016 Migration. The amount of space this batch of mailboxes will take up on the destination database wont put it at 10% used.

    ReplyDelete
  4. facing the same issue in 2016 CU12. Steps did not fix it for me. Even tried restarting source and target server.

    ReplyDelete
    Replies
    1. Hi Avdhesh, I recommend you to log a ticket with Microsoft.

      Delete
    2. Avdhesh .. How did you solve it ??

      Delete
  5. I had the message (StalledDueToTarget_MdbCapacityExceeded) immediately after submitting the batch. Within 15 minutes, the status had changed (StalledDueToTarget_Processor). I didn't make any changes to the system or the batch. Patience was all I needed.

    ReplyDelete