Tame Your Exchange 2013 Logzilla

First Things First

The term Logzilla, as used in this post, has nothing to do with the syslog analysis solution named LogZilla.

Big things are sometimes called Whateverzilla. Oversized Exchange 2013 Managed Availability logs beg for the name. Let me know if it is upsetting to anyone.

Now that IP has been taken care of, skip to The Solution section below if you know all about Logzilla and just want to tame it, or read on for some background information.

Short Primer

Managed Availability is a great feature of Exchange 2013 in large deployments where automation is important. However it can be a hindrance for the small business with limited resources and little experience.

In a nutshell, Managed Availability constantly monitors every aspect of an Exchange 2013 environment, and where appropriate, it automatically takes proactive action to ensure that service levels aren’t affected. Fix first, analyse later. When we consider that Microsoft has hundreds of thousands of servers in Office 365 servicing millions of users, you can immediately see the benefits.

Also, Microsoft’s stance is that it shortens service call resolution time if the events that caused an outage are caught as soon as they happen. We no longer have to turn on logging, reproduce the issue, then turn logging off, because Managed Availability logs will have captured the original event already. It makes perfect sense in the case of the odd issue that’s difficult to reproduce.

To meet these goals, Managed Availability continuously gathers system performance data. As we can imagine, detailed logs need quite some space to be stored. This is why Microsoft recommends “at least” (a.k.a. minimum) 30Gb free space for the disk on which Exchange is installed. As we know from past experience, when Microsoft says “minimum”, practically it means that the software will install, however it will be unable to do anything useful.

The Issue

Exchange can be installed in a variety of storage configurations. Traditionally, it is not uncommon to see a 50-60Gb C: drive, where along the OS, there is Exchange also. If logging needs to be turned on, then traditionally the log location could be easily redirected to another drive with sufficient free space so that C: doesn’t fill up.

With Exchange 2013 you will quickly find that C: will fill up steadily and you can do nothing about it. At least not easily. And you haven't even turned on any logging!

The cause is Managed Availability: logs just keep (re)growing like cancer. By default. Unstoppable (well, MA can be disabled but we don't want to).

At this point you have probably crawled the Internet for solutions. You’ve probably come across articles which tell you to reconfigure the log location in Perfmon or in the Registry. You even tried it, only to find that the logs are back on C: in a few days. In your desperation you want to disable Managed Availability altogether, but that's not a good thing either.

The Future

I have recently attended the O365 Summit in Sydney where I had the chance to ask, straight from the source, how to relocate Logzilla.

The bad:

• There is no supported way to move Managed Availability logs.
• There are no plans to enable administrators to configure log paths in any future Exchange 2013 patch.

The good:

• The Exchange team has heard the community and is considering granting administrators the option to move the logs. However it will only be available in Exchange v.Next.

The Solution

If you are installing a new Exchange 2013 server then install it on a drive other than C:. A 100Gb drive should be sufficient, then you shouldn’t have to worry about moving logs, queues etc – just leave them in their default location. Optionally you may still want to move or logrotate IIS logs as by default they are stored on C:, regardless where Exchange is installed.

However if you already have an existing deployment which struggles with free space, and you cannot move your e-mail to a new Exchange server, then here is what you can do to address the immediate issue:

1. Download Sysinternals’ Junction tool from here.
2. Identify the folders which hold the largest logs, and those which grow the fastest on your C: drive. You can use a tool like WinDirStat.
3. Add a new volume with sufficient space to the Exchange server (100Gb will do)
4. Create a folder structure where you will store the logs. I prefer something that resembles the original structure.
5. Restart the server in Safe Mode. This is needed because some services run in the System context (PID 4) and will not let you implement the changes for the files will be locked.
6. Delete the original log folder, for instance “ETLTraces“ in “C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Diagnostics\”. If you want to preserve the log files then move them first to the new location.
7. Create a junction point in the original location, pointing it to the new target. For example:

JUNCTION.EXE “C:\Program Files\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Diagnostics\ETLTraces” “L:\Logs\Microsoft\Exchange Server\V15\Bin\Search\Ceres\Diagnostics\ETLTraces”

NOTE: Just to make it very clear, to successfully replace the folder with a junction, the original folder must be deleted first, and the new target folder must already exist.

8. Restart the server.
9. Optional but recommended: migrate to a server with a properly designed storage as soon as practical and decommission the trouble server.

WARNING:

1. While I found this solution to work and I could identify no ill effects, it is NOT supported by Microsoft. Implement it at your own risk and test it in a lab first. I accept no responsibility for any damage, downtime or loss of any kind it may cause.
2. Read point 1 again.

There you have it. Hope it saves you some headache.