Report Email Traffic By The Hour

It’s a well known fact that reporting is the sexiest topic in IT. To that end, I thought I’d post a quick one liner about email flow reporting in your organisation. This came about following a request from one of my favourite customers, who needed a way to report on how much email was being sent and received out of hours.

Get-MailTrafficReport -StartDate 01/14/2018 -EndDate 01/22/2018 -AggregateBy Hour -EventType GoodMail | select Date,Direction,MessageCount | Export-csv C:\users\emily\Desktop\mailflowreport.csv

This PS command is run in Exchange Online Powershell and will result in a CSV which shows an hourly breakdown of email sent / received in a given time period. It’s possible to add specific times to the dates (eg “01/14/2018 05:00”). I used the -EventType GoodMail variable to only report on Accepted mail in this example. You can also filter on -Direction (Inbound or Outbound). Below is a snapshot of the results:

Date Event Type Direction Action Message Count
------ ---- ---------- --------- ------ -------------
 15/01/2018 14:00:00 GoodMail Inbound 430
 15/01/2018 15:00:00 GoodMail Inbound 230
 15/01/2018 16:00:00 GoodMail Inbound 187
 15/01/2018 18:00:00 GoodMail Inbound 57
 15/01/2018 18:00:00 GoodMail Outbound 124
 15/01/2018 19:00:00 GoodMail Inbound 34
 15/01/2018 19:00:00 GoodMail Outbound 87

The TechNet article on the Get-MailTrafficReport cmdlet is here

This is a very versatile reporting function which can yield interesting data. This data can then be fed into PowerBI or a.n.other reporting tool to add some visual showmanship to the results!

Exchange 2016 on Server 2016 – A reboot from a previous installation is pending

Recently I was attempting to install Exchange 2016 on Server 2016. On attempting to run the setup.exe /preparealldomains /iacceptexchangeserverlicenseterms command, I was receiving a failure when checking prerequisites which stated that:

PS E:\> .\Setup.EXE /preparealldomains /iacceptexchangeserverlicenseterms

Performing Microsoft Exchange Server Prerequisite Check

Prerequisite Analysis FAILED

A reboot from a previous installation is pending. Please restart the system and then rerun Setup.
For more information, visit: http://technet.microsoft.com/library(EXCHG.150)/ms.exch.setupreadiness.RebootPending.asp

I had rebooted the server a few times and ensured that no restarts were pending.

In versions of Server prior to Server 2016, I would be looking for the UpdateExeVolatile registry key and the PendingFileRenameOperations registry key under HKEY Local Machine. However these didn’t appear to be in their normal place. Eventually I did a search of the Registry and discovered that PendingFileRenameOperations has moved to:

HKLM\System\ControlSet001\Control\Session Manager\PendingFileRenameOperations

The previous location of this key was:

HKLM\System\CurrentControlSet\Control\Session Manager\PendingFileRenameOperations

Removing the entries in PendingFileRenameOperations resolved the problem in this case.

 

Public Folder Migration Fail #2

Another day, another Public Folder migration failure. This time, on testing your Public Folder migration to Office 365, they appear to be unavailable and are not visible in the Outlook client.

I always follow the wonderful guide provided by Microsoft on how to migrate your Public Folders from Exchange > Office 365 (I’m not being sarcastic, it is actually a good guide) available here: https://technet.microsoft.com/en-GB/library/dn874017(v=exchg.150).aspx

The last two times I have run through this process, I have attempted to test the PF Migration on a single user prior to going live for all users. Microsoft suggest the following command for doing this:

Set-Mailbox -Identity <Test User> -DefaultPublicFolderMailbox <Public Folder Mailbox Identity>

However since the Exchange 2016 wave of Office 365 has gone live, this command no longer appears to have the desired effect. What seems to happen is that because the -IsExcludedFromServingHierarchy parameter is set to $true, the command does not fully enable the Public Folders for that user.

In both situations, I have taken the plunge and enabled Office 365 Public Folders for all users by running:

Get-Mailbox -PublicFolder | Set-Mailbox -PublicFolder -IsExcludedFromServingHierarchy $false

The end result (after a little patience) is that Public Folders become available for all users. I’m not sure if this is a general bug or a result of the Exchange 2016 backend of Office 365, but I’d be interested to hear your experiences!

 

Public Folder Migration Fail

The above title isn’t a surprise for anybody working in IT, but unusually for Public Folders, this one has a fairly simple fix!

The situation is thus; when attempting to complete a Public Folder migration to Office 365, you come across the following error:

Before finalizing the migration, it is necessary to lock down public folders on the legacy Exchange server (downtime required). Make sure public folder access is locked on the legacy Exchange server and then try to complete the batch again.

Public Folder migration error

The problem with this error is that you have already locked down Public Folders on the legacy Exchange Server by running:

Set-OrganizationConfig -PublicFoldersLockedForMigration:$true

So what’s an admin to do when they’ve already run the command they are being told needs to be run?! Some googling may lead you to the idea of rebooting the server, or restarting the Information Store. Both of these will work, but a much simpler solution is simply to dismount the Public Folder database/s, and then mount them. The PFs are already locked so are unavailable to the users so there is no negative impact of doing this.

TL;DR – turn it off, and turn it on again.

 

Script to change Exchange internal URLs

All Exchange consultants will have been through this situation at least once now; a customer is using a split namespace in their Exchange environment with a .local name internally, and due to the new requirements for purchasing SAN certificates, they can no longer purchase certificates with a .local name on them. One way of remedying this is to change all the Exchange internal URLs to use the public name, and add in an internal DNS zone and record to point the public name at the Exchange environment.

Changing the URLs for multiple virtual directories and servers can be a pain. There are many scripts like this out there on the internet, but I frankenstein’ed this one to fit the needs I had. I wanted it to prompt for the server name, public FQDN and autodiscover FQDN and then change the directories on that particular server to reflect the names I had entered. I also love simplicity, so I wanted the most simple script possible. If you wanted to change this to do the external URL’s, then just do a find and replace internal with external! This script will work on Exchange 2010 and 2013.

Also, if you are using a wildcard certificate, be sure to run the below command to force the name match, otherwise you may get certificate errors on your clients!

Set-OutlookProvider -identity EXPR -certprincipalname msstd:*.domain.com

Here is the script I use:

#get variables
write-host "Set Exchange 2010 Internal URLS" –Foregroundcolor Yellow
$urlpath = Read-Host "Type CAS Array FQDN starting with https://"
$autodpath = Read-Hosts "Type Autodiscover FQDN starting with https://"
$CASserver = Read-Host "Type internal server FQDN"
#change urls for all internal directories
Set-AutodiscoverVirtualDirectory -Identity "$CASserver\Autodiscover (default web site)" –internalurl “$autodpath/autodiscover/autodiscover.xml”
Set-ClientAccessServer –Identity "$CASserver" –AutodiscoverServiceInternalUri “$autodpath/autodiscover/autodiscover.xml”
Set-webservicesvirtualdirectory –Identity "$CASserver\EWS (default web site)" –internalurl “$urlpath/ews/exchange.asmx”
Set-oabvirtualdirectory –Identity "$CASserver\OAB (default web site)" –internalurl “$urlpath/oab”
Set-owavirtualdirectory –Identity "$CASserver\OWA (default web site)" –internalurl “$urlpath/owa”
Set-ecpvirtualdirectory –Identity "$CASserver\ECP (default web site)" –internalurl “$urlpath/ecp”
Set-ActiveSyncVirtualDirectory -Identity "$CASserver\Microsoft-Server-ActiveSync (default web site)" -InternalUrl "$urlpath/Microsoft-Server-ActiveSync"
#get commands to to doublecheck the config
get-AutodiscoverVirtualDirectory -Identity "$CASserver\Autodiscover (default web site)" | ft identity,internalurl
get-ClientAccessServer –Identity "$CASserver" | ft identity,AutodiscoverServiceInternalUri
get-webservicesvirtualdirectory "$CASserver\EWS (default web site)" | ft identity,internalurl
get-oabvirtualdirectory "$CASserver\OAB (default web site)" | ft identity,internalurl
get-owavirtualdirectory "$CASserver\OWA (default web site)" | ft identity,internalurl
get-ecpvirtualdirectory "$CASserver\ECP (default web site)" | ft identity,internalurl
get-ActiveSyncVirtualDirectory "$CASserver\Microsoft-Server-ActiveSync (default web site)" | ft identity,internalurl

Hybrid Configuration Wizard and Multiple Domains – Get-FederationInformation cmdlet had thrown an exception

When running the Exchange Hybrid Wizard for multiple domains, you may find it fails and shows you the error below:

Execution of the Get-FederationInformation cmdlet had thrown an exception. This may indicate invalid parameters in your Hybrid Configuration settings.

Federation information could not be received from the external organization.

In addition to this, if you check the Update-HybridConfiguration log file in the Exchange Logging directory, you will find that the failure occurs just after the command Get-FederationInformation is run on one of your domains.

The first test you can run is to login to Microsoft Online PowerShell and try running:

Get-FederationInformation -DomainName domain.com

If this comes back with an error, then you likely have an issue with Autodiscover. It may be that autodiscover is not configured for all your domains, which is quite a common occurrence. There are 2 traditional ways to get around this:

  1. Configure multiple SRV DNS records to point Autodiscover at your primary Autodiscover service
  2. Add all your autodiscover domains to your SAN certificate and configure A records to point autodiscover to the public facing IP of your Exchange server/s

As of Exchange 2010 SP3 UR6 and Exchange 2012 SP1 however, there is a much cleaner way of doing this.

  1. Make sure Autodiscover is configured and working on your Primary SMTP domain (use https://testconnectivity.microsoft.com/ to verify functionality)
  2. Run the HCW (Hybrid Configuration Wizard) for just your Primary domain. This should complete without issues
  3. Go into the Exchange Management Shell on your On Premise Exchange environment and run:

Set-HybridConfiguration -Domains domainb.com,domainc.com,domain.com,autod:domaina.com

Where domaina.com is your Primary SMTP domain. This sets your autodiscover domain for all domains to domaina.com.

  1. Re-run the HCW. You should now see all domains populated and the HCW should complete successfully.

Exchange Server 2016 Announced at Ignite!

Exchange Server 2016

Great news for all you Exchange consultants and architects out there. A whole new version of Exchange is on it’s way for us to learn about and deploy!

The Ignite 2015 conference played host this year to the announcement of Exchange 2016 as the on premise successor to Exchange 2013. This new version will be released in the latter half of 2015 and has a few notable changes / features added:

  • The Client Access role has been removed completely. In Exchange 2013 the Client Access role was simply an intelligent proxy service and had no real involvement in traffic other than proxying it to the correct location. Now clients will connect to client access services (running within the Mailbox role) and those requests are routed to the Mailbox server holding the active database for that mailbox.
  •  Search has received various improvements, primarily to performance, which was a bugbear in Exchange 2013 due to the slowness of search in Online mode and OWA in particular.
  • Document Collaboration is Microsofts headline feature. This will leverage Office Web Apps and SharePoint servers, and potentially Microsoft Online Services to allow for document versioning and collaboration within the attachment mechanism in Outlook/OWA. In essence, a user can attach an item to an email and not have the issue of dealing with manually merging multiple versions of the same document. This versioning will be handled automatically.
  • Outlook Connectivity will now be handled by MAPI/HTTP by default. This connectivity protocol reduces bandwidth and latency requirements and provides a more stable Outlook experience.
  • Coexistence with Exchange 2013 is going to be a hoot. When deploying Exchange 2016 in your environment, you will not need to move the namespace in order to migrate mailboxes as there is backwards compatibility with the Exchange 2013 namespace model built into Exchange 2016. This means that it will be simple to introduce Exchange 2016 into your environment and start using it in anger. You will still need to migrate this namespace eventually but when you do this is up to you!
  • Hybrid functionality will, of course, be improved, allowing for customers to more easily decide which parts of the component stack they want to remain On Premise and/or move to the Cloud.

Armed with the information we have so far been furnished with, I think that we can all look forward to a more simplified and powerful Exchange On Premise experience. Microsoft must have learned a lot from running the largest Exchange Organisation in the world in Office 365, and I hope that all the On Premise environments around the world will benefit from their lessons learnt.

The operation couldn’t be performed because object ‘EXCHANGE\First Storage Group\Mailbox Store (EXCHANGE)’ couldn’t be found on ‘DC.domain.local’.

A most inventive and amusing title to this post, if I do say so myself!

After installing Exchange 2010 into a legacy Exchange 2003 environment, you may be faced with the following error when viewing the properties of an Exchange 2003 mailbox in the Exchange Management Console. In addition to this, if you attempt to migrate a mailbox to Exchange 2010 from 2003, you may see this error:

Mailbox database “EXCHANGE\First Storage Group\Mailbox Store (EXCHANGE)” doesn’t exist.

This is a permissions issue, and the fix is relatively simple:

1. Log into your Exchange 2003 server and open the Exchange System Manager (affectionately known as ESM).

2. Go to the properties of the Mailbox Store mentioned in the error message:

Mailbox Store Properties

3. Go to the Security Tab of the Mailbox Store and select the Advanced option. Tick the box to ‘Allow inheritable permissions’ , and Apply your changes.

Mailbox Store Properties

4. If this doesn’t fix your problem, or if the ‘Allow inheritable permissions’ box is already ticked, then do the same thing (Advanced settings under the Security tab, make sure the ‘Allow inheritable permissions’ box is ticked) but to do this, go into the Properties of the server itself, not the Mailbox Store.

5. If even this doesn’t work, then you should manually add in the Exchange 2010 server into the permissions group for the Exchange 2003 server and give it Full Control.

 

Hope this helps!

Office 365 Hybrid Mailbox Move stuck in ‘Removing’ state

This is an issue I’ve come across more than once now. An attempted mailbox move from Exchange 2010/2013 to Office 365 has failed and you want to remove the migration batch and try again. You try to remove the batch, but it just gets stuck in the ‘Removing’ state for an extended period of time. We need to give this request the finger and start from scratch, but how?

First things first, lets check the status of the move using Powershell, as Powershell will never lie! Login to Exchange Online Powershell, and run:

get-migrationbatch -identity <nameofbatch> | fl

If the status does read as ‘Removing’ and it’s been a long time since you started the removal, then you likely have a corrupted batch. Let’s forcefully remove it. To remove the batch, run:

Remove-migrationbatch -identity <nameofbatch> -force

If you now run the get-migrationbatch command above, you should get an error which states that the batch does not exist. Good news! We now just need to clear out the migration user requests which will still be lingering. To see which user requests exist, run:

Get-MigrationUser

If the only users in here are the users which were associated with your migration batch, then you can run:

Get-MigrationUser | Remove-MigrationUser -Force

to remove all of the migration user requests. However if there are other user requests in here which you do not want to remove, then remove the users individually by running:

Remove-MigrationUser <Identity> -Force

Now if you run the Get-MigrationUser command, you should see that the users who were in the corrupted batch are no longer listed. You can start a new batch once you’ve resolved whatever issue caused the mailbox move to fail and all should be tickety-boo 🙂

In our case we were running mailbox export commands at the same time as mailbox migrations, and we had some timeout issues with the Mailbox Replication Service. The error we received in the migration report was “Relinquishing job because of large delays due to unfavorable server health or budget limitations”. Simple fix, just remove the migration batch once the exports were complete, and start again. What we didn’t bank on would be that the migration batch would become corrupted. To resolve this, we allowed our mailbox exports to complete, and then restarted the Microsoft Exchange Replication Service. We then cleared the corrupted batch using the commands shown above, and started in again. It completed successfully this time.

Working with date specific PST exports using PowerShell

With all the various email archiving tools in place across the world, invariably in the world of Exchange consulting we get involved in lots of mass exports/ingestions of data to and from various services. One task which is performed often is exporting mail from Exchange mailboxes from a specific date range.

In order to do this, you first need to have the required permissions to actually export data from Exchange 2010/2013. This is not part of your permission set as a member of the Organisation Management role group (which some admins assume is an account with god level rights). So to begin with, we will run some commands to create a new custom role group, and then add ourselves into said role group. If you try and run the export commands and receive the following error, then you need to follow the below process to setup a new role group.

The term ‘New-MailboxExportRequest’ is not recognized as the name of a cmdlet, function, script file, or operable program.

Open up your Exchange Management Shell (as Administrator of course!), and run the following commands:

New-RoleGroup "Mailbox Import-Export Management" -Roles "Mailbox Import Export"
Add-RoleGroupMember "Mailbox Import-Export Management" -Member DavidD

You will now have the required permissions to allow you to run the New-MailboxExportRequest commands. By the way, this powershell command only became available as of Exchange 2010 SP1 so if you are mad enough to be running Exchange 2010 RTM, this command will not be available.

In order to have access to your lovely new cmdlets, you will need to close and reopen the Exchange Management Shell (as Administrator!). Now you can run the command as shown below, just tweak the settings marked in bold to get your desired effect 🙂 As a side note, the -lt stands for less than, and the -gt is greater than. You can also use -le, which is less or equal to, or -ge, which is greater or equal to.

New-MailboxExportRequest -ContentFilter {(Received -lt '01/04/2014') -and (Received -gt '12/02/2012')} -Mailbox "DavidD" -Name DavidDExport -FilePath \\myserver\pst\DavidDExport.pst

At this stage, I’d like to point out a little gotcha to do with this command. As I am in the UK, the servers I work on are configured with UK regional settings, including date and time. This means that dates are displayed in a DD/MM/Year format rather than the American MM/DD/Year format. If your regional settings reflect the UK configuration, then the trick is to use UK date format but never use a number above 12. So if you were to use 15/03/2015 (15th March 2015) this would queue the request but would fail after a minute or two with the error:

“The value “15/03/2015 00:00:00 AM” could not be converted to type System.DateTime.”

However if you use 12/03/2015 (12th March 2015) this would work and would export the correct date ranges. If you used 03/12/2015 in the UK, Exchange would think you meant the 3rd December 2015. Obviously if you are in the US this is not a problem, but I struggled with this in the UK. If anybody has seem differently or knows a way around this, please comment and let me know! My advice at the moment though is to use UK date formats, but never use a number above 12 for the day.

Once your request has started, you can run the below command to see the status of your request.

Get-MailboxExportRequest

If your request shows a status of failed, use the below command to retrieve some useful information about the failure.

Get-MailboxExportRequestStatistics -Identity DavidD\DavidDExport

Hopefully this can get the ball rolling for you when attempting to export mail out of Exchange 2010/2013.

Thanks for reading!