Microsoft Exchange on-premise servers cannot deliver email starting on January 1st, 2022, due to a “Year 2022” bug in the FIP-FS anti-malware scanning engine.
Starting with Exchange Server 2013, Microsoft enabled the FIP-FS anti-spam and anti-malware scanning engine by default to protect users from malicious email.
Microsoft Exchange Y2K22 bug
According to numerous reports from Microsoft Exchange admins worldwide, a bug in the FIP-FS engine is blocking email delivery with on-premise servers starting at midnight on January 1st, 2022.
Security researcher and Exchange admin Joseph Roosen said that this is caused by Microsoft using a signed int32 variable to store the value of a date, which has a maximum value of 2,147,483,647.
However, dates in 2022 have a minimum value of 2,201,010,001, which is greater than the maximum value that can be stored in the signed int32 variable, causing the scanning engine to fail and not release mail for delivery.
"According to additional research on this issue, this is happening because
Microsoft is using a signed int32 for the date and the new date
value of 2,201,010,001 is over the max value of "long" int32
being 2,147,483,647. @MSFTExchange -
Not sure why it was structured this way??"
— Joseph Roosen (@JRoosen) January 1, 2022
Temporary fix
However, for on-premise Exchange Servers currently affected, admins have found that you can disable the FIP-FS scanning engine to allow email to start delivering again.
To disable the FIP-FS scanning engine, you can execute the following PowerShell commands on the Exchange Server:
Set-MalwareFilteringServer -Identity -BypassFiltering $true
Restart-Service MSExchangeTransportUnfortunately, with this unofficial fix, delivered mail will no longer be scanned by Microsoft’s scanning engine, leading to more malicious emails and spam getting through to users.
Remediation is now made available from Microsoft
Microsoft, over the weekend, rolled out a fix to address an issue that caused email messages to get stuck on its Exchange Server platforms due to what it blamed on a date validation error at around the turn of the year.
Microsoft said in a blogpost– “We have addressed the issue causing messages to be stuck in transport queues of on-premises Exchange Server 2016 and Exchange Server 2019. The problem relates to a date check failure with the change of the new year and it not a failure of the AV engine itself. This is not an issue with malware scanning or the malware engine, and it is not a security-related issue. The version checking performed against the signature file is causing the malware engine to crash, resulting in messages being stuck in transport queues.”
Automated Solution
- Download the script here: https://aka.ms/ResetScanEngineVersion
- Before running the script, change the execution policy for PowerShell scripts by running Set-ExecutionPolicy -ExecutionPolicy RemoteSigned.
- Run the script on each Exchange mailbox server that downloads antimalware updates in your organization (use elevated Exchange Management Shell).
Edge Transport servers are unaffected by this issue. You can run this script on multiple servers in parallel. After the script has completed, you will see the following output:
To mitigate the Y2K22 problem, Microsoft is recommending customers to download a PowerShell-based scan engine reset script called “Reset-ScanEngineVersion.ps1” that can then be executed on each Exchange mailbox server used for downloading antimalware updates. It’s worth noting that the update will also change the version of the engine to 2112330001.
“The newly updated scanning engine is fully supported by Microsoft,” the company outlined. “While we need to work on this sequence longer term, the scanning engine version was not rolled back, rather it was rolled forward into this new sequence. The scanning engine will continue to receive updates in this new sequence.”
