So I upgraded my VMware virtual machine from Windows 2003 R2 to Windows 2008. The process went almost smoothly, but I had to switch the network card type from VMXNet 3 to E1000 to get network connection working. Alright, I can deal with that – who needs 10Gb network connections anyway? That’s sarcasm, actually. 😛
After this I discovered that all hell broke loose: I could not browse the server by its name (while browsing by its IP address worked fine and Remote Desktop Service worked, too), SQL database was inaccessible and Event Viewer‘s Security log was full of NULL SID login failure events with ID 4625, Status 0xc000006d, Sub Status 0x8009030e:
An account failed to log on. Subject: Security ID: NULL SID Account Name: - Account Domain: - Logon ID: 0x0 Logon Type: 3 Account For Which Logon Failed: Security ID: NULL SID Account Name: Account Domain: Failure Information: Failure Reason: An Error occurred during Logon. Status: 0xc000006d Sub Status: 0x8009030e Process Information: Caller Process ID: 0x0 Caller Process Name: -
The friendly view of the Details tab added some more data to the failure reasons:
SubStatus0x8009030e LogonType 3 LogonProcessName Kerberos AuthenticationPackageName Kerberos
“OK, let’s google ‘Kerberos error 0xc000006d’ then,” I thought. Boy, did this turn up a bunch of useless solutions such as removing the server from the domain and adding it back (with and without deleting the computer account), verifying time services and DNS settings, and up to formatting and reinstalling everything. Of course, I tried the first one several times. Most of the questions had no working solutions other than complete reinstallation – not good enough.
So I applied all updates available in Windows Update, disabled Windows Firewall, reset computer account with and without command prompt, removed all unneeded services, even broke and repaired one of the domain controllers in the process – and the errors kept on coming.
The problem was clearly related to Kerberos, but I had no more clues until I compared the output of the klist tgt command between the problematic server and a working one. Yup, the Session Key had KeyType starting with RC4 on the bad server and AES-256 on the working one. So I googled some more and found out that you can set Kerberos encryption types via Group Policy. But, hey, Windows 2008 has no such setting in its gpedit.msc. Damn, it’s available since Windows Server 2008 R2 and Windows 7!
Okay, my domain controllers were both 2008 R2, so I ran Group Policy Management Editor there, opened Default Domain Controller Policy, expanded Computer Configuration, Policies, Windows Settings, Security Settings, Security Options, and double-clicked Network security: Configure encryption types allowed for Kerberos.
I turned on the Define these policy settings option and ticked all checkboxes.
Then I did the same with Default Domain Policy (that actually applies to workstations and servers that are not domain controllers), ran gpupdate /force on domain controllers and the problematic server and voila! browsing worked like a charm. 😀
A quick look at the klist tgt output revealed that the session key type was indeed AES-256 now.
So you need to update Group Policies to get rid of the Kerberos error 0xc000006d that has Sub Status 0x8009030e: apply the solution above for both Default Domain Policy and Default Domain Controller Policy. If your domain controllers are not Windows 2008 R2 or newer, you can use RSAT (just google it) to edit the Kerberos encryption types setting described above.
A few days later I wanted to see if the setting could be limited to just “AES256” and “Future encryption types”… and failed miserably again. If you have Windows Server 2003 devices, they will soon be inaccessible. You cannot browse their shares and printers, or run gpupdate /force or klist (available in Windows Server 2003 Resource Kit only) without receiving Kerberos ticket errors 0x8009030e on these old servers until you enable all checkboxes in Group Policy again and then log on locally (not over RDP!) to the Server 2003 device.