Daily IT Matters, this is the place where I post my daily findings on IT.

Thursday, February 14, 2013

RDP Authentication issues Windows 2008 R2

Today we had a nasty encounter with sudden access denied on RDP connection to our Terminal Server Farms.

The day started that users could not connect to our Navision environment, this environment constists of an Terminal Server farm with an SQL 2008 Cluster. After analyzing first symptoms we saw that the SQL Cluster had failed over but that the quorum disk was stil on the Node that had "sudden" issues.

Thus we gracefully brought the faulty node down through Cluster Manager and started it up again.
In the meantime we were notified that there were still issues, then we saw profile redirection errors caused by a faulty location and permissions of the user profile location. We left the profiles at default location and focussed on the permissions we fixed them, and users started to report that they could log in again.

But now problems started to appear out of now where users could not connect to our other Terminal Servers farms for other application, we where now more then 3 hours after the initial problems appeared.
While trying to find several causes for these issues we focused on the first symptoms that were reported, we should not have done that but that is with hindsight.

We focussed on Kerberos issues because we have a mixed 2003 and 2008 forest and 50 domains, we sometimes run into the infamous KDC ticket being to large. We now set it to 48000 as per microsoft recommendation for windows 2012 with its base http encoding that it no longer can be set at its largest dword value it can take.
Read this for Kerberos changes in 2012 it will save you in large environments.

But we came to the conclusion it was not the rootcause, simultaniously we started to suspect group policy changes but checking the change date for all applied policies ruled that out rather quickly.

After pinpointing the problems to a single datacenter we realized what just had happened, it was the BlueCoat that was causing the problems, the day before it was put back in service after former issues.

Bypassing the Bluecoat solved the issues.....

pwhmmmmwwwwweeh.. what a day in the office

No comments: