I use a randomly generated key for my wireless connection. I had a guest over that wanted to connect to my network. So I connected to my router and read off the key to him. As I was reviewing all my settings, I saw the SSL option was not selected. I decided it was best practice to enable it. I turned the setting on, reconnected with a https page, and my browser did not like the cert. So I set out to resolve the issue.

I knew the cert was not trusted, so I had to figure out a way to get it trusted. I found these steps over on StackOverFlow.com. They almost worked for me. It got me on the right track anyway.

In those steps, you have to set the site as a trusted site. Reconnect and ignore the warning. Then view the cert. This time it will give you an option to install it. After installing it, I was still getting an error message. But it was a little different this time. It was saying that the cert was issued for a different site. I don’t know if everyone has this issue, but I did change my ip range and I expected that to be the cause. Looking at the cert, it indicates that it was issued to Linksys. This gave me an idea.

I was using the IP address and certs are issued to server names. So I just needed a DNS entry that maps the IP address to the name that it expected. But I do not host dns on my network. I just use the DNS from my ISP. DNS is not the only way to set up those mappings. The host file also does that. Open up C:\Windows\System32\drivers\etc\hosts and add a line with my router IP address and the name linksys.

Now when I type in https://linksys I get my router with a trusted SSL connection that is valid.

This started out very intermittent.  The same user could go from one computer to another. It would work on some and not on others.  It would work for our IT group every time.  The first common connection I made was the use of offline files.  We don’t like offline files. We find it causes us nothing but problems.  I will save those for another time.  I did find that machines that had offline files off had this issue.  If offline files are turned on, the problem went away.

We recently started over with our group policies as we rolled out Server 2008 and Windows 7.  We used the new folder redirection feature to redirect documents to the home folder and set favorites to fallow the home folder.  By default this enabled offline files to sync those locations.  There was a registry setting that disabled this default behavior that we later found and set.  But this still left offline files running.  It may be on by default.  I don’t remember any more.

Because offline files was on for most people this problem was fairly rare.  We finally had enough problems with offline files that we decided to kill it for everyone.  The resulting problem was everyone lost favorites.   So I set out to find a solution.  The better I wrapped my head around this the stranger it felt.

The other important detail was that our network administrators did not have this problem.  So that led me to permissions.  Our users only have security to home folders, not the folder structure under them.  Before we would map the UNC path of the profile to the N: drive.  Then we would point everything to N: including documents.  The new way to do it in group policy wants to use the UNC path as the redirection point.  That is introducing issues with some apps.  IE favorites included.

The problem goes away if I give the user full access all the way from the root of the share to the home folder location.  This is a bit much but it does work.  With this information in mind, I set out to see the minimum permissions that I needed to grant the users for this to work.  If I get the user just read access to the folders it worked.  Here is the list of minimum permissions that those sub folders needed to have.

List folder / read data
Read attributes
Read permissions

That looked like a simple enough solution but something still didn’t feel right.  If that is what the issues was, then I should have been able to find lots of other people with this problem.  For as much as I searched for this and as little as I was able to find, I knew something else was strange.  After some more experiments, I narrowed this down to be a very rare and unusual bug.

I found that I only had to set those permissions on the folder under the home folder.  None of the other parent folders needed any permissions at all.

I also found that the depth of the home folder made a difference.  If I moved the users home folder up just one directory location (with out any special permissions) the problem went away.  I even created a new folder structure of the same depth on the same root share to test this out.  It had the same results.

I also found that not using DFS (distributed file system) would also make this problem go away.  Same folder depth but not located on a dfs share.

So if we left Offline Files on, or used a folder structure that was one folder shallower, or did not use DFS, or did not redirect favorites with the server 2008 redirection policy, or did not use IE, then we never would have had this problem. I can see why it was so hard for me to find information on this.  I hope that the next person dealing with this is able to find this post and something in here will give them a glue to solve whatever issue they have.  I expect I am missing some simple detail but I was able to get it working in the end.

If you wanted to know how deep our home folders are, they UNC path looks something like this:  \\domain.name\rootshare\home\depart\username

The thing that made this stranger was my access was correct if I connected from a share on another computer. This was only a problem if I was on the physical server. I also discovered that the domain administrator account did not have this issue. It looked like server 2008 was not resolving group membership on folders. Domain Admins had full control of the folder and I was a member of that group.

For a while I would just login as the domain admin. After a little hunting I found that UAC was causing the issues. I didn’t even consider User Account Control because of the way it mucked up the permissions. The solution is to disable UAC and reboot. Once I did that, the problem went away.

I feel strongly that UAC is bugged in the way it handles this situation. When UAC is off and you click on a folder, your group permissions are checked and you are granted access. The system does not automatically add your name to the folder you try to open. Your group membership is enough to grant you access. That is how it should be. Now in the situation where UAC is turned on, you are prompted to grant yourself permissions. I don’t mind that it has to prompt you, but the action it takes after the prompt is the problem. It should check the group membership and see that you have access then let you go on. But what it does instead is give you full permissions directly even when your group membership is correct.

Who would think that this is the desired operation? I would expect most people that discover that this happened, the first thing they do is remove themselves from the security permissions. The other problem that I ran into what waiting for it to propagate those permissions. This can take a very long time when its on the root share for all your users.

This functionality also acts differently from other situations where UAC is protecting the system. If I try to do operations on the program files folder on a Windows 7 computer, I am prompted to make my changes but the underlying security is never changed.

UAC is one of those features that I want to embrace but it keeps shooting its self in the foot.

The first one where the issue was reported would lock up not only IE but also other printing.  After the crash, the print queue would not act correctly.  Printing test pages would give me an error saying that the server was not responding and that I needed to reboot it.  Other people could still print and test prints originating from the print server would print.  I also could not add new printers while this was happening.  Relogging did nothing.  a full reboot would return things to normal until it locked up later.

After a few crashes, we connected it to IE8 printing.  With a little more testing I discovered it would print fine with protected mode off but failed with it on.  We are trying our hardest to run secure by default so we want this protection.  Too bad it protected us from printing.

I returned to my computer and mapped the printer to my computer.  I was easily able to reproduce the problem.  My print spooler recovered better and I could print other things, but IE crashed hard every time.  I did a little hunting on the web and had a hard time finding content on topic.  Here is what I did find.

When IE runs in protected mode, it runs with limited privileges.  In my case the print driver wanted to run an exe from a location that is restricted by those privileges.   With that in mind, here are your options when faced with this problem.

1) Turn protected mode off
2) Run IE as administrator
3) Reinstall with a basic driver
4) Install printer locally (skip print server)
3) Don’t print to that printer
4) Create rules to elevate the processes

Some people are able to install more basic drivers but that is hit or miss.  On my computer I was able to install it locally avoiding the print server and it worked fine.  That was an interesting detail.  Our print server is server 2003. 

I was able to resolve the issue by creating a rule to elevate the processes needed for printing.  The post that gave me this idea is here. I did exactly what was suggested and it fixed the problem.  There are registry keys for IE that tell it what to elevate under protected mode.  I created a rule for the same 3 files mentioned: HP1006MC.EXE, HP1006SM.exe, P1006SSL.exe.  After closing IE one more time, it picked up those rules and it worked fine.

I exported those keys to reg files for more testing and it worked on the original computer that the problem was reported on.  I merged those settings into our domain policy and I hope that’s the last I have to deal with that. 

I found those files in the C:\Windows\System32\spool\drivers\w32x86\3 folder.  If I have more problems like this, my plan is to create a rule for all the exe files in that folder.  I was able to do the exact same 3 that the other guy mentioned.  If the problem continued, I was prepared to make rules for all 10 files I found.

I went over to our test environment. It was up to current patch levels, but the data was a few months old. Almost all of my broken records in live were broken in test. That gave me hope that its an existing data issue and not something actively breaking records.

The nice thing about our test environment is that I am usually the only person using it. That makes it easy to toss the SQL Profiler on with out having to filter it to just my traffic. I started the profiler at the point just before clicking the button that generated the RTE and paused it just after the message came up. I closed the app and repeated this process a few times for both working and broken records.

Looking at the queries that each ran, the broken records all stopped at the same point after the same query was run. That tells me that the application is processing the queries as it runs them and the last one before it stopped is my suspect query. If both the broken and the working records had the same full list of queries I would have had to check them all. That would have indicated to me that it pre-loads most of the data before processing it. I was in luck.

The query was a large one but the only difference between all records was the record ID. So it is the same query running for both working and non-working records. Running the query for all of them gave me different sized record sets. Some had more a lot of records and some had very few. One thing did jump out at me as I looked at the raw data.

A few null values jumped out to me in a column that looked like it should have a value. With my experience with the data I knew it should have had a value. I saw one column that had data in it directly related to reference I expected instead of that null value. Because this looked like a complicated query, my first guess was a bad join.

That column was not the only one with nulls in it but every time it had a null the 3 columns to the right also had nulls. That’s what I expect a bad join to look like. So I started reformatting the query with proper indents and white space. It did contain a few union calls. The first query in the union practically returned the same results. It looked the same except for a few less rows. My null values were still mixed into the good data.

I focused on the table columns that were null and the one column that contained directly related information. The joins between them looked correct. Exactly how I would have written them. After rewriting the query as simple 2 table join that used the same logic, I ended up with the same results. It was a join that I expected to be valid every time but in this case there are times where its not.

It turned out to be a flaw in the application logic where it allowed some changes to one set of tables that should have enforced changes in another. But they were enforcing them at the application level instead. Other parts of the system enforced it correctly, this one point in the application did not. It overlooked the fact that it needed to enforce it and I almost expect the developer to make the same assumption I made.

I know this is a little different from my usual problem/solution posts but this was mostly an internal product and I wanted to document the process I took to solve it.

[OLE/DB provider returned message: New transaction cannot enlist in the specified transaction coordinator. ]
OLE DB error trace [OLE/DB Provider 'SQLOLEDB' ITransactionJoin::JoinTransaction returned 0x8004d00a].
Msg 7391, Level 16, State 1, Line 2
The operation could not be performed because the OLE DB provider ‘SQLOLEDB’ was unable to begin a distributed transaction.

The only message the application gave me was the first line but I uncovered a sql command that let me test it without the overhead of running my application. Here is that command:

BEGIN DISTRIBUTED TRANSACTION
select * from linkname.databasename.dbo.tablename
COMMIT TRAN

I was able to get the inner command to run on its own. As a distributed transaction, it would fail every time.

I eventualy found this KB from microsoft to get me in the right direction. http://support.microsoft.com/kb/839279

You may receive a 7391 error message in SQL Server 2000 when you run a distributed transaction against a linked server after you install Windows Server 2003 or Windows XP Service Pack 2

It turned out to be a settings issue with the Microsoft Distributed Transation Cordinator (MS DTC). I had no idea what that realy was. It turns out MS DTC should be enabled when SQL is installed but by default its locked down (or is with 2003/xp sp2).

I used dcomcnfg to get to the component services. Console Root -> Component Services -> Computers -> My computer -> Properties . On the Default properties enable distributed COM on the computer. Under MSDTC -> Security Configuration check the option for Network DTC Access.

I did make those changes on both servers, I don’t know if that was needed or not. I did the linked server last and it worked as soon as I applied the changes. I know those were the exact changes I made on the 2nd server. The first one prabably has every thing checked or opened up trying to figure it out.

Other thing you may try if having issues it to make sure the Distributed Transaction Coordinator is running as network service and not local system. Reinstalling it as a service and rebooting were a few other tips I read while trouble shooting this problem.

My first guess was to remove a stick of ram. I have heard that before and I was at 4G with the 2 sticks. That did not help this time. I ran the memory tests and everything was ok. I left that second stick out while doing the rest of my tests.

I looked for the error code and I found a KB article (KB930984) that described my exact problem. It indicated that my firmware on my DVD drive was causing the issue. Either update the firmware or copy the files to the disk. I copied all the files to my C: and rebooted. I let it boot from the hard drive and ran into the same issue. I saw another site suggest that the DVD drive needed to be the slave on the IDE cable. Changing the jumper didn’t fix that either. I disconnected the DVD drive and the installer required that I had one.

I thought that was odd. I had an old DVD drive that windows would see but could not see any disk inserted. I set it up and started the install over. It still complained about the DVD drive. After placing a copy of all the files and the main disk and booting from it, the install would still look for the install files on the DVD. I still had the install disk in the drive during some of my tests. That would explain me still getting the error when I thought I had ruled the DVD drive out.

After some more searching, I found the solution. Installing Vista from Hard Disk, USB Disk or Memory Stick. I selected the repair option instead of install after reading that. Ignored the drive selection and started the command prompt. I ran the fallowing commands:


c:
cd /boot
del bcd
bcdedit /createstore
cd ../sources
setup


The install went smooth after that. After the install finished, I reformated and installed Vista 64 bit. I was using my 32 bit disk because its a known good one. I was having issues so I wanted to eliminate variables as much as I could.

I copied all the CD files to the C: again but it failed to boot. I booted to the DVD but droped to a command line and ran the install off the hard disk. I did not have any other issues with the rest of the install. I even did that last install with both sticks of ram.

I expected problems because I knew I did not have the Sub Main defined in my project. I added a class file and entered the fallowing code:

Public Class Main

  Public Shared Sub Main()

     Application.Run(New Form1)

  End Sub

End Class

Several examples I saw had main defined with (ByVal args() As String) as the parameter. One thread on the subject indicated that you can add a module instead of a class. Here is the syntax for that.

Module Main

   Public Sub Main(ByVal args() As String)

      Application.Run(New Form1)

   End Sub

End Module

You will notice that the shared keyword is left off inside the module.

public class TestEvent{    public event EventHandler MyEvent;    public void RaiseMyEvent(){          if(MyEvent != null){         MyEvent(this, null);       }   }}

All the samples I was looking at did not check for a null value, so neither did I. Now that I had the problem, I see why I need to check.

public class TestEvent{   public event EventHandler MyEvent;   public void RaiseMyEvent(){      MyEvent(this, null);   }}

This uses the default EventHandler that returns a object source and a EventArgs ex. If you want to pass back different parameters, you will need to create your own delegate. The delegate we just used looks like this:

public delegate void EventHandler (object source, EventArgs ex);

We dont have to define it because .NET already defined it for us. Here is a second example where we built a custom delegate and raise it.

public delegate void MyCustomEventHandler (string message);

public class TestEvent2{public event MyCustomEventHandler MyEvent2;public void RaiseMyEvent(){MyEvent2("Message sent with event");}}

Now that I explained how to raise events, there is a bug in that code. I’m saving that for my next post: NullReferenceException raising an event in C#