Closing your Database Connections

Posted in Code - C# by MidniteBlogger on April 5th, 2008

Sometimes working with a managed language like C# can make you lazy. Having cut my programmers teeth on C++, I learned very early that if you “new” something you must also “delete” it. Forget and you get a memory leak.

After moving to C#, however, I occasionally become complacent and begin to think that the garbage collector (GC) will solve all wrongs with my coding style. I re-learned that lesson the hard way with database connections recently, and almost had a site shut down because of it. Perhaps that experience will save you some pain.

Managed code is awesome. You can “new” and allocate all kinds of resources however you want and the GC will just clean up for you. Right? Well, not exactly. It is true that the GC will free “new”ed memory once all references to it are released. The problem (especially with a web app on a busy site) is that by the time the finalizer queue runs, the server box could be out of a scarce resource like database connections and further requests will fail often causing the site to block for extended periods of time.

Let me give you an example. Let’s take a class derived from DbConnection like SqlConnection (for SQL Server databases) or OdbcConnection (for generic connections to databases like MySQL). We’ll use it by creating our own derived class and holding an instance of the connection as a class member. Just to be safe, we’ll let the finalizer close the DB connection for us (since database connections don’t tend to close themselves) so we don’t leak any resources.
Good idea? Let’s see:

public class DBUtilities
{
  private SqlConnection m_dbSqlConn;      

  ~DBUtilities()
  {
     // Our safety net?
      if (m_dbSqlConn != null)
          m_dbSqlConn.Close();
  }     

    public SqlConnection GetSqlConnection()
    {
        try
        {
            m_dbSqlConn = new SqlConnection();
            m_dbSqlConn.ConnectionString = "MyConnectionString";
            m_dbSqlConn.Open();
            if (m_dbSqlConn.State != ConnectionState.Open)
                return null;
        }
        catch (Exception e) {}     

        return m_dbSqlConn;
    }     

}    

//Use the class:    

void UseDatabase()
{
    DBUtilities dbu = new DBUtilities();
    SqlConnection sqlConn = dbu.GetSqlConnection();
    SqlCommand cmd = new SqlCommand("Select * from aTable", sqlConn);
    SqlDataReader dr = cmd.ExecuteReader();
    dr.Read();
    //Do something with the data
    .....
    dr.Close();
}

Cool. The finalizer for DBUtilities will close the database connection for us, won’t it?
Well, sure, but the important question is “when?”. The answer is “whenever the GC gets around to it.” That could be several seconds later, or longer. In my case, the games web site (The best online games) where I was using the code above would simply start blocking after a few database requests. Bad news. It gets worse.

The hosting company (CrystalTech) I was using at the time called me with a problem and an ultimatum. The problem was that my site was hogging resources (I had to figure out that it was database resources) and the ultimatum was that they were shutting off my site until I could prove to them that it was well-behaved again.

Scramble. Scramble.

Fortunately it didn’t take too long to dawn on me that the scarce resource was the database connections, and doing something as simple as calling

sqlConn.Close();

when done with the SqlDataReader reads and taking the Close() call out of the class finalizer solved the entire problem. Hard to believe? It was for me, and it was a hard lesson in non-deterministic finalization.
I think calling Dispose on the connection would have the same effect, but have not proven that yet.

In the end, I switched the site to Godaddy because of CrystalTech’s non-helpful attitude and now everything is humming along smoothly for the happy games players.

midniteblogger

Making AJAX calls from a .NET web page - Part 2

Posted in Code - XML by MidniteBlogger on November 18th, 2007

Continuation of Part 1 

AJAX sample code - calling from a .NET page with C#

Now the good stuff.  How do we implement AJAX in our .NET pages?  The answer is surprisingly and pleasantly simple, as it turns out, and is not actually particular to any one scripting language, be it C#, php, perl or whatever.

The secret to AJAX is the XMLHttpRequest object.  Unfortunately, it has not been implemented in a totally consistent manner across browsers (what else is new?), but as it turns out, the code branching logic needed to overcome this problem is not bad at all.  All we have to do is test for the existence of the appropriate object with an “if” statement and we are ready to roll. 

For the purposes of this simple AJAX tutorial, when my .aspx page loads, I am going to request data from a mythical web service (could be Amazon, Ebay, or any number of others), and present it in an asynchronous manner to the user when it comes in.  If the service is slow or if I am querying many web services in this page, the user will not be left waiting for them all to come back with a response.  The data will show up as it comes in.  

Pieces to the puzzle

The first thing we need is a JavaScript function to call on some kind of event.  I have chosen an “onload” event for the page, but it could be on a button click, edit box “blur,” image load or whatever.  For the example here, all we need to do is put the following in our HTML page body tag:

<body onload=”AsyncDataRequest(1)”> 

When the page loads, the browser will make sure that AsyncDataRequest gets called with a parameter of 1.  For the following functions, make sure all the code is in a <script language=”JavaScript”> </script> tag in the <head> section of your .aspx page.  Also, put a <span id=”ProductInfo1″>  </span> in your page where you want the response data to go.

The data request function simply looks at the page parameter passed in (optional), formats a request url and calls the function that actually makes the Http Request:

function AsyncDataRequest (page)
{
    var varProduct;
    var prodSpan;
    if (1 == page)
    {
        prodSpan = 'ProductInfo1';
        varProduct = 'Product1';
    }
    url = 'http://www.mysite.com/DataRequest.aspx?Query=' + varProduct;
    loadXMLDoc (url, processRequest, prodSpan);
}

We pass in a “page” parameter so we can more easily branch our logic based on what object is making the request.  Here, for page “1″ we specify the name of a <span> tag (”ProductInfo1″) that will be filled in with our data later on and a product tag (”Product1″) that lets us return more specific information from our query.

We simply formulate the URL with the desired parameters and call another JavaScript function “loadXMLDoc.”  That function looks like this:

function loadXMLDoc (url, rsFunc, prodSpan)
{
    var reqHTTP;
    if (window.XMLHttpRequest)
    {
        reqHTTP = new XMLHttpRequest();
        reqHTTP.onreadystatechange = function()
        {
            rsFunc(reqHTTP, prodSpan);
        };              

    reqHTTP.open("GET", url, true);
    reqHTTP.send(null);
    }
   else if (window.ActiveXObject)
   {
        reqHTTP = new ActiveXObject("Microsoft.XMLHTTP");
        if (reqHTTP)
        {
            reqHTTP.onreadystatechange = function()
            {
                rsFunc(reqHTTP, prodSpan);
            };              

        reqHTTP.Open("GET", url, true);
        reqHTTP.send();
        }
    }
}

This function tests for the existence of the XMLHttpRequest object and makes our asynchronous request for data. 
Notice how we have to branch our XMLHttpRequest creation with the if/else if so both IE and Firefox are happy.  With Firefox, XMLHttpRequest is a built-in object, but with IE, it is an ActiveXObject.  Such is life;  just accept it and go on.

The interesting thing here is the assigning of the “onreadystatechange” event of the request object to the function we want to be called when the event fires.  In case you haven’t seen it before, the onreadystatechange = function () syntax allows us to define what is known in JavaScript as an “anonymous function.”   The cool thing is that we can pass parameters to our callback function, and the script code will take care of preserving those parameters for the callback, even though it occurs later. 

Note that “rsFunc” is the parameter we passed in to loadXMLDoc, which we called “processRequest.”  That is the function that will be called when the onreadystatechange event of the request fires (basically just when the request gets more data or an error condition).  More on that in a second. 

The only things left to do here are call “Open” with the URL and “send.”

What does “processRequest” look like? I was hoping you would ask:

function processRequest(reqHTTP, prodSpan)
{
    if (reqHTTP.readyState == 4)
    {
        if (reqHTTP.status == 200)
        {
            response = reqHTTP.responseText;
            document.getElementById(prodSpan).innerHTML = response;
        }
    }
}

This is simple, boiler-plate code that looks for hardcoded numbers in the response. The readyState of the request has to
be 4 and the status has to be 200. Don’t worry about it, just accept it. Only under those conditions will the request give you good data.
Notice that we passed the request object that we created in loadXMLDoc to this function along with the <span> id originally specified in AsyncDataRequest.
Here, we get the response text from the request object, get the element of the document specified by id (prodSpan) and set its innerHTML to the response text. Voila. The text magically appears in the browser in the place where the <span> tag is.  Very cool.

Last piece to the puzzle — data!

So what does the code in DataRequest.aspx look like?  That’s mostly up to you.  That file does the dirty work of retrieving data from a database, 3rd party web service, validating user input against business rules, etc., etc.  Here’s a sample of what it might look like:

      String strQueryString = Request.QueryString["Query"];
      switch (strQueryString)
      {
        case "Product1":
            Response.Write(MyObject.GetAmazonData("Books");
            break;      

        //Other product feeds responses here      

        default:
            break;
      }

Here we are just branching our logic based on the Request string sent in the URL we formatted in the AsyncDataRequest function. Feel free to substitute whatever Request QueryStrings suit your needs.
In the GetAmazonData function you would request your data from the appropriate web service, format it (HTML would be desired here) and write it to the Response stream. Every thing else is automatic. Awesome.  You have to supply the logic in MyObject, however. :) 

It’s OK for the GetAmazonData function to block for as long as it needs to, because the user is no longer waiting for the page to load.  This is the sweet part.   Here you could also do secret calculations or password validation, etc.

If you have done everything correctly, whatever HTML you put in Response.Write will show up in the ProductInfo1 <span> tag.

This should be a complete example that you can slice and dice to fit your own needs. If not, please post and let me know. Also, if you have found this helpful, please post your experience for others to read.
As you can see, this could easily be adapted to use in a php environment. The only thing that would be different is the trivial differences in syntax for the DataRequest page.

As an example, this link leads to a games site that has implemented Amazon Web Services as a monetizing tool to present games for sale:
Time Killing Games

Enjoy

midniteblogger

Making AJAX calls from a .NET web page - Part 1

Posted in Code - XML by MidniteBlogger on November 16th, 2007

AJAX, (which stands for “Asynchronous JavaScript and XML”) is a really cool technology.  Although not a perfect solution, AJAX has solved some difficult problems related to creating a “rich” user experience in a web browser.  

With a traditional web page, the browser makes an http request to the server.  The web server delivers the HTML content to the browser and the connection between the browser and the server is then broken.  If the user performs some action on the page (e.g. fills out a form), the browser has to do another full round trip to the server, resulting in a page refresh, in order to get any information back to do something as simple as data validation.   This is a somewhat less than satisfying experience for the user, especially if the connection is slow.

Enter AJAX. 

In this posting, I intend to cover the following subjects relating to AJAX:

-What AJAX is
-Reasons you would use AJAX
-Sample AJAX code that you can use on a .NET (.aspx) web page

A brief perspective

When I was first introduced to dynamic web page creation some time ago, I was very excited about the possibilities.  No more static .html pages and no more stale content.  Through the magic of the .php or .NET server pages, I could now introduce logic, database query results, 3rd party content and many other things on my sites.  This immediately led me to try integrating web service content from such vendors as Amazon in an effort to monetize my traffic.

While the integration was successful, the results of doing so were less satisfying — especially when querying more than one remote web service from a single web page at a time.  If there was any latency in one of the query connections, the whole page would block until all of the data was returned.  From a user’s perspective, this often meant that they could be left staring at a blank web browser (especially noticable under IE) for as much as 20 seconds — just waiting for the multiple query results to come back.   

How AJAX can help

What is AJAX?

As mentioned above, AJAX stands for “Asynchronous JavaScript and XML.”  This means that it is JavaScript, executed in the browser, called in a manner that returns asynchronous results, frequently formatted as XML.  Most modern browsers (IE, Firefox, Safari, Opera) support JavaScript, so it should be a seamless experience for your site users. 

The bad thing about using traditional JavaScript is that all your business logic is delivered to the client browser.  This makes it hard to manipulate any kind of passwords or trade secrets such as formulas with it.  The cool thing about what AJAX adds, is that it lets you keep all your business logic, password validation, secret calculations, etc. where they belong — on the server. 

Reasons you would use AJAX

Here are a few reasons for using AJAX on your web pages:

-You want to integrate more than one 3rd party data source from typical Web service suppliers and want to avoid the multiple connection latency problem.
-You want to do some kind of data validation of user input (e.g. filling out a registration form) without either delivering all the logic to the client in JavaScript or making a round trip to the server with the resulting full page refresh.
-You want your web application to have the more natural feel and responsiveness of a traditional GUI desktop application.

More in Part 2

Parsing Amazon Web Services (AWS) with C# and .NET

Posted in Code - C#, Code - XML by MidniteBlogger on July 29th, 2007

Amazon web services are one of the most feature and information-rich sources
of queryable product data available. They even have user and editorial reviews that
you can present to your customers to induce them to buy through your site. The
best part is that it’s free for developers (Get it at Amazon.com).
Here I want to present a short tutorial on how to access the data in AWS with
C#. It’s not really hard to use at all from .NET, and in fact is quite a bit
easier than some of the feeds I have dealt with before (e.g. Apple iTunes feed).
There are no namespaces in the nodes to worry about so our job is a bit simpler.

I know using XSLT might be preferable, but someone just might be looking for a
DOM-centric way to parse an XML feed.
In order to query the Amazon information database, you need an AWSAccessKeyId.
In order to get paid for any sales originating from your site, you need an
Amazon AssociateTag.
So the query URL would look something like this (substitute your AccessKeyId for
“12345″ and your AssociateTag for “mytag-20″):

http://webservices.amazon.com/onca/xml?Service=AWSECommerceService
&AWSAccessKeyId=12345&AssociateTag=mytag-20

For this operation we will search for some items in a category (Mario VideoGames
in this case), so we append to the URL:
&Operation=ItemSearch&SearchIndex=VideoGames&Keywords=Mario

We also want to know about the items and get some prices and images so we tack
this on to the URL:
&ResponseGroup=Request,ItemAttributes,OfferSummary,Images

Lastly, we desire to see some reviews done by customers and editorial staff so
we append Reviews,EditorialReview

So the final URL looks like this:

http://webservices.amazon.com/onca/xml?Service=AWSECommerceService
&AWSAccessKeyId=12345&AssociateTag=mytag-20&Operation=ItemSearch
&SearchIndex=VideoGames&Keywords=Mario
&ResponseGroup=Request,ItemAttributes,OfferSummary,Images,Reviews,EditorialReview

The response is big, so I’ll just deal with the relevant parts here. Let’s
suppose we want to get the following information from the first item in the
feed:
-DetailPageURL (The page you direct your users to go to buy the item)
-SmallImage (Graphical representation of the product)
-Title of the game
-LowestNewPrice (lowest price for the item on Amazon)
-Text of a customer review of the item

The simplified feed looks like this:

<ItemSearchResponse>
  <Items>
    <Request>
      <IsValid>
       True
      </IsValid>
    </Request>
    <Item>
      <DetailPageURL>
        http://www.amazon.com/gp.......more stuff
      </DetailPageURL>
      <SmallImage>
       <URL>
         http://ec1.images.... more stuff
       </URL>
      </SmallImage>
      <ItemAttributes>
        <Title>
          Super Mario Bros.
        </Title>
      </ItemAttributes>
      <OfferSummary>
        <LowestNewPrice>
          <FormattedPrice>
            $26.90
          </FormattedPrice>
        </LowestNewPrice>
      </OfferSummary>
      <CustomerReviews>
        <Review>
          <Content>
            Bla Bla Bla. This is the greatest game ever.
          </Content>
        </Review>
      </CustomerReviews>
    </Item>
  </Items>
</ItemSearchResponse>

What to do first? Let’s open the XmlDocument:

String strURL = "http://webservices.amazon.com.....<see above>";
XmlDocument doc = new XmlDocument();
doc.Load(strURL);         

//See if the response is valid - Amazon sends this info in a
//node of the XML tree:          

XmlNodeList lstIsValid = doc.GetElementsByTagName("IsValid");
//This is nice because .NET automatically drills down a couple
//of levels to find the "IsValid" node tag         

if (lstIsValid.Count > 0 && lstIsValid[0].InnerXml == "True")
{
  //InnerXml holds is the actual data from the node
  //Ready to party on the data because the response is valid         

  XmlNodeList lstItems = doc.GetElementsByTagName("Item");
  //Also automatically getting info a couple of levels
  //deep - sweet
  //Get a NodeList of the "Item" nodes - they hold our info         

  if (lstItems.Count > 0)
  {
    //could use a foreach loop here to get them all
    XmlNode nItem = lstItems[0];
     //all children of the first "item" node      

    foreach (XmlNode nChild in nItem.ChildNodes)
    {
      //Clunky way to find a child node.
      //This is the best I could
      //come up with from the MSFT docs.
      //Anybody have a better way?
      if (nChild.Name == "DetailPageURL")  //string compare !
      {
        String strURL = nChild.InnerXml; //our data !
      }
      else if (nChild.Name == "SmallImage")
      {
        //Now we have to look through the children of the
        //SmallImage node:
        foreach (XmlNode nURLImg in nChild.ChildNodes)
        {
          if (nURLImg.Name == "URL")
          {
            String strImage = nChild.InnerXml;  //our data !
          }
        }
      }
      else if (nChild.Name == "ItemAttributes")
      {
        foreach (XmlNode nIA in nChild.ChildNodes)
        {
          //Look through each "ItemAttributes" to find the one
          //we want again
          if (nIA.Name == "Title")
          {
            String strTitle = nIA.InnerXml;  //our data !
          }
        }
      }
      else if (nChild.Name == "OfferSummary")
      {
        foreach (XmlNode nOS in nChild.ChildNodes)
        {
          //looking again
          if (nOS.Name == "LowestNewPrice")
          {
            foreach (XmlNode nLNP in nOS.ChildNodes)
            {
              if (nLNP.Name == "FormattedPrice")
              {
                String strPrice = nLNP.InnerXml;  //our data !
                break; //done looking here.
              }
            }
          }
        }
      }
      else if (nChild.Name == "CustomerReviews")
      {
        foreach (XmlNode nCR in nChild.ChildNodes)
        {
          if (nCR.Name == "Review")
          {
            foreach (XmlNode nRev in nCR.ChildNodes)
            {
              if (nRev.Name == "Content")
              {
                //Our review text!
                String strReview = nRev.InnerXml;
              }
            }
          }
        }
      }
    }
  }
}

You get the idea. Do what you need to do with the data, but this is how to
get it out. I liked the fact that C# would “drill down” a couple of levels to
the relevant child nodes without me precisely specifying like I would have
had to do in XSLT. I did NOT like having to iterate through the nodes and do
string compares to find child nodes. Perhaps there is a better way, but it is
the only one I could find in a lot of searching.

If you want to see the results of parsing the feed in a live site, check out
this games web site that offers game reviews, courtesy of AWS:

midniteblogger

Using C# to parse XML feeds.

Posted in Code - C#, Code - XML by MidniteBlogger on June 10th, 2007

What with the big push to have XML adopted as a cross-everything data format, it seems like it should be fairly straightforward to parse an XML file in .NET.  For the most part, I would say that it is, but I ran across a situation recently that was a bit difficult to solve — mostly because of the lack of available information on the subject.
The problem is parsing an XML file that has namespaces embedded in the nodes.  Unusual, you say? 

That’s what I thought until I came across the RSS iTunes feed from Apple. 
It’s not totally clear to me why they want to embed namespaces in some of the nodes, but it is the way they do it, nonetheless.
RSS, as you probably know, is just a standard for formatting data delivered by content providers (e.g. Web services).  It has some basic, standard node names, but is also extensible so that a consumer of the data can extract the extra information if it is aware of it.
I won’t go into the details of RSS here, but basically, each item in the feed is sandwiched between <item> node tags (this is RSS 2.0), with associated <title>, <description> and other nodes.
In the case of the Apple feed, a number of the proprietary nodes are formatted with namespaces like so: <itms:artist>, <itms:artistLink>, etc.  I had never seen a node with a namespace before (don’t get around much, I guess), but it didn’t seem like it should be too hard to extract the data.  Boy, was I wrong !
Just finding out how to do this took hours of searching around on the web. I finally found a small example on a non-English web site or I would probably still be looking.  All the Microsoft examples of XML parsing I came across didn’t have any nodes with namespaces on them. PHP doesn’t have any trouble with the namespace qualified nodes, but C# is a very different story :)

I am going to present some small code samples to fix this problem that might save you a lot of trouble trying to figure out how to do the same thing. (I realize XSLT would be easier, but you might need to do it this way for some reason.)

So the Apple RSS feed (simplified) looks something like this:

<rss version=”2.0″  other stuff…>
  <channel>
    <item>

      <title>
      Title here
      </title>

      <itms:artist>
      Artist name
      </itms:artist>

      <itms:album>
      Album name
      </itms:album>

      <itms:releasedate>
      Release Date here
      </itms:releasedate>

      <itms:coverArt height=”100″ width=”100″>
      theImage.jpg
      </itms:coverArt>

    </item>

…More items here

  </channel>
</rss>

The nodes with the <itms: > namespace are the proprietary ones. 

I’m just trying to get the “theImage.jpg” string from the <itms:coverArt> node.  Simple?  Let’s see…

Loading the document is easy:

XmlDocument doc = new XmlDocument();
doc.Load("MyXML.xml");

Getting the item nodes is easy:
//RSS 2.0
XmlNodeList items = doc.SelectNodes("/rss/channel/item"); 

Now I have a collection of the “item” nodes.

Looping through them is easy:
foreach(XmlNode itm in items)
{
  try
  {
        //Here’s the tricky part.
        //To get the data in the nodes with namespaces, I have to jump through some hoops!
    XPathNavigator nav = itm.CreateNavigator();

        //Here I am setting up IXmlNamespaceResolver to pick nodes that have a
        //namespace of “itms”
        //Don’t ask me to explain the syntax.
    IXmlNamespaceResolver  nsResolverImg =
        nav.Clone().SelectSingleNode("//*[namespace::itms]");

        //Picking a node that is a “coverArt” node, in the itms namespace with an
        // attribute of height=100
    XPathNavigator img =
       nav.SelectSingleNode("itms:coverArt[@height='100']", nsResolverImg);

    String strImage = img.InnerXml;  //retrieves "theImage.jpg"
  }
  catch (Exception ex)  {  }
}

Simple?  Yes, once you know the trick.  Intuitive it is definitely NOT.
What’s with all this //*[namespace::itms] stuff anyway?  Why can’t I just call this:
XPathNavigator img =
nav.SelectSingleNode("itms:coverArt[@height='100']");
without all the IXmlNamespaceResolver stuff?  Isn’t that just as clear?  Well, apparently the designers at Microsoft didn’t think so. 

Hope this has helped you in solving a similar issue. If you run across any variations of this, please post them for the benefit of all.  I would especially like to see some details on how the “//*[” works with other scenarios in the SelectSingleNode method.

Microsoft guys: How about some better docs - with examples - on this stuff?

midniteblogger.

Getting Vista to recognize more than 3 GB of RAM?

Posted in Vista by MidniteBlogger on June 9th, 2007

Having recently purchased a new Gateway desktop computer recently, I was pleased to learn it had Vista Ultimate loaded on it.   This is Microsoft’s top of the line OS so I was expecting great things.
Having also heard that Vista really sucks up the RAM, I purchased way more than recommended — 4 GB.
After firing up the PC and booting into the OS, however, I was disappointed upon checking Task Manager and learning that Vista was only recognizing 3 GB of my RAM!  Bummer.
BIOS told me there were 4 GB present, so it must be a 32 bit Vista issue with the OS only seeing 3.
Being a programmer, I concluded that 32 bit Vista could only handle 3 GB of RAM.  (Seemed to make sense at the time but I started thinking, why should it be a problem for this cool new OS to see all the memory in my machine?)
Because I was at Tech Ed in Orlando this week, I decided to ask the friendly folks at the Microsoft Vista booth about my dilemma in losing the use of 1 GB of RAM.
The guy in the booth didn’t really understand why it wasn’t working and thought Vista might need some kind of /3GB entry in the boot.ini file.
 He looked around for awhile on Live Search and came up with a couple of articles on MSDN about modifying the boot.ini that looked a bit dated on the subject and not totally relevant.
In the end, he wasn’t much help at all.
I’m having a hard time believing I have to spring for a 64 bit OS just to get the use of all 4 GB of RAM (Couldn’t even NT 4 use more than that?)
Would love to hear from anyone with ideas on this.
Midniteblogger

Hosting an Active-X Control on an Outlook Inspector page with C#

Posted in Code - C# by MidniteBlogger on May 5th, 2007

This article is going to carry on some ideas from earlier posts. Those posts dealt with how to customize an Outlook Contact by adding a custom tab. If you missed the earlier information, you can catch up on it here: http://www.midniteblog.com/2007/04/12/writing-an-outlook-add-in/.

Now we are going to try to something with the blank tab we added. Let’s add an Active-X control to the tab. Assume you already have an Active-X control with a Progid of “MYCONTROL.MyControlCtrl.1″ and it does some super special thing that no one else in the world can do so you want to add it to your custom Outlook tab.

Easy enough. We are working with C# here, so add the following 2 reference lines at the top of your connect.cs file:

using Outlook = Microsoft.Office.Interop.Outlook;
using System.Reflection;

Add project references to Microsoft.Office.Interop.Outlook.dll and Microsoft.Vbe.Interop.Forms.dll.
Add the following 2 members to your Connect class:
Outlook.Application m_outlook;
Outlook.Inspectors m_inspectors;

Add the following to your OnConnection method:
m_outlook = application as Outlook.Application;
m_inspectors = m_outlook.Inspectors;
m_inspectors.NewInspector +=
new Microsoft.Office.Interop.Outlook.
InspectorsEvents_NewInspectorEventHandler
(Inspectors_NewInspector);

Here we are telling Outlook to notify us when a “NewInspector” event fires. This happens basically every time someone double-clicks to open up an Outlook Contact.

Add the following class method to Connect class:


public void Inspectors_NewInspector(Outlook.Inspector inspector)
{
  if (inspector != null && inspector.CurrentItem is Outlook.ContactItem)
  {
    Microsoft.Vbe.Interop.Forms.UserForm userform;
    object obTemp = ((Outlook.Pages) (inspector.ModifiedFormPages)).Add("My Page");
    userform = (Microsoft.Vbe.Interop.Forms.UserForm) obTemp;
    Microsoft.Vbe.Interop.Forms.Control mc;

    try
    {
      mc = userform.Controls._GetItemByName("myControl");
    }
    catch (Exception)
    {
      mc = null;
    }

    if (null == mc)
    {
      mc = (Microsoft.Vbe.Interop.Forms.Control) userform.Controls.Add("MYCONTROL.MyControlCtrl.1", "myControl", true);
    }

    inspector.ShowFormPage("My Page");
  }
}

So in this method we are adding our custom “My Page” to the Outlook inspector window (as we did before) and now we are asking the “Controls” collection of the userform to add an Active-X control by Progid (”MYCONTROL.MyControlCtrl.1″). This will CoCreate the AX control and parent it to the Outlook custom tab ! There’s your custom control hosted in Outlook - cool.
Note that if you do not do the check for finding any existing control (e.g. if this is not a new contact) with _GetItemByName you will get an additional control created for every time you open the contact.
This seems very counterintuitive, but apparently Outlook persists the state of each contact page, and when you “Add” a page on a subsequent open of the contact, it loads any controls that were there before.

Now let’s suppose you want to tell the AX control to do something. Let’s further suppose that the control exposes a method called “ShowMyData” that takes a BSTR in param. Continuing on with the example, add the following just before the ShowFormPage call we added above:
Type ty = mc.GetType();
object[] paramList = new object[1];
paramList[0] = "My Data";
ty.InvokeMember("ShowMyData",
BindingFlags.Default | BindingFlags.InvokeMethod,
null, mc, paramList);
Here we are using reflection to bind to the control at run-time and then invoking on it. The exposed method is called “ShowMyData” and the parameter string is hardcoded here to “My Data.” When the AX control gets the string, it can do with it what it likes. Amazing that with the big push to .NET, we still have to use interop to customize Office, but if you want to host a legacy Active-X control on an Outlook custom tab, that’s the way to do it.
midniteblogger

Problem deploying Visual C++ application compiled under Visual Studio 2005

Posted in Deployment by MidniteBlogger on April 24th, 2007

Target machine is a clean install, Windows XP Tablet PC edition, never had any development tools on it. After doing XCopy deploy of .exe and associated .dlls, trying to run the application gives error Message Box:
“The application failed to initialize properly (0xc0150002). Click on OK to terminate the application.”
From the Microsoft posting “How to: Deploy using XCopy” http://msdn2.microsoft.com/en-us/library/ms235291(VS.80).aspx, you might think that running vcredist_x86.exe (comes with Visual Studio setup) would solve the problem. It doesn’t.
Going to try the dependency walker tool to see if I can determine what else might be needed.

It turns out that the dependency walker (”Depends”) tool was very informative and pointed out missing dependencies msvcr80.dll, msvcp80.dll and winmm.dll. However, that was only half of the story.
My .exe still wouldn’t run - it kept giving the same “The application failed to initialize properly (0xc0150002).” error. Hmmmm.

Time to pull out the File system monitor, otherwise known as Filemon (http://www.microsoft.com/technet/sysinternals/FileAndDisk/
Filemon.mspx). This is a truly amazing tool that monitors all file system activity. It has saved my rear more than once.
After working with it for a while, I kept noticing a NOT FOUND result in Filemon when csrss.exe was trying to access C:\WINDOWS\WinSxS\Policies\x86_policy.8.0.
Microsoft.VC80.DebugCRT…. Suddenly a light bulb came on - Policies…Debug…Hmmm. Sure enough, recompile in release and redeploy the app - now it runs fine on the target machine.
Looks like there are policies that need to get set up in order to run debug code on a machine.
Live and learn.

Writing an Outlook Add-in with C# (Epilogue)

Posted in Code - C# by MidniteBlogger on April 18th, 2007

Outlook Add-in Deployment issues

So after you’ve built your new whiz-bang Outlook Add-in, you will want to deploy it to your user’s machines. It runs fine on your development machine, but when you copy it (or use the installer to install it) to another box, it doesn’t run so fine. Here are some deployment issues I ran into that weren’t really documented very well. Hopefully they will save you some time.

- Outlook recognizes your Add-in and shows it in the COM Add-ins dialog, but won’t load it. The COM Add-ins dialog will say something like “error in loading.” This is a perplexing one and it was quite a problem to solve. I ended up downloading a file system sniffer and discovered that Outlook was trying to load something called “Extensibility.dll.” What in the world is that?
Technically speaking: “This assembly is necessary for referencing the IDTExtensibility2 interface that shared add-ins must implement, as well as creating command bars and command bar controls that provide the primary interface for users of the managed COM add-in in the application.”
In other words, it is a Microsoft file that Outlook needs to talk to plug-ins. It is also, apparently, not on the list of redistributable components. This is quite an unfortunate circumstance.
From what I have read on the Microsoft team forums (e.g.
http://forums.microsoft.com/MSDN/ShowPost.aspx
?PostID=1436656&SiteID=1
)
there isn’t a really good solution.
The best one I saw was to install the “VS SDK” which contains an MSI that contains Extensibility.dll. Are you kidding? Install the VS SDK on a user’s machine so an Outlook Add-in will load????
If anyone else has better ideas please post them. (See “The Answer” below)
- Does the Outlook Add-in have to be in the GAC? As best as I can tell, the answer is No. I never had to put it there to get it to load and run. I did, however, strong sign the Add-in because of the additional security that provides, and it will need to be strong-signed if you are going to put it in the GAC.
- A dependency I noticed only on Vista was msvcr80.dll. I suspect it will be needed on other OSes as well, however.

Development Issues

Here are some issues I ran into when developing the Outlook Add-in.

- When you add a reference to Microsoft forms (fm20.dll) (if you are going to place an Active-X control on your new Contact tab), make sure it is the reference that creates the Microsoft.VBE.Interop entry in the Object Browser (View/Object Browser). There was another Microsoft forms to choose from when I added it in the “Add Reference” dialog and apparently I picked the wrong one because it did not work until I took out the incorrect reference and added the other one that creates the Microsoft.VBE.Interop reference in the Object Browser.
- To debug your Add-in with Visual Studio, go to Project/Properties/Debug tab and under “Start Action” choose “Start external program” and put the path to Outlook on your machine in the edit box there. This will make your life much easier in fixing problems.
- If your Add-in suddenly stops working it is possible that Outlook unloaded it (Outlook will mercilessly unload any Add-in it thinks has misbehaved). In Outlook, go to Tools/Options then “Other” tab, choose the “Advanced Options” button, then the COM Add-Ins button which will bring up the COM Add-ins dialog. Make sure your Add-in is still checked.
- If Outlook really thinks your Add-in has been bad, it will totally disable the Add-in. In this case (this was a hard one to figure out because of the lack of docs) go to Help/”About Microsoft Office Outlook” and click on the “Disabled Items” button there. If your Add-in was very naughty it might end up here in the Disabled Items list. Click on it and then the Enable button to reenable.
- I did NOT need to use caspol to set code access security policies. Not to say that it won’t be needed at all, but I don’t think it is as critical to getting your app running as some of the discussions would lead you to believe.
- In Outlook 2007, in order to get your Add-in recognized, you have to go to Tools/Trust Center…, then click on Add-ins and make sure your Add-in is in the “Active Application Add-ins.” If not, pick “COM Add-ins” from the drop down at the bottom, then click on the “Go” button. This will bring up the familiar “COM Add-ins” dialog. Pick your Add-in from the “Add-Ins available” list and you should be good to go.

Security Issues

- Under the Outlook dialog found at Tools/Macro/Security on the “Security Level” tab, if “High” is checked and on the “Trusted Publishers” tab, if “Trust all installed add-ins and templates” is NOT checked, Outlook will prompt the user on startup to see if they want to run the add-in. This is a pain, but just the way Office security works.

Vista-specific issues

- When you try to register (with regsvr32.exe) your unmanaged C++ shim (if you wrote one) in Vista with UAC on, you will have permission issues and it will not register. According to Microsoft, this is “expected” with UAC on. They suggest changing the way your component registers (presumably not trying to write to HKLM), or to run regsvr32.exe from an “elevated command prompt” which you get by right-clicking on command prompt icon and choosing “Run as Administrator.”
They also suggest running Visual Studio in “admin mode” which you can do by right-clicking on Visual Studio icon and choose “Run as administrator” from the context menu. (More info here:
http://blogs.msdn.com/vcblog/archive/
2006/09/06/742187.aspx
).

General Info

- Registry key where Outlook tries to find add-ins:
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Office\
Outlook\Addins

The Answer:

How to redistribute Extensibility.dll
After looking long and hard for the answer as to how to redistribute Extensibility.dll (which shared Office add-ins need in order to reference IDTExtensibility2), I decided to give in, grit my teeth, and submit the question to Microsoft support. After spending 2 hours on the phone and getting transferred no fewer than 5 times, someone that was reasonably intelligent-sounding promised to get right on the problem and get back to me.
After a day I got another e-mail from someone else in Microsoft support asking again what the problem was, to which I replied “Is Extensibility.dll redistributable or not” and, “If so, is there an install for it?”
After being sent yet again to another support engineer, it began to look like we might get a real answer. Another couple of days went by trading e-mails with this new person, and I finally got the answer.

Here it is, in case you are trying to solve the same problem.

MSFT support claims extensibility.dll is redistributable, but you have to do it through a specific .msi (no explanation as to why we need the .msi as opposed to just distributing the .dll).

The KB fix (VS2005-KB908002-ENU-x86.EXE) at the following address provides a copy of the needed .msi along with some other Office fixes:
http://support.microsoft.com/kb/908002/en-us

The fix is called “Shared Add-in Support Update for the Microsoft .NET Framework 2.0 (KB908002),” and Microsoft says that it provides the required fixes in Office and the .NET Framework together with the solutions that you write in Visual Studio 2005.

When you run the .exe there as instructed, it gives you an .msi (extensibilityMSM.msi), as expected, in C:\Program Files\Microsoft Visual Studio 8\SDK\v2.0\BootStrapper\Packages\KB908002. According to my support contact, this .msi is distributable (There is a eula.txt there in the “en” directory underneath the .msi with the legal details).

So the answer is to let your install kick off extensibilityMSM.msi and then Extensibility.dll should be in place on the user’s machine.

What an ordeal just to find out such a simple answer from Microsoft support.
Microsoft
: Is a week’s response time the best you can do to support engineers that pay a premium for your services — especially on such an easy question? Also, would it be too much to ask to bring some of the support back to native English speaking countries so we are not trying to communicate with ESL people in India? I will certainly be advising my organization against buying Microsoft “support incidents” for all our developers in the future. Better to pay for the incidents on an emergency, last-ditch basis.

midniteblogger

Writing an Outlook Add-in with C# (Part 4)

Posted in Code - C# by MidniteBlogger on April 17th, 2007

In the last 3 sessions, we have developed an Outlook Add-in that adds a tab to a Contact window when it is opened. We began to discuss some further issues with which we must grapple.

Problem #1. There is a Microsoft documented problem with Outlook hanging in memory after closing it even if an Outlook Add-in is properly written. It doesn’t always happen, but it is a known issue and can really be annoying. The answer here is to implement a manual “disposer” class. More on this later.

Problem #2. Because our Add-in is in C# Outlook doesn’t load it directly (discussed earlier). Outlook really loads mscoree.dll instead of our Add-in, and mscoree.dll loads our Add-in. Why is this a problem? As noted before, if Outlook thinks our Add-in misbehaved, it will unload mscoree.dll (our Add-in loader).

What is worse, Outlook will then refuse to load ANY other .NET Add-in. This also means that another managed Add-in that misbehaves could also cause our Add-in not to get loaded. (Supposedly this has been remedied in Outlook 2007). The answer here is not exactly easy. The only real solution I found was to write a shim in unmanaged C++ that would be registered with Outlook, and it would, in turn, load our managed Add-in. It does work well. I can post on this if anyone is interested.

So let’s look at Problem #1 - Outlook hangs in memory.  Amazing but true.  There is a problem with Outlook and the way it handles unloading Add-ins.  What makes it worse, is that it doesn’t always happen.  Yuck.   You just have to deal with it.  So what do we do?

This is a problem that Microsoft has documented, and this is the solution I have come up with (with the assistance of the Microsoft Office site).  So if you have been following along, you may have been wondering why we registered for
ExplorerEvents_10_Event_Close in the first place. Your patience is about to be rewarded. In the event handler
Explorer_ExplorerEvents_10_Event_Close we need to do the following. First, tell the explorer class we don’t want any more callbacks like so:

m_explorerclass.ExplorerEvents_10_Event_Close
-= new Outlook.ExplorerEvents_10_CloseEventHandler(
Explorer_ExplorerEvents_10_Event_Close);

We also have to unregister for “NewInspector” events:

m_inspectors.NewInspector -= new
Outlook.InspectorsEvents_NewInspectorEventHandler(Inspectors_NewInspector);

Then, in the part that really hails back to the days of COM, we have to manually dispose of any object that has registered as an event listener. You heard me right. So much for the glories of managed code. It’s time to get your hands dirty! For all you who have never had the pleasure of releasing a COM object, your time has come!

What we need to do next is, in the same
Explorer_ExplorerEvents_10_Event_Close callback, manually release the m_explorerclass and m_inspectors objects. They were the objects that originally registered as event listeners (see Part 3 of this article if you missed that).
Here’s how we release the objects:
Make sure you have a “using” statement for System.Runtime.InteropServices and then put in the following code.
try
{
  if (m_explorerclass != null)
  {
    int iCount = Marshal.ReleaseComObject(m_explorerclass);
    while (iCount > 0)
    {
     iCount = Marshal.ReleaseComObject(m_explorerclass);
    }
  }
}
catch (SystemException ex)
{
}
finally
{
  GC.Collect();
  GC.WaitForPendingFinalizers();
}

You have to do this for the m_explorerclass and m_inspectors objects. What we are doing here is performing an old-time COM Release(), then telling the managed garbage collector to do its thing and waiting on it.  This is actually the solution Microsoft proposes as the answer to the problem of Outlook hanging in memory when it is supposed to exit.

If you don’t believe me you can read more about it on the MSDN technical articles site (http://msdn2.microsoft.com/en-us/library/aa155703(office.10).aspx).  Just look for Calling UnInitHandler and Disposing of Objects. Even though this Microsoft article is about Outlook 2002, I don’t see any indication that it is different for Outlook 2007.  If so, someone please correct me.

There were a few other strange things about this manual disposing business.  I ended up putting the above code in a routine and calling it both from Explorer_ExplorerEvents_10_Event_Close and Connect.OnDisconnection just to make sure it always happened.

See the epilogue for final issues and comments.