Spring 2008 Toolkit

Below is my complication of free (and mostly open source) development tools and other applications for Spring of '08:

• 7z 4.57 - The most intensive compression tool available
• FireFox 2.0.0.12 - Best internet browser available especially when you install some of the great add-ons out there
  
• Microsoft Enterprise Libraries - Building blocks for building enterprise applications
• MyGeneration - Free/open source code generation for variety of tasks
  
• Nunit 2.4.6 - Unit testing for every version of .Net
  
• NHibernate 1.2 - ORM for .Net
  
• Spring.net 1.1 - framework for doing powerful OOP in .Net including remoting, AOP and more
• WinMerge 2.1.14 - clean & quick merging and diff application
  

Review of SCM & VCS

The basic need for both Version Control Systems (VCS) & Software Configuration Management (SCM) have been needed since the beginning of programming. However in recent years more mature products have hit the market and are worth considering.

1. CVS - old school basic VCS. Not much to delve into here, a quality product. It is a client-server system which is a plus, has no SCM capabilities & is known to have encoding issues from different platforms, basic security implementation. Overall a descent VCS.
   
2. SVN (subversion) - seen as the successor to CVS. Architecture is also client-server, can use HTTP through Apache. No SCM & basic security implementation. A quality VCS.
   
3. VSS (Visual SourceSafe) - basic VCS, some basic security, flawed design requires users read/write access to directories, leaving repository open to tampering. Architecture is client based. Probably the worst of all offerings, however good integration into Visual Studio is cheap and is better than nother.
   
4. StarTeam - my personal favorite, works with Eclipse, Visual Studio, Togethersoft & many other IDEs. Provides full VCS & SCM. This is a very mature product. Integration is seamless & is more intuitive than MS TFS (Team Foundation Server)
5. TFS (Team Foundation Server) - I really wanted to like this one but I didn't. The integration into Visual Studio seems cumbersome, not as seemless as VSS or StarTeam & the bug tracking was horrible. As for the other features if I cannot get passed the bug tracking then cannot be seriously considered a SCM product. This is a version 1 Microsoft product, hopefully it will become more mature over the years. Our QA personnel at the time also despised the product, installing Visual Studio on a QA persons laptop seems backwards.

There are many other products out there, but I think I covered the main ones. I would be more than happy to review any others that are recommended.

Results
The winner here is Borland's StarTeam, it is a mature product has good integration into a wide variety of IDEs and is both a quality SCM & VCS. Whether you are a heads down coder, PM or QA guy you cannot go wrong here. I also want to give a plug to another great product from Borland, Togethersoft. It integrates into the major IDEs & provides object modeling, data modeling as well as code refactoring and generation. I cannot say enough about this product.

For basic VCS needs CVS, SVN or VSS all fit the bill and is better than nothing and cost nothing. Just make sure that you still have a backup plan regardless of which product you decide to deploy.

Regex vs string.replace() Python vs PHP

RegEx or regular expressions have become popular now in every programming language. It involves a special string which identifies ways to match other strings. It's wonderful for performing a match with a sequences of strings especially if the logic for the match is somewhat complex. RegEx can greatly increase the performance of an app or slow it down & obfuscate it.

Before deciding to use a RegEx, first figure out what you want to do. Lets take two common & basic tasks: input validation of email & replacing a matching string.

First, since the rules for email can be somewhat complicated (for string matching) this would make a good candidate for RegEx, especially if the validation is taking place on the client (who cares about client CPU cycles?).

Benchmark results of 10K operations (smaller number is better)
Large String Length: 65181 chars
Small String Length: 97 chars
Simple String: 45 chars
Complex RegEx: email validation is a RegEx 60 chars

Python:
Testing 100K Loop
0:00:00.025730

Small simple replacement
string.replace()
0:00:00.032380

RegEx (compiled)
0:00:00.055416

RegEx (uncompiled)
0:00:00.054967

Large simple replacement
string.replace()
0:00:04.781832

RegEx (compiled)
0:00:03.997719

RegEx (uncompiled)
0:00:03.141389

Complicated RegEx
compiled
0:00:00.034440

uncompiled
0:00:00.080525

PHP
Testing 100K Loop:
0.0390350818634

Small simple replacement
string replace()
0.0481541156769

RegEx
0.0470488071442

Large simple replacement
string replace()
1.68269109726

RegEx (uncompiled)
1.76776599884

Complicated RegEx
0.0417048931122

Conclusion
Overall
Comparing language to language Python is faster in all categories except where the replace expression is simple & the target string is large, here PHP whooped up very nicely. As you can see there is no one size fits all rule, I would expect every language to have sorted results. In preliminary results with C# (.Net 2.0) the RegEx seemed very ineffecient, I hope to post results later.

As to precompiling Reg Ex, it only makes a difference if your expression is complicated, the more complicated your Reg Ex is the bigger boost in performance you get. As these tests show, precompiling your Reg Ex is not always faster.

Note: PHP doesn't support Unicode strings or precompiled RegEx, Python supports both of these, however Unicode RegEx results seemed fairly slow.

In Python
if the target string small & your matching expression is simple use string.replace(), if the target string is much larger then use RegEx uncompiled. With complicated Reg Ex use RegEx compiled.

In PHP
for small target strings with simple matching expression RegEx is slightly faster, differences are negligible. For larger target strings with simple matching string.replace is a bit faster.

Ubuntu Linux 6.06 Review

Ubuntu Linux is the biggest Linux distribution to hit the open source software (OSS) community since , well forever. Ubuntu's motto is "Linux for Human Beings." For many its the usability or ease of use, for others it's the power and flexibility, regardless of what you want it's here.

For the Windows-only users, yes Ubuntu has GUI and rich GUI. OOB it has media, instant messaging, networking, internet browsing (FireFox), Bit Torrenting and Open Office technology. When you want to install Ubuntu it is very straight forward, you boot off of a CD and boot directly to Ubuntu. You can try it out and/or install it now, just double-click the "Install" on your desktop.

Feel free to browse the internet or instant message will this is going on. If you have Windows installed you can either erase it or resize that partition and install alongside it. After it installs you will reboot and that's it

Where to go from here? Browse the Ubuntu Guide wiki which is your best source for most everything you want to do on Ubuntu.

Ubuntu 6.06 also has a server version, this one is commandline only and has an optional LAMP configuration. So next time you want to setup a Linux server consider Ubuntu.

Ubuntu uses a Debian core with Gnome and lots of other nice add-ons. Some commercially based packages to not come included (such as mp3 support) because of legal reasons. Other negatives are nice haves such wireless management GUI needs to be installed (thoguh WiFi itself is supported OOB).

Download Ubuntu (.iso or torrent)
Rating 8/10 (Great)

Coffee

The drug of choice for many of us who choose to go without Ritalin. This upper, served hot normally delivering over 100MG of caffeine per cup.

Unfortunately something goes wrong when people make it. Whether it's chewing on coffee grinds, tastes like cigarette butts or as weak as hot tea at Denny's. Apparently we need help. This covers basic coffee brewing and where to find good coffee.

I'm going to cover two main brewing techniques: Drip & French Press. Most companies employ the drip coffee machine, which is fine. Chose a metal filter over a paper filter, or a natural paper filter over a bleached white paper filter, you get the idea. If using paper filters just use ONE, more than that will over filter the coffee.

Now fill the filter up about 3/4 of the way, I don't like to measure for this because it takes too long and every machine is different, use some trial and error. Always brew coffee on the stronger side, because you can easily make it weaker adding hot water, you can't make it stronger.

Next fill your coffee up with water, if there's not a direct water feed. Do not neglect this step, use filtered or bottle water, I really don't recommend using tap water for this for taste reasons. Finally start the machine, then wholla.

Lastly if your company doesn't have the proper tools for good coffee (i.e. good coffee, machine, filter, water), then use this technique. Buy a Brita water filter pitcher keep a good amount of water in this at all times. Next buy yourself a French Press, whole bean coffee and a grinder. You can however grind the coffee at the store but I recommend fresh grinding before use.

Grind your beans coarsely, then fill your French press with the appropriate amount and fill the glass container with hot boiled filtered water and stir. Let the steep for ~5 min, plunge and serve.

The best way to ruin a good cup of coffee is by taking CoffeeMate (or other artificial creamers) and dumping it into your expensive Java. At this point just pick up some coffee at McDonald's or something, YOU RUINED IT!!! Use a dairy product, preferably Half & Half or Whole milk, but any milk product will be better than coffee mate.

However if you do like CoffeeMate, you can substitute it with some Karo (corn syrup) and then have your coworker spit in it, wholla!!

Lastly, there is the coffee bean itself, the most expensive product to coffee making. Regardless of what commercials tell you, the worst part of waking up is Folgers in your cup, therefore get some good coffee. Starbucks is a good high quality brand so are many local roasters. Smelling the coffee is a good indicator of it's flavor and quality. Whole Foods, Central Market and Costco all sell great coffee, and I don't mean the smelly stuff in the tin cans.

With these tips your can be a better coworker, team member and never ruin someone's morning again.... well not with coffee...

Visual Studio 2005 Settings

Visual Studio 2005 is definitely one of the richest development environments I've ever experienced. This can be a good or bad thing. If you are new to programming a language within Visual Studio 2005 it can be helpful for learning. However if you are an experienced programmer you probably liken it to being babysat. Below are my Visual Studio 2005 Advanced Programmers settings.

TV: Treeview

1. Within Visual Studio 2005 open Options
1. If I have to tell you where this is then you probably don't need these settings
2. Stop showing Start Page at IDE startup
   
1. TV: Environment -> Startup -> change At startup to Show empty environment
3. Show line numbers
   
1. TV: Text Editor -> All Languages -> check Line numbers
4. Change indenting to use Tabs
1. TV: Text Editor -> All Languages -> Tabs -> click Keep tabs
5. Turn off paranoid autocomplete
1. Text Editor -> C# -> IntelliSense -> uncheck Show completion list after a character is typed

Archiving & Compression Utilities

We’ve all been using archiving & compression utilities forever, whether our first one was tar & bz or PKZip. As compression algorithms have evolved newer archiving utilities have come out that promise new functionality and better compression. Two of them are Rar (WinRar) and 7z. A few months ago I switched from WinRAR to 7z.

7z promises better compression and is free/open source. Upon first using it I encountered little difference from other archiving/compression utilities and it supports all the right-click actions you would expect. Up until now I haven’t had a chance to compare RAR compression to 7z since I haven’t compressed anything large and assorted with it.

I needed to zip up BizTalk 2006 Developers edition CD, which is 450MB. This is my first test.

Results:
RAR (Best) compression – 123MB
7z (Ultra) compression – 92MB

Conclusion
7z offers better compression with best compression than RAR and is FREE. Worried about switching to 7z? It can compress 7z & ZIP files as well as extract RAR archives.

http://sourceforge.net/projects/sevenzip/

See also: http://www.maximumcompression.com/

Microsoft Enterprise Library & The Gac

For a while we have been installing a few of the assemblies from the Microsoft Enterprise Library to the GAC. The time came when I broke down and decided to install them all into the GAC.

There are many reasons to install the Enterprise Library into the GAC, they include:

1. Component reuse, less duplicity and more consistency
2. Performance benefits
3. Allows use in other systems such as BizTalk
4. Allows you to consistently manage changes throughout your organization

First step is creating strong name keys for all of the assemblies. From a command prompt (see earlier post regarding setting environmental variables):

sn -k Microsoft.Practices.EnterpriseLibrary.Data.snk

That will do the Data blocks.

Next you must modify the AssemblyInfo.cs file:

[assembly: AssemblyKeyFile("..\\..\\Microsoft.Practices.EnterpriseLibrary.Data.snk")]

Lastly you have to figure out how you want to distribute your assemblies. While you can create an installer to do it, you will need to create Installer classes to do this as you need to install them into the GAC. I think the most pragmatic approach is a batch file. You can create one batch file to install all the assemblies you choose into the GAC.

gacutil -i Microsoft.Practices.EnterpriseLibrary.Data.dll
pause

This installs the data blocks into the GAC and the pauses so that you can see the results if you want to double-click the file from Windows explorer.

In order to install all of the assemblies into the GAC you can manually do this to all of them or you can script it. I have written a small command line application that I will post later that will recursively setup assemblies to be installed into the GAC and includes all of these steps. If you do this installation manually NOTE: you do not want to install into the GAC any assemblies that end in .Design or any of the tools.

Tom Hollander has also blogged about this subject, you might want to see his comments. He is the Product Manager for Microsoft Patterns & Practices.

Using Development Tools via command line

In order to access many handy .Net tools or BizTalk for that matter, you must navigate to the proper directory first, not an option, or launch Visual Studio .NET 2003 Command Prompt.

A way around this is to add Paths to your system Path Environment variables. To do this, do the following:

1. Right-click My Computer, click Properties
2. Click the Advanced tab, then the Environment Variables button
3. Under System variables select Path, then click Edit
4. Add the following to the end of the Variable value field:
5. ;C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\;C:\Program Files\Microsoft BizTalk Server 2004\

Now you can perform the following utilities (and more) from any command prompt: installment, gacutil, ilasm & btsdeploy.

Best Practices

I've been working on a best practices document containing sections for C#, .Net, BizTalk 2004, SQL Server, and IIS. Below is the best practices for each of the following technologies, according to me (Scott White).

Feel free to comment, challenge or add to the respected lists. If change makes sense, I'm more than welcome to make it.

Thanks!

Best Practices: C# & .Net

1. Private and class-level variables should be written using camel casing
2. Public members should be named using Pascal casing
3. Use proper namespacing within all your .Net classes & projects such as: [CompanyName].[BusinessUnit].[ProjectName].[VSNetProjectName]
   Example:
   Microsoft.Office.HelloWorld.Ui
   Microsoft.Office.HelloWorld.Data
4. Create separate projects and namespaces for your UI, Data and Business layers
5. Use DataSet when retrieving results from database and the data needs to be reread, serialized, cached or multiple sets of data need to be related and joined
6. Use DataReader when retrieving results from database and the data needs to be read once and displayed quickly
7. Use XPathDocument when reading XML and needing to extract elements (use XPath, etc) out of the XML document
8. Use XmlDocument when needing to perform two or more of the following: read, write or serialize.
9. Use Command objects to execute SQL code or stored procedures
10. Use EnterpriseLibrary (Application Blocks) in your application where possible, as they are a collection of best practices in working with .Net
11. Use StringBuilder object to construct strings instead of using plus (+) sign to concatenate
12. Use regular ‘for’ loop instead of ‘for each’ loop where possible, as traditional ‘for’ loop is faster and more efficient
13. Avoid using DataAdapter objects
14. Avoid using the DataGrid for inline editing, especially the more complex the process and logic is, as this creates something resembling spaghetti code
15. Add attributes to web service classes and their public exposable members by using [WebService] and [WebMethod]. Properly namespace your web service and document the web service and your web methods this way
16. Add XML Comments to all private and public members
17. Use source code control

Best Practices: BizTalk 2004

1. Although simple mappings can be performed using BizTalk Mapper, use XSLT to perform mappings if they get to long and/or complex
2. Do not use Human Workflow Services (HWS) or even install it, as it is not a good Human Workflow engine and Microsoft is going to deprecate it
3. Fix all SQL Server jobs that are created when you install BizTalk, most of them will not successfully run when BizTalk is initially installed
4. After installation of BizTalk (assuming you installed Enterprise Single-Sign On), make sure you backup and document your master secret
5. Use separate Active Directory Users for each BizTalk service and Groups for BizTalk Hosts. This will keep BizTalk more secure by granted minimum access needed as well as making your BizTalk environment easier to scale out.
6. Create separate BizTalk Hosts for each solution deployed
7. Create separate BizTalk projects and namespaces for Orchestrations, Schemas and Pipelines
8. Following:
9. [CompanyName].[BusinessUnit].[ProjectName].[VSNetProjectName]
   Example:
   Microsoft.Office.HelloWorld.Orchestrations
   Microsoft.Office.HelloWorld.Pipelines
   Microsoft.Office.HelloWorld.Schemas
10. Use Enterprise Library Configuration blocks within your Orchestrations to avoid hard coding information in them
11. To avoid manual deployment with BizTalk, script it (using batch files) or create an MSI package to install it
12. Use late binding for send/receive ports
13. Name your send/receive ports after your solution’s namespace, using Pascal casing
14. Use source code control

Best Practices: SQL Server

1. Casing for SQL Server objects should be similar to that of .Net: Public members (tables, sprocs, views, columns) Pascal cased & Private members camel cased
2. Use Store Procedures everywhere possible
3. Document and model all security access to database objects that need to be granted, then write the script for this
4. Write scripts by hand using a text editor (Query Analyzer, UltraEdit, TextPad, VisualStudio.Net) or an intelligent tool such as Erwin
5. Avoid triggers and give much thought to how you write them if you write them as they can have bad side effects
6. Avoid using cursors within your stored procedures as logical code such as this should normally be performed in your application (middle) tier not your database tier
7. Use source code control

Best Practices: IIS

1. Create entirely separate web sites for each ‘Solution’ or ‘Project’ that is deployed to your web server. Do not put all web sites beneath the ‘Default Web Site’ and ‘wwwroot’
2. Document and then modify the NTFS permissions of your web root directories as needed by your web site
3. Use HTTP Header to host multiple web sites on one server/environment
4. Do not install Front Page extensions on a web server

Overloading Web Service Methods in .Net

A largely misunderstood issue in .Net web services is the capability of overloading web service methods. Overloading web service methods is possible and very easy but it helps to have some understanding of SOAP first.

If you navigate to your web service and click the link "Service Description", or just add ?WSDL onto the end of your web service within your browser, example:

http://localhost/MyWebServices/WebService1.asmx?WSDL

You will see the Web Service Description Layer (WSDL). Inside your WSDL you will see your web service methods represented as operations such as:

<wsdl:operation name="Add">
...
</wsdl:operation>

Now if you go into your Add method inside .Net and add try to overload it, the problem you have is that method names within .Net map to both operations and message elements within your WSDL and you cannot have two message names that are the same, example:

<wsdl:message name="AddSoapIn">
<wsdl:message name="AddSoapIn">

Cannot exist twice. The fix is simple, add a WebMethod attribute to both methods and make sure they have distinct names between them.

[WebMethod(MessageName="Add_TwoNumbers")]
public int Add(int x, int y)
{
return x + y;
}

[WebMethod(MessageName="Add_ThreeNumbers")]
public int Add(int x, int y, int z)
{
return x + y + z;
}

Now your WSDL will have message elements with names:
Add_TwoNumbersSoapIn
Add_TwoNumbersSoapOut
Add_ThreeNumbersSoapIn
Add_ThreeNumbersSoapOut

But your operations (within WSDL) will have the following:
<wsdl:operation name="Add">
<wsdl:operation name="Add">

Thus when you generate your proxy class for your web service (IE: Web Reference, or WSDL.exe) then you can open your proxy class and see how it maps overloaded methods to the operations properly.

While it's not necessary to rewrite the WSDL yourself, understanding how to manipulate the WSDL through attributes within your .Net web service is invaluable.

See Also: WSDL spec from the W3C

Run NUnit Tests on Compilation

Test driven development is an approach to programming where you write your unit testing code to test for the functionality & requirements first. Then code is written to write the unit of code (class, etc).

For .Net programmers, the tool of choice for unit testing is NUnit. For more information about NUnit, see toolkit below.

I recommend creating a project for your unit testing, .UnitTesting will suffice. To automate the process of testing, it is helpful to have unit testing performed upon compilation, thus every time your code changes you can rerun your unit tests and not have to manually rerun NUnit.

To do this, you need to run NUnit's commandline utility, located in NUnit's Program Files folder \bin.

Create your testing class and methods:

[TestFixture]
Class1
{
[Test]
public void Test1()
{
Project1.MyUnit1 minute = new Project1.MyUnit1();
Assert.AreEqual(myUnit.Add(1,2), 3); // assert an exception if 1 + 2 <> 3
}
}

Next, go to the project properties for your NUnit project, click Build Events on the left-hand side. In the Post-build Event Command Line field paste the following:

"%programfiles%\NUnit 2.2\bin\nunit-console" "$(SolutionDir)\Project1.nunit" > "$(SolutionDir)\Project1.nunit.log"

This assumes your .nunit file is in the root of your solution. Variables above might need to be changed to work with your project. Once you build this logic into your solutions, unit testing can become seamless.

XSLT Date/Time & Upper Case functions in .Net Framework

Due to Microsoft's lack of support for XPath 2.0, as well as their dropped support for most of the own extended fuctions, all that remain accessible now is customizable script support in XSLT, for those who need functionality outside of the scope of XSLT 1.0.

To perform many useful functions such as upper case, date/time formatting, etc, you must write custom script. Below is an example of date/time formatting I'm performing within my XSLT document for a BizTalk project, although this will work for any .Net application.

First: Add the following namespaces to your XSLT declaration

xmlns:msxsl="urn:schemas-microsoft-com:xslt"
xmlns:user="http://www.tempuri.org/User">


Second: Write Custom functions

<msxsl:script implements-prefix="user" language="C#">
<![CDATA[
public string FormatDate(XPathNodeIterator node, string xpathExpr, string format)
{
XPathNavigator nav = node.Current;
if(nav.MoveToAttribute(xpathExpr, ""))
{
DateTime dt = DateTime.Parse(nav.Value);
return dt.ToString(format);
}
else
{
throw new ApplicationException("Cannot transform document, specified XPath expression is not valid.");
}
}

public string ToUpper(XPathNodeIterator node, string xpathExpr)
{
XPathNavigator nav = node.Current;
if(nav.MoveToAttribute(xpathExpr, ""))
{
string fullName = nav.Value;
return fullName.ToUpper();
}
else
{
throw new ApplicationException("Cannot transform document, specified XPath expression is not valid.");
}
}
]]>
</msxsl:script>


Finally: Call your custom XSLT function, inline within your XSLT


<xsl:value-of select="user:FormatDate(., 'Value', 'M/d/yyyy h:mm:ss tt')"/>
<xsl:value-of select="user:ToUpper(., 'Value')"/>

Test Driven Development - NUnit

NUnit Pocket Reference by Bill Hamilton, published by O'Reilly is a good overview and reference of test driven development and the usage of the NUnit tool for .Net software development.

Google Print has a section of this book for free and is a good place to start learning test driven development.

Scott's Toolbox - 2nd Quarter 2005

Quarterly I will begin posting a toolbox, common tools, utilities and service packs that I tend to keep downloaded and usually recommend developers to install when they get a new box. This quarters toolbox is listed below.

Altova XML Spy

Microsoft .Net Framework 1.1 SP1

Microsoft BizTalk Server 2004 SP1

Microsoft BizTalk Server 2004 Product Documentation Refresh - 3/28/2005

Microsoft BizTalk Server 2004 SDK Refresh - 4/1/2004

Microsoft MDAC 2.8 SP1

Microsoft SqlXml 3.0 SP3

Microsoft XML Parser 3.0 SP5

Microsoft XML Parser 4.0 SP2

Microsoft Web Service Enhancements 2.0 (WSE) SP3

NDoc

NUnit 2.2

UltraEdit 11

Microsoft removes XQuery from .Net framework 2.0

Microsoft has unfortunately decided to remove support for XQuery (XML Query) language from the .Net Framework 2.0. XQuery was planned to be in Microsoft's next version of the framework and was even somewhat implemented in previous beta versions.

Microsoft has also chosen to not implement XSLT 2.0 standard (4.0 working draft) in .Net framework 2.0.