
In a recent post to the XP mailing list, Adrian Howard said:
Many people seem to have a great deal of trouble going “meta” with mistakes. People will happily write code, discover it has bugs, spend half an hour with debugger finding bugs, fix bugs and repeat without ever saying to themselves “Is there a way I could stop adding those bugs in the first place”.
How often do you see this within your own team, and within yourself? I know that I am certaintly guilty of this. I’ll write something like:
using(SqlConnection conn = DataAccess.GetConnection())
{
using(SqlCommand command = conn.CreateCommand())
{
command.CommandText = foo;
command.ExecuteNonQuery();
}
}
So what’s wrong with it? It’ll fail every time because the connection is not open (unless CreateCommand() opens the connection when it returns it). And the normal thing I do is run the code (usually the test that caused me to write the code), stare at the failure, look at my code, smack my head, and add:
using(SqlConnection conn = DataAccess.GetConnection())
{
conn.Open();
using(SqlCommand command = conn.CreateCommand())
{
command.CommandText = foo;
command.ExecuteNonQuery();
}
}
Then run my tests again and watch them pass. Next, I’ll merrily go along the way, not wondering if there is something I could do to keep this from happening once and for all. As Adrian says: “It’s not really not wanting to learn, but not being able to see that an opportunity to learn exists.”
I see this too often in day to day dealings. A lot of developers just want to get something working, get it out the door, go home and be done with it. Not that I’m advocating against that – I like spending as much time as possible at home. But if you make the same mistake over, and over, and over again, what part of our minds doesn’t raise a flag that perhaps we should investigate /why/ we are making the same mistake.
Willie Crawford says in an article, “That next step that we often need to take is simply in the doing. Knowing is not doing. You can understand something as deeply as possible on an intellectual level, but until you do something with that knowledge, you’re no better off than someone who doesn’t understand.”
By I would take it one step back. I believe that often times people just haven’t experienced the same things, and therefore can’t see that mistakes are being made. In my above example, I just assume that I will always make that mistake, and remind myself to check for it. I may not have read an article, or a newsgroup posting, or talked to someone about the issue. And because of that, I don’t know I need to improve.
A good friend and fellow developer told me recently that he doesn’t need tests because his code doesn’t have bugs, and doesn’t need refactoring tools because his designs are dead-on. I admire that spirit, and hope that I am able to learn a lot from that. I, unfortunately, am not infallable. Give me a SqlConnection and I’ll forget to open it every time. But what if he has to interface with my code that surely has bugs in it? And what if all those tests that drove the design of that code help us track down that bug rapidly, and we are able to write a new test or two which seals that hole? Maybe I will get a convert to TDD that may not have otherwise happened if I wasn’t a second-class developer. After all, if your code just works, and you don’t have to modify it, Unit Tests and Refactoring tools are going to be useless to you.
So, I guess I’m glad I make mistakes, not just so I can learn, but so I can introduce a way of walling those bugs to others who might not otherwise have seen it.
Amazing how much you can get out of forgetting to open a connection!
Brian Slesinsky posted a nice lightweight approach on the XP list about running servlets in the same process as JUnit using Jetty:
One lightweight approach that most people don’t seem to know about is using Jetty to run servlets in the same process as JUnit. Jetty has a much nicer internal API than Tomcat. After writing a wrapper class you can have tests like this:
protected void setUp() {
webserver = new WebServer(); // starts on localhost with an
arbitrary unused port
servlet = webserver.addServlet(MyServlet.class);
}protected void tearDown() {
webserver.shutdown();
}You can test using HttpUnit as usual, but it saves Tomcat deploys and you can also access objects in the servlet directly since you’re in the same process.
If your webapp isn’t too complicated, you can also write a main() method to run your webapp without bothering with building a war file.
Weekly Nuggets:
- “Quality in real money terms (and that is the way we keep score in business) is the inverse of the expense of technical support. If technical support is a profit center, then quality in the terms you think about normally are inverted. Therefore, if you wish to have high quality, you should ensure that technical support never becomes a profit center.” (Kelly Anderson)
- “One of the NLP founders had a saying that he used on occasion: ‘there is only one way to fail with this material. Do it exactly the way you think the book says.’” (John Roth)
- Ron Jeffries: “Imaginary customers? I don’t understand. Do imaginary people buy software?” John Maxwell: “No, complex people do.” Ron Jeffries: “That’s real funny, when you think about it …” (http://groups.yahoo.com/group/extremeprogramming/message/110549)
- “The standard warns that it may only be used to represent dates before 1582 ‘by mutual agreement’.” (Joda-time FAQ about ISO8601)
- “I look at it this way: Every day, each programmer is going to program something.
That something should have a beginning, so that they can come to work, check out their code, and get started.
It should have a middle, with a purpose, so that they can program joyously all day.
It should have an end, so that they can check in their code and go home.
At every moment in time, all the code that exists should be tested, and should be working. (If not, then we are only dong half the work.)” (Ron Jeffries)
- “Assume that anything you didn’t like was the funny stuff.” – (Jim Shore)
- “It is interesting to read this discussion in the light of Alastair’s definition of the three levels of knowledge.
Level 1 – Do this one thing that works
Level 2 – Well, there are several options to do the same thing
Level 3 – I understand what this thing is all about and hence can do the right thing without thinking about it.
Level 1 – 40 Hr week
Level 2 – Sustained pace
Level 3 – Energized work
The problem is that Kent is a Level 3 player. Most of the coaches are level 2 striving to 3. Most of the teams are level 1 getting to 2. There’s a problem with communications here…” (Amir Kolsky)
- “I can belch the alphabet, but that doesn’t mean I should.” (J.B Rainsberger)
When you are running an assembly test in NUnit, you may need to access App.Config values. NUnit loads the App.config files by looking for a file with AssemblyName.config.
However, in the default configuration, that .config file has to be in the same directory as the .nunit project file. For example, we had the following setup:
\src\MyProject\MyProject.nunit
\src\MyProject\MyProject\MyProject.sln
\src\MyProject\MyProject\bin\debug
putting the MyProject.config file in either \src\MyProject\MyProject or in \src\MyProject\MyProject\bin\debug doesn’t work – NUnit can’t find it. Putting it in \src\MyProject works like a charm.
Today I was looking for a simple way to apply an XSL stylesheet to an XML document. I didn’t want resolvers, files, or anything like that. I wanted to take an XSL string and an XML string, and apply it to get the output.
I started to play around with the System.Xml stuff, and then remembered that one of the tools I use pretty much every day is a frickin XSL Transformer I wrote! What better place to look then somewhere I had already solved it.
The core of the code is around a Transformer class that can take in either two XmlDocuments or two strings and applies the Xsl to the Xml, returning both the XmlDocument and the Xml string:
1 using System;
2 using System.IO;
3 using System.Xml;
4 using System.Xml.Xsl;
5
6 namespace MobileHWY.DAL.MSSQLCommandMapper
7 {
8 ///
9 /// Helper class to wrap XSL transformations and provide
10 /// utilities for preparing documents
11 ///
12 public class Transformer
13 {
14
15 #region Declarations
16 private XmlDocument xmlDoc;
17 private XmlDocument xslDoc;
18 private XmlDocument transformedDoc;
19 private String transformedString;
20 #endregion
21
22 #region Constructors
23
24 public Transformer()
25 {
26
27 }
28
29 public Transformer(XmlDocument xmlDoc, XmlDocument xslDoc)
30 {
31 this.xmlDoc = xmlDoc;
32 this.xslDoc = xslDoc;
33 }
34
35 public Transformer(String xmlDoc, String xslDoc)
36 {
37 try
38 {
39 this.xmlDoc = ConvertStringToDocument(xmlDoc);
40 }
41 catch(Exception ex)
42 {
43 throw new Applic
44 }
45
46 try
47 {
48 this.xslDoc = ConvertStringToDocument(xslDoc);
49 }
50 catch(Exception ex)
51 {
52 throw new ApplicationException("An error occurred transforming the XSL Document", ex);
53 }
54 }
55
56 #endregion
57
58 #region Properties
59 public XmlDocument XmlDocument
60 {
61 get
62 {
63 return xmlDoc;
64 }
65
66 set
67 {
68 xmlDoc = value;
69 }
70 }
71
72 public XmlDocument XslDocument
73 {
74
75 get
76 {
77 return xslDoc;
78 }
79 set
80 {
81 xslDoc = value;
82 }
83 }
84
85 public XmlDocument TransformedDocument
86 {
87 get
88 {
89 return transformedDoc;
90 }
91 }
92
93 public String TransformedStrin
94 {
95
96 get
97 {
98 return transformedString;
99 }
100 }
101 #endregion
102
103 #region Methods
104 public void Transform()
105 {
106 if(!ValidateInput())
107 {
108 return;
109 }
110
111 //Set up the XSL transformer
112 XslTransform transformer = new XslTransform();
113
114 //Create a memory stream so we can transform the document into memory
115 //instead of to a file and wrap it with an XmlTextWriter.
116 MemoryStream memStream = new MemoryStream();
117 XmlTextWriter memWriter = new XmlTextWriter(memStream, null);
118
119 //Load the XSL Document into the transformer
120 transformer.Load(xslDoc, null, null);
121
122 //Transform the XML document using the loaded XSL and place the
123 //output in the memory stream
124 transformer.Transform(xmlDoc, null, memWriter, null);
125
126 //reset the byte position in the memory stream so we can read
127 //from the beginning
128 memStream.Position = 0;
129
130 //Create a reader to get the document back out of the memory stream
131 XmlTextReader outReader = new XmlTextReader(memStream);
132
133 //Create a writer to place it in. We will also be creating a seperate
134 //XmlTextWriter so we can control the formatting of the output.
135 StringWriter sw = new StringWriter();
136 XmlTextWriter outWriter = new XmlTextWriter(sw);
137
138 //Set up the output formatting instructions
le="color: teal;"> 139 outWriter.Formatting = Formatting.Indented;
140 outWriter.Indentation = 4;
141
142 //actually write our the document
143 outWriter.WriteNode(outReader, true);
144
145 //clean up after ourselves
146 outReader.Close();
147 outWriter.Close();
148
149 //place the results in the appropriate variables
150 transformedString = sw.ToString();
151 transformedDoc = ConvertStringToDocument(transformedString);
152
153 }
154
155 private XmlDocument ConvertStringToDocument(String doc)
156 {
157 XmlDocument mpDoc = new XmlDocument();
158 mpDoc.LoadXml(doc);
159 return mpDoc;
160 }
161
162 private Boolean ValidateInput()
163 {
164 if(xmlDoc == null)
165 {
166 throw new Exception("The XML Document must be loaded before calling Transform");
167 }
168 if(xslDoc == null)
169 {
170 throw new Exception("The XSL Document must be loaded before calling Transform");
171 }
172 return true;
173 }
174 #endregion
175
176 }
177 }
ationException("An error occurred transforming the XML Document", ex);
g
Here’s an example NUnit test showing how to use it:
public void TestTransformXmlDocument()
{
string xsl = ““
+ “
+ “ xmlns:xsl=\”http://www.w3.org/1999/XSL/Transform\”>”
+ ”
+ “ “
+ ” “
+ “”;
string xml = “
string expectedOutXml = ““;
XmlDocument xmlDoc = new Xm
lDocument();
xmlDoc.LoadXml(xml);
XmlDocument xslDoc = new XmlDocument();
xslDoc.LoadXml(xsl);
Transformer transformer = new Transformer(xmlDoc, xslDoc);
transformer.Transform();
Assert.AreEqual(expectedOutXml, transformer.TransformedString);
}
More tasty nuggets from the past week:
The usual advice we pass out is to start out without a data base. This usually runs into the objection that since they’re going to have to have one eventually, they should start out with one. Then it gets into a “we’re older and wiser than you”, “no, you’re decrepit” shoving match. This doesn’t help.
I find it a bit more productive to point out that you’re going to run without a data base while developing: there is no other way to run large numbers of tests in the short window available during the TDD cycle. Then when you get the program to the point where it needs one, you are _still_ going to be running without one during the TDD cycle; the presence or absense of a database becomes a configuration option.” (John Roth – http://groups.yahoo.com/group/extremeprogramming/message/110199)
My experience suggests that since I’m fallible, my dedication to working like a beaver will falter, and worse yet, my brain will fail to detect something to work on in time to avoid the debt. Sometimes, duplication and other bad stuff will be in the code before I notice.
So my //second rule// is: consider all technical debt to be bad, and if it gets in the way, fix it immediately. This means that if some crufty code is involved in a customer-scheduled story, include making the code squeaky clean as part of that story’s estimate. Don’t move beyond the code that needs to be cleaned as part of the story, but fix all the code that’s really involved.
And my //third rule// is: never, ever, schedule work to clean up code that isn’t involved in a story. Is there bad code somewhere in the system, and it’s just ticking you off, even though there are no stories around to let us work on it? Fine. Ignore it, or let’s have a couple of us come in on Saturday, on our own time, to clean it up.” (Ron Jeffries – http://groups.yahoo.com/group/extremeprogramming/message/110298)
Have a great weekend!
Last week, our team made the decision to start using Subversion. Previously we had been using CVS – sort of. Some of our team were actually using Visual Source Safe, and then checking in their code every couple of days to CVS. In addition, we had no real development tree in CVS. Our network admin had set up a Subversion server, but other than that, no one really knew much about Subversion, or, as it turns out, source control systems in general.
I got tasked with getting us up and running, mostly because I’ve been pushing us hard to use CruiseControl.NET so we could do automated builds from source control. It was an interesting road, and I finally got it up and running and wanted to capture some thoughts from the process.
First, you need to get a repository running. Creating a repository and accessing it is fairly straightforward. Grab the latest Subversion package and install it. Then from a command-line run svnadmin create C:\Path\To\Repo. This will create an empty repository at the location you specified. You can make sure it worked by typing snv list svn://machine/Repo. So, for example, if you wanted to put your Repositories in a folder called Data on your C drive, and your wanted to name it Projects, you would run:
svnadmin create C:\Data\Projects
svn list svn://localhost/Projects
At which point you would just see a blank line returned.
Now that you have the repository set up, anyone can access it by doing the svn list command from a remote computer. Of course, they can’t do anything with it, because you have to set up security. Within the directory of the repository you just created, you should see several new folders, including one called “conf”. Go into it and modify the svnserve.conf file to look like:
[general]
anon-access = none
auth-access = write
password-db = password.conf
Now, create a password.conf file in the same directory which should look like:
[users]
user1 = user1pw
user2 = user2pw
Now, user1 can log in to the repository using the password user1pw.
Second, you have to decide how easy you want to make it for people to add new modules and check in and out files. We settled on a VS.NET plugin from PushOk software. I had used their CVS plugin, and it worked well. The Subversion one so far has worked well – I found that at first I had to disable my internet security firewall, but before I had a chance to sniff around and find out why that was blocking it, it began working. There is also TortoiseSVN which adds a context menu to Windows Explorer.
Now, with the ability to create a repository under your belt, and a way to access it, you have to figure out what kind of structure you want your development tree to look like. This was the hardest part for us, because we have three lines of business that have several projects going on underneath them. Add to that the concept of branching, tagging, and a development line, and it was a bit confusing.
We finally decided to create one repository for each line of business. This was a two-fold decision. One, it allows us to eventually limit access to the repository to just those developers working on it. Second, because Subversion uses global revision numbers, we won’t see projects bump up 80 or 90 revision numbers between checkins. An example repository looks like:
/LineOfBusiness
|-trunk
| |-Platforms
| | |-Laptop
| | | |-Jurisdictions
| | | | |-Juri1
| | | | |-Juri2
| | | |-SharedLibraries
| | |-WAP
| | |-Jurisdictions
| | | |-Juri1
| | | |-Juri2
| | |-SharedLibraries
| |-SharedLibraries
|-tags
|-branches
This setup allows us to have a main development tree, while allowing us to tag certain builds as nightlies or as releases. We will be using CruiseControl.NET to automatically checkout, build, and tag code. In addition, if we need to branch off part of the code (say, for a UI refactoring), we can do that without stopping work on the main development tree.
One thing that was incredibly helpful to me was _Version Control with Subversion_ which is available as a book, or a download from http://svnbook.red-bean.com.
Now that we have the initial setup, it will be interesting to see if everything folds out the way we think it will. I’ll keep this blog posted as we tweak things.
John Carter of Tait Electronics in New Zealand recently posted to the XP list his implementation of a Universal Unit Test in Ruby. Because the files section of the XP list is kind of hard to get to, I agreed to host it here on the site. Here’s the original post:
My TDD, DRY, YAGNI, yin/yang, closed loops and the Universal Unit Test post left many people saying, “Huh? Show me the code.”
http://groups.yahoo.com/group/extremeprogramming/message/110090
So to make sure the maximum number of brains explode, I have implemented it in Ruby and uploaded it to the files section of this group.
It is a “Warts and All”, blow by blow example of TDD developing a Universal Unit Test class in Ruby. Note that it includes a unit test for the universal unit test.
Limitations? Well, a couple are uncovered in the process of development and the obvious one that it applies only to transforming code.ie. I can’t, at the moment, think of a sane way of applying it to itself…
But I suspect that may be temporary. :-)
I realized this week that with all the mailing lists and web sites I come across a ton of “nuggets” of goodness each week. I am going to start summarizing some of the good ones on Fridays here. This weeks covers everything from TDD to Haikus and Linux Audio:
And finally, this email from a coworker regarding Linux Audio Recording:
I decided to give Linux one last shot for the audio work, and it paid
off. It is working great, stable, and good clean audio now. Major
lessons learned if you opt to try it….1. Use an external sound card. USB or Firewire. Don’t even try internal unless it is musician grade (e.g. M-Audio Delta 66). I bought a $35 Griffin iMic and it is working great.
2. Check sound card support on the ALSA site. ALSA IS the sound system for Linux now so make sure it is support and READ THE NOTES ON THE CARD.
3. Use Fedora + the Planet CCRMA distribution. It gives you 40+ audio apps + the latest ALSA sound system in one easy distro. If you think you can do better than the sound and music people at Stanford you go for it Spanky!
http://ccrma.stanford.edu/planetccrma/software/installtwosix.html
4. You need to install the linux kernal with real time support. The Planet CCRMA release has one. See item 5.
5. Jack is the audio server on linux. I had everything going but cracks in the audio. Turns out you must run Jack and your digital recorder app as root or sudo root to be able to kick in Jack as realtime capable. After that the audio was clear.
Here is a great tutorial
http://www.djcj.org/LAU/ardour/Basic_recording_howto.html
6. Check your mainboard for a game controller – if it has one, it is probably a MIDI controller too. You can buy cables for that from CompUSA.
I opted for an external USB controller from Maudio. Note that they use a really weird driver model where EVERY TIME YOU ATTACH THE DEVICE IT LOADS THE FIRMWARE TO IT. Go here
http://sourceforge.net/projects/usb-midi-fw/
7. Remember these two items – alsamixer (archaic style UI that controls ALSA levels and sources) and /etc/modprobe.conf (kernel modules = audio drivers unless you love compiling kernels and have nothing better to do than hack C code). If it aint working, 70% of the time it is one of these two items.
My primary weapons of choice now are Ardour for the record and Muse for sequencing. Lots of people use these… I know Rosegarden also is a popular app for sequencing. If you do Planet CCRMA there are literally 50+ apps to play with and you will be busy for some time.
Have a great weekend!
Over the past few weeks there has been a lot of frustration at work. I feel like we have some just great developers, but because we are in a transition to becoming a small company from a startup, there isn’t a lot of structure. This leads to a lot of duplication, and not a lot of sharing of ideas.
After an IM conversation with Brady (who starts with us on the 29th! Booyah BoA!) he said a classic line – “I’ll leave you to your demons”. That got me to realize that it was my demons that I was dealing with, and if I was frustrated with how things were being run, then I needed to pony up to our VP and lay it out.
So yesterday I did just that. I told him I was frustrated by the lack of communication from Senior Management, the stopping of the stand-up meetings, and the lack of communication as a whole. I was frustrated by him throwing more and more resources at a project which is our highest priority, but shouldn’t need that many resources. Most of all I was frustrated with his lack of management of the team, and I told him we needed him to be a manager, not a frickin developer. Oh, and that none of us could stand it when he asked us a question and then walked off halfway through us answering him.
Anyway, after some discussion of the projects and the teams, I convinced him that we have some talented people, who I’m sure would love to be more agile in their approaches, but because they haven’t done it before, they would have a hard time starting it on their own, and needed a push. So he approved for me to jump projects and help bring more agility into it through various XP Practices.
After lunch I started to really evaluate various tools I could use to make things easier. I’m already a big fan of NUnit and NUnitAsp, so those were no brainers. My next focus was on the build process. While I had heard a lot about NAnt and CruiseControl.NET, I had never made the push to actually try them out. Ctrl-Shift-B from VS.NET worked just fine for me. :)
So I downloaded them, and gave them a spin. I followed this great tutorial called CruiseControl.NET From Scratch to get everything up and running. Having never used NAnt or CruiseControl.NET before, I was surprised how quickly I was able to get up and running. One thing was that we use Subversion instead of VSS, but even with that it didn’t take me long to get it working.
Now, it wasn’t without a couple of gotchas. NAnt has a companion project called NAntContrib which provides integration with Source Control and a whole host of other features that haven’t found their way into NAnt. Make Sure NAnt and NAntContrib are the same *exact* version! I was running NAnt 0.83, and NAntContrib 0.84-rc3, and they didn’t work. Moving them both to .84-rc3 and they worked. Only took me 3 and 1/2 hours (grumble, grumble).
But once I got it working, I moved one of my existing projects over to it, with all of the NUnit tests. There were some gotchas with references, and because part of the project is ASP.NET I had to learn about the webmap attribute of the solution task. And then, the console app said it had built, and I pulled up the web interface, and there it was in all it’s glory!
So now on to getting it documented and setup to move on to playing more with ReSharper. Good stuff.
(On a slightly related note – with the hiring of Brady (and Bill just a few weeks ago) we are beginning to realize the bringing back together of the infamous Wachovia .NET team. Now if we can just get Travis, Mark and Daniel here…)