Brendan Enrick

Daily Software Development

Have you tried the Toughest Developer Puzzle Ever?

Toughest Developer Puzzle Ever 2Earlier this week, the second iteration of this developer-targeted online challenge was released. The Toughest Developer Puzzle Ever started last year, and you can do either of the puzzles. I recommend tackling TDPE 2 if you haven’t done either one yet. There will be prizes sent out to the first 100 people to finish the puzzle, so get solving! Very few people have solved the puzzle so far.

TDPE Classic was created mostly by Jeff Blankenburg with assistance from Sarah Dutkiewicz and me. The puzzle ended up being 30 levels, and I it was a challenging set of puzzles. It was great fun working on it, and solving the puzzles was also a blast.

This year, we kicked things up a notch. We made harder puzzles, we made more puzzles, and we got extra help working on them. If you were looking for a challenge, I think we have a good one this year.

This years puzzle making team includes:

And to help you on your way I’ll give you some advice.

  • Don’t believe everything you see in the puzzle.
  • A puzzle may seem tedious even when it isn’t.
  • There may be more than one way to solve a puzzle.
  • Find all available clues, not just the obvious ones.
  • Use your head.

I hope you enjoy the puzzle! Good luck!

Expect Unique Exceptions

One too common programming practice that irks me when I find it in source code is throwing or catching non-specific exceptions. I believe firmly that it is important to fail fast, and exceptions are a great way to ensure fast-failing code. If you drag out a failure you can hide the initial issue or allow a process to continue after an error has happened. Using non-specific exceptions will also hide information. Failing quickly is important as is being able to respond correctly to an exception and track down the cause.

Steve Smith posted recently saying that one should not throw duplicate exceptions. He gave a great example of where people will check for two different cases and throw duplicate exceptions.

public void Foo(SomeClass someArgument)
{
if(someArgument == null)
{
throw new InvalidArgumentException("someArgument");
}
if(!someArgument.IsValid())
{
throw new InvalidArgumentException("someArgument");
}

// Do Real Work
}

Checking for null as well as custom validity is important, and you want them to be separate cases. The suggestion from Steve is that you give them unique messages or use a more specific exception to allow people to tell what caused the error.

public void Foo(SomeClass someArgument)
{
if(someArgument == null)
{
throw new ArgumentNullException("someArgument");
}
if(!someArgument.IsValid())
{
throw new InvalidArgumentException("someArgument");
}

// Do Real Work
}

public void Bar(SomeClass someArgument)
{
if(someArgument.Quantity < 0)
{
throw new InvalidArgumentException("someArgument",
"Quantity cannot be less than 0. Quantity: " + someArgument.Quantity);
}
if(someArgument.Quantity > 100)
{
throw new InvalidArgumentException("someArgument",
"SomeArgument.Quantity cannot exceed 100. Quantity: " + someArgument.Quantity);
}

// Do Real Work
}

I lean toward using unique exceptions for handling this. It allows for better handling of the result and less duplication of the code. For example in the method listed above instead of using an InvalidArgumentException when the Quanitity is negative, I could use a NegativeSomeClassQuantityException. For the second one I might use an ExceededSomeClassQuanityException with the following resulting code.

public void Bar(SomeClass someArgument)
{
if(someArgument.Quantity < 0)
{
throw new NegativeSomeClassQuantityException("someArgument", someArgument.Quantity);
}
if(someArgument.Quantity > 100)
{
throw new ExceededSomeClassQuantityException("someArgument", someArgument.Quantity);
}

// Do Real Work
}

This is great because my code isn’t concerned with how to format these exceptions. My code just passes the required information via constructor parameters and the exception knows how it should be formatted. The calling code can catch each of these exceptions if it has a special way of handling either one or through the use of polymorphism it can handle both of them.

We want to make sure that we inherit from a relevant exception as well, so we could use the InvalidArgumentException.

public class NegativeSomeClassQuantityException : InvalidArgumentException
{
public NegativeSomeClassQuanitytException(string argumentName, int quantity) :
base(argumentName, "Quantity cannot be less than 0. Quantity: " + someArgument.Quantity)
}

public class ExceededSomeClassQuantityException : InvalidArgumentException
{
public ExceededSomeClassQuantityException(string argumentName, int quantity) :
base(argumentName, "SomeArgument.Quantity cannot exceed 100. Quantity: " + someArgument.Quantity)
}

User Group Fluidity

hudsonsc-logo Last year, Steve Smith, Rich Henning, and I started a local user group called HudsonSC. As a software craftsmanship group, we focus our efforts on improving one’s abilities as a developer. I’m not sure I can say enough about the need for developers to practice and learn to increase their abilities. Developers are providing a service in the form of software development. We should be proud of what we achieve through our efforts.

Our last meeting of our Software Craftsmanship group focused on the idea that software development is not the creation of things, but a service which when provided results in a new thing every time. We have this great topic and great meeting of the group thanks to a great idea and presentation from Michael Falanga. I will say that he is almost as good as Kevin Kuebler at keeping his talks short. Michael’s slides are available on SlideShare.

My favorite part of the talk was the comment that, “we’re not building cars. We are creating something new each time.” This I think is a very important point to make, and I agree with it. I don’t make the same thing every time. I am using techniques I’ve learned, and those aren’t the same techniques as anyone else. We all do things a little bit differently, and we’re all creating different implementations of the same kinds of things.

Our group is free-form and guided by our members. Sure, we have an agenda, but we hardly stick to it. It’s a rough outline for when things don’t spontaneously change.

At StirTrek I was discussing the group with Michael, and he was saying that he wanted the group to get back to the roots of software craftsmanship and discuss some of the finer points of the trade. The consensus of those around us at the time was that the programming exercises at the end of the events were great, but we should also focus more on the “why” we’re doing things the way we are.

If you’re user group isn’t discussing what you want, take the reins and guide it. It worked very well in this case as this meeting was one of the best our group has had.

Anyone can bring a talk to discuss in our open spaces or by giving a short talk, and I believe that this makes for a great user group. The occasional topic from a speaker is great, but I get a lot more out of discussing a topic than having someone tell me a topic. (This of course tells you that there was some debate on certain aspects of Michael’s talk.)

Take charge of your developer community and make it a better place for everyone involved. If you’re in the area, you should stop by HudsonSC and let us know what you want to discuss. We’re always looking for your great ideas.

Test Driven Development is Very Approachable

Othello_board Growing up I always enjoyed a good game of Othello. The rules of the game are very simple, but it takes a lot of work to become really good at Othello. There is a great tagline used to describe this game; “A Minute to Learn...A Lifetime to Master”.

The same can be said about Test Driven Development. In fact, when NimblePros hires interns in the summer, we go to recruiting events and set up laptops for people to write some code using TDD. When there are no students in the room, we’re the ones working on the code. Even our seasoned developers need to practice using TDD or we’ll never become better at using it.

The great part about this is that you don’t need any experience with TDD in order to sit down and start. You don’t want to sit down and start using TDD on legacy code, but simpler problems are a great way to get started.

We like doing these exercises as it allows us to not only see who can write code, but also who is willing to take the leap and try new things. This is one of our self-selecting techniques of evaluating people. Would you hire an employee not willing to go out on a limb and try something new? I wouldn’t.

I have two points to make: it’s easy to get started with TDD and if you don’t practice it you’ll never master it.

Sure, I can grab students from a college who may have only heard of unit testing in a class and have never really implemented it, but will those students ever get good at TDD. I think that they will if they work hard and actually use TDD. You have to practice it though. TDD will slow you down at first, which is why you need to practice it.

Agile Planning with Fewer Stakeholder Fights

If you’ve ever had a team responding to the needs of multiple stakeholders I am sure you’ve seen some interesting trends. Some stakeholders will sit quietly and have their requests ignored while others will always push their needs to the front. This of course happens in a lot of different fields. There are always the pushy people who want their stuff done first, because it’s “top priority”. Maybe it is, but that’s not for the team to decide.

So how can you prevent problems from arising from this type of behavior?

I recommend that you spread as much information about things as possible. In the past I’ve found that when dealing with decision makers it is best to make them aware of the other things you’re working on. If your team has an informative workspace that makes available all of the tasks you’re working on they’ll be able to see what is currently being worked on.

Keep your backlog of tasks in priority order and make the whole thing visible. Make sure also that adding something to the front will require physically moving another task back. It’s much harder to ignore the displacement of someone else’s request when that request must be physically moved.

Another great way of handling this is to make sure all of the stakeholders are present when ordering the list of tasks. This will make it quite clear if one person’s requests are all being put at the front of the line, and can help allow for some compromises to occur instead of an argument.

Disagreements over whose need is more important are not the place for the development team to be involved.I’m not saying that a fist fight will break out, but you want to avoid upsetting anyone. Your best situation is when you don’t have to make a decision about whose task to work on. I find that people are more considerate of other people’s needs when they’re made visible, so make sure that stakeholders see the displacement of another’s work.

Let Others Work on Your Code

Don’t be overprotective of your source code. Allow others to make modifications, improvements, refactorings, and even defects (as long as those get fixed immediately). I am talking about two different Extreme Programming practices here: collective code ownership and pair programming. A non-developer friend of mine has been reading about agile software development and asked me last week about pair programming, "do you guys do this 'pair programming' thing?"

I think even among developers the pair programming concept is one of the strangest aspects of extreme programming.

Yes, a lot of agile development teams use pair programming. We even equip our facilities to make pairing easier. Our computers have 2 large monitors, 2 keyboards, and 2 mice. Some teams pass the keyboard and mouse back and forth, and that works just as well.

What is Collective Code Ownership?

This is a great term that basically says that every member of a team should be able to work on any part of the system. If you’re usually working on the user experience you can still spend some time working with your application’s service layer, and if you’re the database guy you can work on some business logic.

Wouldn’t someone be able to do that job better? Yes, there probably is someone who knows each part of the application better than anyone else. You can work with that person if you want, and the two of you will be able to create some great code. What if that person who is better at that task is on vacation? Wouldn’t it be great if someone else knew the code.

Keeping the team familiar with the project means that the team can focus on certain parts of the system at a time and achieve great results. You don’t have to split your team based on layers or modules of your application. Let them intermingle. Fresh ideas and eyes can greatly improve things. They might come along and fix a defect or two for you later; saving you a great deal of work.

Why Pair Program?

Pair Programming The mind is the important part of programming, so why not have two? After all, programming is certainly not just typing. If you believed that how could you ever support pair programming?

If you pair program for every line of code in your application then you know that you’ve had at least two sets of eyes look at every line of code. This is a good deal of added security.

When you create some new piece of code that the whole team will need to work with, pair programming can be the solution to disseminating this information. You and the guy you created it with can split up and pair with another person each, and while working on the code you will each be showing your new partner how the new code should be used. This allows new information to spread quickly through the team.

Even the simplest things as learning keyboard shortcuts are improved by pair programming. When I use a cool keyboard shortcut, I can tell my partner what it is without breaking my train of thought. We can keep on typing, and when he has the keyboard in a minute or two, he can try it.

Pair Programming Computer Just like athletes, a programmer needs someone else there to help him keep the pace. You partner can help you avoid distractions, and most importantly he can keep you from taking shortcuts. When we write code we might say, “I can fix this later”. The partner will make a note of it and be sure you get it fixed later. If it can’t wait and you’re in the middle of something you can always pass it to another pair. Your partner can do that while you keep on coding. Remember that the other pair can work on any code because of the collective code ownership.

I am sure I missed plenty of reasons to pair program, but this list feels mostly adequate to get the point across.

Please let me know why you do or do not pair program. Is it work related? Are you an unbeliever?

Show Your Customer Unfinished Products

In what I would call the worst-case scenario,a customer’s involvement in a project could consist of setting up initial requirements at the start of a project and then viewing the results at the end of the project. This is of course not ideal for either the customer or the team developing the software.

In the next best scenario the customer could show up at the beginning and end of each release. This is still not enough. Why? At that point there is no slack remaining. What if the customer doesn’t like how something was done? What if there was a misunderstanding? What if the customer has something new that your design makes as a new requirement?

If the customer doesn’t like your design and you need to go and redo things, your time and the customer’s money has been wasted. I know it can be challenging with some customers, but you want to show them what you’re working on before you’re done. The customer can make changes based on what you show them.

When you mock up a design for something, show your customer. When you first set up the screen, show it to them before you’re done.

The customer will like that you’re keeping them involved. They can make changes now rather than at the last minute, which is good for you. You get the benefit of having questions answered. You can remove guesswork. If you got it wrong, don’t worry, the important thing is that it didn’t release that way.

Disconnects between the customer and the development team can happen easily. The customer doesn’t know why things take so long or even what you’re working on if you don’t tell them. You may feel like the customer is sabotaging your success if you wait until the end to show them what you’ve created. Get your constructive feedback earlier and you and the customer will be happier about the results.

This applies to agile teams as well. If you’re an agile team you may think that your weekly iteration gives your customer enough involvement since you can show them at the end of an iteration. It isn’t enough. You need your customer’s feedback throughout the iteration. In order for agile development to work you need to make sure that you are done  with your work at the end of every iteration. You can’t be done until your customer signs off on it, so make sure you’re getting them to sign off on things during the iteration. If you don’t you’ll never finish an iteration on time.

Keep Databases in Source Control

Well, sort of. Your database can be rather large, so you probably don’t want to actually have the database file in source control. I do think you should have your database in source control, but really just your schema. Remember in my post about keeping your binaries in source control I mentioned that it is important to be able to go back to any point in time for your application. This means the database will need to match as well. How can we allow ourselves to do that? Keep databases in source control.

We want to be storing the subset of SQL usually defined as the Data Definition Language. These are all of the statements that create tables, define columns, create indexes, and make any alterations to our database schema. There are plenty of ways to store these. In fact I recommend after reading this post that you attend a local user group and offer this topic as a discussion for an open space. We’ve done this topic at HudsonSC and you get to find out a lot about what solutions other teams are using and why they’re using them.

Why store the database in source control?

We get a few advantages here. This is not an exhaustive list, but it does include some good reasons for following this practice.

  • You can set up your project on any computer and have a local database to test against.
  • Always be able to rebuild your database if anything happens to it.
  • Go back to any point in time and be able to create the database.
  • Incrementally change things and be able to see what changes have been made at any point.

How can I store my database in source control?

There are plenty of ways in which you can store your database in source control. I’ve used a few different techniques for doing this. There are some tools which can help you do this. Keep in mind that you can build your own as well. They’re not very difficult to create.

I have used Visual Studio Database projects before. These are nice because they’re a full package for handling the changes. This is the heavyweight approach to managing your database changes. This keeps each table, index, key, view, stored procedure, etc. as its own file. This built-in Visual Studio system allows you to update the project and use the project to update database.

The lightweight approach is to make each change using a sql script. You can write the alter, create, and drop statements by hand and store each one in a file. This approach makes it harder to see how each table has changed over time, but gives a better view of what changes were made at what points in time. In order to deploy the database updates you’ll either need to write your own system or use a tool like Tarantino to keep track of the changes.

There are of course other tools for handling this situation. What tools are you using for managing your database?

Other considerations

You will probably also want to have some code which can fill your test database with data to be used for locally running your application. I recommend having this code be compiled code (not SQL scripts). The reason for this is so that you can have the compiler assist you in ensuring that the scripts aren’t out of date. You also want this to be part of your build process, so that no one can forget to update it. It should be run by your continuous integration server, so that the build will break if there are any issues.

This will also make updating your production database a lot easier since you will have stored scripts which define exactly which operations need to be performed in order to update your database according to the changes you’ve made.

Sql Table Refactoring Solution

Steve Smith posted an interesting SQL Table Refactoring Challenge on his blog. I am prepared to go out on a limb and share my solution for how to make a table more efficient.

The first step I would take is to change the columns of the table a little bit.

  • Change CountryCode to a char(2) since they are all 2-character ISO standards. The varchar does limit itself based on the size of the data, but has an overhead of 2 bytes. This will also prevent anything longer from being entered into the table.
  • If you’re feeling very ambitious you can make a CountryId column that is of type smallint and uses the id of the country instead of the 2-characters. This will require an update to all of the existing data so I would say this isn’t worth the effort.
  • AboveFold can be changed to a tinyint. Note: these values are 0 to 255 so we will need to remap the values in the database.
  • A smallint can be used for the Clicks column, since it doesn’t need to get very large.
  • Period can change to a date. This will take up 3 bytes which is much smaller than the 8 for the datetime.

As an extra note I believe that int is required to get up to 50,000 for the tables which require it. If we could drop the limit to around 30000 we could use a smallint for those.

Once I have that table created I create 2 duplicate tables (duplicated schema not all of the data). [lq_ActivityLogLoad] and [lq_ActivityLogLoadNext] At any given time we will write to one of these two tables and not the huge table.

Then I create a job which will switch to which of these two tables we are writing. After switching the job will load all of this data directly into the large table using an Upsert (Update or Insert). After loading the data from the table it can empty out that load table and wait a minute and perform this action over again. Doing this loading pattern will make it so the writes will be on these small tables which aren’t being read from.

I would change the primary key to be based on the ID and put a unique non-clustered index on the columns currently used for the primary key preserving the safety from the unique constraint.

Here is the table I created for this.

CREATE TABLE [dbo].[lq_ActivityLog](
[ID] [bigint] IDENTITY(1,1) NOT NULL,
[PlacementID] [int] NOT NULL,
[CreativeID] [int] NOT NULL,
[PublisherID] [int] NOT NULL,
[CountryCode] [char](2) NOT NULL,
[RequestedZoneID] [int] NOT NULL,
[AboveFold] [tinyint] NOT NULL,
[Period] [date] NOT NULL,
[Clicks] [smallint] NOT NULL,
[Impressions] [int] NOT NULL,
CONSTRAINT [PK_lq_ActivityLog] PRIMARY KEY CLUSTERED
(
[ID] ASC
)
 

Keep Binaries in Source Control

No, not all of them. Just the ones required for your build process. Doing this will make your life a lot easier. You want to reference all of your assemblies locally and keep them in source control. You also want all of your testing executables kept there. If you have anything extra your build process depends on make sure you include that as well. A great example is a post I did about building a library which uses OpenAccess on a build server. The build server of course doesn’t have OpenAccess installed. That would be crazy.

You get a lot of great benefits out of keeping these files local and in source control. Since libraries change infrequently, they will not require much source control storage space. This also means that when updating from source control you will rarely have to wait for an update. If you do have to wait you probably needed it anyway, and would have just been waiting on an Internet download instead.

Build Server Benefits

If you have a build server this will allow you to build your application without having to install your tools and libraries on the server. When the server gets the latest version of your project from your source control server, it will already have the current versions of all of your tools and libraries. This keeps things simple and easy to work with.

If you don’t go this route you will have to install everything on the build server and make sure that all of the paths still match. Ever dealt with the Program Files (x86) folder?

New Computer Benefits

Have a new developer joining the project or just a fresh install of Windows on your computer? Great! Now just go and find the 10 different libraries and tools that are required to build this project. Once you have those make sure that they’re in the correct folders… Wouldn’t that be annoying?

If you keep those tools in source control and always reference with relative paths, no one will have to hunt for all of the tools required for the build. If you were referencing libraries in their install locations you would have all kinds of problems if something installed in a bad place. For example I’ve seen libraries try to install in a user-specific directory. Yes, that is a worst case example, but it can happen.

Keep everything relative and you can move your code anywhere. I want to be able to copy everything in source control onto a freshly installed copy of Windows and have it build. If I can do that, I know the project has a good structure.

Old Versions of the Project

Ever want to go back to an old version of your project and look at something? I bet you had some trouble building if you’ve changed the versions of any libraries you’re using. If the specific version you need is with that code you never have to worry. At any snapshot in time the repository has the correct build scripts, libraries, tools, utilities, and source code. You’re all set to go back to any point in the history of the application and take a look.

You might be saying, “what if I have changed versions of my IDE?” It shouldn’t matter. If you’ve got build scripts in place you can use those to get that app running.

Some people try to keep track of which versions of which tools are used for which release of an application, but if you’ve got them in source control you don’t have to worry about it. Just get the version you want, and the required files come with it.

Now that’s what I call a clean, self-contained build.