Programmer Musings

Don't Lose Sight of the Basics

I've recently been reminded of something I used to tell entry level programmers repeatedly: Review the basics.

And really I mean all of the basics:

• data structures
• basic algorithms
• basics of algorithm analysis
• language syntax
• core CS concepts
• testing strategies

In many programming positions, you can get away with coasting on these things for a long while. But, as you let your grasp of the basics deteriorate, you lose flexibility. You begin to use similar solutions to different problems you encounter. If you don't reverse the trend, you can become that guy that applies the same library/data structure/programming paradigm to every problem, no matter how badly it fits.

Some would ask why you should be familiar with lots of basic tools, when you can do your job with just a few. After all, if your employer is happy with your solution, who cares if it is the best? This is ultimately quite short-sighted. My argument has two parts.

The Practical Argument

From a practical standpoint, being familiar with more data structures and algorithms means that you are a more valuable employee. Even if your employer does not know what these tools are, who is more likely to get the raise: the guy whose caching algorithm fell over as the number of things to cache increased by a factor of 10 (because it was based on a linear array search) or the gal whose algorithm handled the change because it was based on a tree or a hash? Do you think the person who reduced the memory footprint on a critical server by changing data structures is going to be the first to lose his job?

Statistically, most of us will change jobs at some point. Do you think you will impress a potential employer with your conviction that no problem should need any structure other than a linked list? How about if you know everything there ever was to know about hash tables, but you can't sort an array? How about if your solution to every problem involves a SQL query?

The Craft Argument

Many of us became programmers from a love of solving problems and learning. At that point, everything was new and exciting to learn. Eventually, programming can become just a job. You fix bugs, write some new code, collect a pay check and go home. But, that does not have to be the end of it. We are all craftsmen to some extent.

To keep up with any craft, you need to maintain your tools. In programming, those tools are:

• your editor,
• your language,
• libraries, and
• your brain.

Part of maintaining your programmer brain is challenging it and honing your skills. When programming was new to you, you probably spent time implementing really simple programs to see how they worked. This exercise added basic tools to your mental toolbox. If you don't use those tools, they get rusty and can fail when you need them.

In times past, when a craftsman worked with his hands, the practice of maintaining and upgrading his tools was critical to being able to do the craft. You can bet that a woodworker kept all of his chisels sharp, clean, and well protected; including the ones that he would only use once in a great while. When he needed that tool, out it comes to solve the problem it was meant for.

We need to do the same with our mental tools.

Conclusion

Over the next few posts, I'm going to review a subset of the basics. This is partly an exercise to force me to clean my mental tools and partly to serve as an springboard for you to have an idea about concepts you might want to brush up on. I don't expect to cover everything you should know, but these posts should serve as a good start.

Programmer Beliefs

The practice of developing software depends on a large body of arcane knowledge and skills. Almost all of this knowledge is based on logical principles and built up to the point that we can do amazing work. Because programming has it's roots in applied mathematics, there is an impression that programming is also a logical, rational process.

The Rational Ones

Many stereotypical programmer types seem to consider themselves to be rational people. In fact, we seem to believe that we are more rational than the average person. Our work requires a level of detail that most people can't deal with (or so we believe). Computers just do logical manipulation of bits, and our programs are (fundamentally) just logical and arithmetic manipulation of bits and numbers. Therefore, our work is definitely logical and rational.

There's just one problem with this view. Some of what we do is pure personal preference. That is not surprising, since we are still human. For that reason, we still make many decisions based on what we like or what we are used to, rather than for purely rational reasons.

Rationalizing the Preferences

However, we tend to believe that (at least in software development) we are making decisions for logical, rational reasons, not because of mere preferences. That sounds too much like choosing something because it's fashionable. Or worse, picking something for no reason. As rational software developers, we would never do that (assume appropriate amount of sarcasm in that statement).

If you want to see programmers completely lose their cool, start a discussion about the best editor, programming language, or indenting or brace-placement style. There's a reason why people refer to these as religious wars. We've all seen programmers get into major arguments over these issues, and they all have careful and sometimes very detailed arguments about why their choice is the right one. More importantly, since each of them believes they have made the choice for rational, objective reasons, the logical conclusion is that anyone who doesn't agree must be wrong. And the rational thing to do is convince the other person of the flaws in their belief and force them to bow to the better objective argument.

The Subjective Preferences

There's just one problem. These choices really are subjective. Each of us made the choices we did for very irrational reasons:

• It was the first style or language we learned.
• Someone we admired used that editor.
• The idiosyncrasies of that tool fits our brain better than the alternatives.
• We want to fit in with the community that uses that tool.

This point was driven home to me years ago, when another programmer and I were discussing brace placement in C. We were trying to develop a standard for our group and the two of us came down solidly in opposite camps. As we were arguing, I suddenly realized that he did not see the code the same way I do. I don't mean this in any deep philosophical sense. I mean it literally. The placement of the braces affects the way I pattern match while scanning code. But, when he saw code, the braces were less important than the indenting.

I actually stopped arguing for a while when I realized that. I finally was able to explain what I saw when looking at the code and we ended up deciding to use the style that was most common in the code, rather than changing to either of our positions.

Don't Assume

When one of these issues comes up, remember that they really are matters of preference and convention. No one approach is provably, rationally, better than the others. It's more important to realize that the other person is making the choice for reasons that make sense to them, and you won't be able to convince them otherwise. The important point is for us to compromise enough to be able to work together, and solve real problems.

I am not a Mac Person

Despite being convinced that we are rational people, many programmers or software types can be rabid about our favorite technologies. If you want to hear a passionate opinion, ask about favorite programming editor, programming language, or operating system. If you want to start a fight, suggest that the preference is not an objective decision. That instead it is just a preference.

After you get out of the hospital, you might wonder why people who pride themselves on rational thinking are so passionately irrational about these things. Over time, I've finally come to understand that different people's minds work differently. (Yeah, we should know that. After all, everyone else isn't rational like us.) Many people wiser than I have told me this over the years. About a decade ago, I finally began to realize they were right.

I am a vim user, it is comfortable and does what I need. Several of my co-workers use emacs. That's fine. I understand that it makes sense to them, even though it seems to fight me at every step the few times I've used it.

Over the last few years, I've been using a Mac for work. This is the first time in my career I've ever used one. I know people (some of whom I really respect) who rave about the brilliance of the Mac interface. Who absolutely adore the way the hardware is put together. Any time I've commented about something I don't like, I've been told it's that I "just don't understand the reasons why this is obviously the right way to do things."

The truth is, I have read about why some of the interface issues were chosen. I've heard the arguments on why I'm wrong. But, truth be told, they don't convince me. Amusingly enough, it's not the big things that give me grief, it's the little stuff.

Minor Nits

There are a few things that a really minor and might even be reconfigurable. They are minor annoyances, that I trip over just often enough to annoy, but not so often that I am willing to take time to find a fix.

• Cmd-C, Cmd-X, etc. instead of Ctrl-* (granted this would be less of a problem if I didn't keep returning to Linux at home.)
• Mouse seems to be required for many operations
• Only able to resize windows from bottom/right. (I'm used to resizing and moving windows into whichever direction I need.)
• No "select with mouse and right click to paste" feature that is available in both Windows and Linux.
• In Finder, select several items and push the delete key. Nothing happens. Instead, you need to navigate to the "move to trash" context menu item.

Actual Annoyances

The next set of issues actually cause me some real trouble. BTW, there's no need to tell me that I just don't understand the superiority of the Mac way of doing things. The arguments about superior design really don't trump the loss of my productivity every time I trip over one.

• Cmd-tab to an application with multiple open windows and all of its windows move to the top
• Closing windows doesn't close the application
• If I'm in a window in one application, open a window in another application, close the second window. (I expect to go back to the first window in the first application. Instead, I seem to be sent either to another window in the second application, or lose focus on any application.)
• Menu bar on the top of the screen, not top of window
• Attempt to create new window on a different desktop puts it on desktop where the others are.

Macbook Annoyances

In addition to the desktop machine, I also work with a Macbook. In addition to the other issues, there are a couple of things that only seem to happen on the Macbook.

• The function keys default to not being function keys, but special hardware functionality instead. (I was recently shown where to reconfigure this.)
• When waking up the Macbook with the screen locked, I begin typing my password and the cursor is often moved out of the login dialog. If the last program I was in was accepting input, that's where my password ends up.

No One, Right Way

Despite how it may sound, I'm not claiming that the Mac interface is horrible or badly designed. On the contrary, I see a lot of polish and good design in the interface. These problems are just mismatches between the world of the Mac and the way my brain works. That mismatch causes a definite loss of productivity. And, that explains why I'll probably never be a Mac person.

The Nature of Exceptions

While I was at YAPC::NA this year, I got a chance to catch up with a few old friends and talk with some really knowledgeable people from other parts of the Perl community. One such discussion started when Rob Kinyon told me that "exceptions are fundamentally flawed by design and should never be used." (I'm paraphrasing here. I didn't take notes on his exact words.)

I've used exceptions in multiple languages in the past decade. I've seen both good and bad usages of exceptions. I've also heard many of the arguments against exceptions over the years. (I actually touched on these years ago in the post Joel on Exceptions.) I asked if he could explain, mostly so I could hear which arguments he would use. I really expected the same old arguments. I was quite pleasantly surprised when Rob made some really interesting points in a direction I had not heard before.

If I understood him correctly, there were two major points to his argument.

• Exceptions thrown by functions down the call stack and caught later introduces coupling between widely separated pieces of code.
• The exception object is not part of the parameters or return of any of the functions, so it acts somewhat like a global variable, which is bad.

These arguments were not like the normal complaints I've heard about exceptions. Rob's arguments were actually fairly well-thought-out and well-argued. During the discussion, we ended up attracting Piers Cawley into the discussion. The discussion turned pretty lively after that.

Coupling

I don't think I presented my thoughts very well in that discussion. So, here goes. Let's start with the coupling argument. The simple example of Rob's argument looks something like this (in pseudo-code).

   sub foo {

       ...

       bar();

       ...

   }

   sub bar {

       ...

       baz();

       ...   

   }

   sub baz {

       ...

       if( bad_stuff ) throw bad( "Shouldn't do this." );

       ...   

   }

Rob suggested that foo() is now coupled to baz() because it needs to be aware that a bad exception may be thrown. This would obviously be bad, because baz() is an implementation detail of bar() and foo() should only know about bar().

In fact, foo() is not coupled to baz() at all. Calling baz() has just expanded the interface of bar() to include the ability to throw a bad exception. In fact, that is no different than if the implementation of bar() were changed to throw a bad exception. This is really no different than having to change the calling signature of bar() to add a new parameter needed by baz(). The foo() function does not need to know what bar() does with the parameter (or exception), just that it is part of bar()'s interface.

What this argument does point out, however, is that the exceptions that may be thrown by a function are a hidden portion of the interface. Now, I'm sure some Java-fans will be getting smug about Java's checked exception system. Using this facility, the programmer must specify all of the exceptions that may be thrown by a method. Unfortunately, it's been my experience that system breaks down in maintenance as changes in lower-level code necessitate touching arbitrary amounts of calling code to fix up the throws clauses to deal with new exceptions.

It seems that many programmers eventually either change to unchecked exceptions or declare that all methods can throw an Exception or Throwable, so that anything is allowed. Better design skills or an architecture that converts exceptions to more generic forms as they move between layers can mitigate this issue. But, it does point out an implementation issue with exceptions. Exceptions seem to either result in a hidden interface or a lot of extra maintenance. Perhaps later implementations or better architectures will remove this issue.

Exceptions as Globals

The other argument that Rob made was that exceptions are basically global objects. Like globals, when you access the exception object you have little idea where in the code it was generated. Possibly the biggest problem with a real global is that it may be changed by literally any piece of code in the system. The only way to find out where the global is modified is to examine every piece of code in the system.

Rob's argument was that it is possible that you would need to examine a large amount of code to determine the context for the thrown exception, since the exception might have been thrown from a point far from where we catch it. Unlike a real global, exceptions are only created in code called by the point where we catch the exception. So, in theory, this limits the amount of code that would need to be examined. In some cases, however, this limited amount of code could still be a large section of the codebase.

Java exceptions reduce the effects of this issue to some extent by providing a stack dump in the exception. Depending on the system, that might be almost as bad as searching the code. Other systems support a method of chaining exceptions. In these kinds of systems, you catch an exception when you have more context to add and throw a new exception containing the old exception and further context information. Done correctly, this can reduce the noise of a stack dump type exception and increase the real, useful context available to deal with an exception.

The Meaning of an Exception

One of Rob's arguments was exceptions are not actually necessary. He argued that when something goes wrong in code, there are basically two possible ways of handling the problem.

1. Automatically recover the error condition near the point of the problem.
2. Code gives up and hands control to a human to handle it.

Piers made the very useful observation that exceptions give us another recovery method in between those. This allows the code to basically say I've got a problem I can't handle, someone handle this for me. If no code handles the exception, it basically degenerates to the second case. But, if higher-level code has more context and can actually handle an issue that the low-level code can not handle, an exception allows the code to turn the second case into the first.

So an exception can actually be looked as a a call for help from the lower-level code for someone else to deal with a problem.

Conclusion

To me, at least, Rob's arguments were interesting, but not compelling reasons to avoid exceptions. They also weren't strong enough to justify the claim that th design of exceptions are fundamentally and fatally flawed. I would also agree that certain implementations are less than ideal and that the best exception implementations only exist in the future.