And really I mean all of the basics:
In many programming positions, you can get away with coasting on these things for a long while. But, as you let your grasp of the basics deteriorate, you lose flexibility. You begin to use similar solutions to different problems you encounter. If you don't reverse the trend, you can become that guy that applies the same library/data structure/programming paradigm to every problem, no matter how badly it fits.
Some would ask why you should be familiar with lots of basic tools, when you can do your job with just a few. After all, if your employer is happy with your solution, who cares if it is the best? This is ultimately quite short-sighted. My argument has two parts.
From a practical standpoint, being familiar with more data structures and algorithms means that you are a more valuable employee. Even if your employer does not know what these tools are, who is more likely to get the raise: the guy whose caching algorithm fell over as the number of things to cache increased by a factor of 10 (because it was based on a linear array search) or the gal whose algorithm handled the change because it was based on a tree or a hash? Do you think the person who reduced the memory footprint on a critical server by changing data structures is going to be the first to lose his job?
Statistically, most of us will change jobs at some point. Do you think you will impress a potential employer with your conviction that no problem should need any structure other than a linked list? How about if you know everything there ever was to know about hash tables, but you can't sort an array? How about if your solution to every problem involves a SQL query?
Many of us became programmers from a love of solving problems and learning. At that point, everything was new and exciting to learn. Eventually, programming can become just a job. You fix bugs, write some new code, collect a pay check and go home. But, that does not have to be the end of it. We are all craftsmen to some extent.
To keep up with any craft, you need to maintain your tools. In programming, those tools are:
Part of maintaining your programmer brain is challenging it and honing your skills. When programming was new to you, you probably spent time implementing really simple programs to see how they worked. This exercise added basic tools to your mental toolbox. If you don't use those tools, they get rusty and can fail when you need them.
In times past, when a craftsman worked with his hands, the practice of maintaining and upgrading his tools was critical to being able to do the craft. You can bet that a woodworker kept all of his chisels sharp, clean, and well protected; including the ones that he would only use once in a great while. When he needed that tool, out it comes to solve the problem it was meant for.
We need to do the same with our mental tools.
Over the next few posts, I'm going to review a subset of the basics. This is partly an exercise to force me to clean my mental tools and partly to serve as an springboard for you to have an idea about concepts you might want to brush up on. I don't expect to cover everything you should know, but these posts should serve as a good start.
]]>Many stereotypical programmer types seem to consider themselves to be rational people. In fact, we seem to believe that we are more rational than the average person. Our work requires a level of detail that most people can't deal with (or so we believe). Computers just do logical manipulation of bits, and our programs are (fundamentally) just logical and arithmetic manipulation of bits and numbers. Therefore, our work is definitely logical and rational.
There's just one problem with this view. Some of what we do is pure personal preference. That is not surprising, since we are still human. For that reason, we still make many decisions based on what we like or what we are used to, rather than for purely rational reasons.
However, we tend to believe that (at least in software development) we are making decisions for logical, rational reasons, not because of mere preferences. That sounds too much like choosing something because it's fashionable. Or worse, picking something for no reason. As rational software developers, we would never do that (assume appropriate amount of sarcasm in that statement).
If you want to see programmers completely lose their cool, start a discussion about the best editor, programming language, or indenting or brace-placement style. There's a reason why people refer to these as religious wars. We've all seen programmers get into major arguments over these issues, and they all have careful and sometimes very detailed arguments about why their choice is the right one. More importantly, since each of them believes they have made the choice for rational, objective reasons, the logical conclusion is that anyone who doesn't agree must be wrong. And the rational thing to do is convince the other person of the flaws in their belief and force them to bow to the better objective argument.
There's just one problem. These choices really are subjective. Each of us made the choices we did for very irrational reasons:
This point was driven home to me years ago, when another programmer and I were discussing brace placement in C. We were trying to develop a standard for our group and the two of us came down solidly in opposite camps. As we were arguing, I suddenly realized that he did not see the code the same way I do. I don't mean this in any deep philosophical sense. I mean it literally. The placement of the braces affects the way I pattern match while scanning code. But, when he saw code, the braces were less important than the indenting.
I actually stopped arguing for a while when I realized that. I finally was able to explain what I saw when looking at the code and we ended up deciding to use the style that was most common in the code, rather than changing to either of our positions.
When one of these issues comes up, remember that they really are matters of preference and convention. No one approach is provably, rationally, better than the others. It's more important to realize that the other person is making the choice for reasons that make sense to them, and you won't be able to convince them otherwise. The important point is for us to compromise enough to be able to work together, and solve real problems.
]]>After you get out of the hospital, you might wonder why people who pride themselves on rational thinking are so passionately irrational about these things. Over time, I've finally come to understand that different people's minds work differently. (Yeah, we should know that. After all, everyone else isn't rational like us.) Many people wiser than I have told me this over the years. About a decade ago, I finally began to realize they were right.
I am a vim user, it is comfortable and does what I need. Several of my co-workers use emacs. That's fine. I understand that it makes sense to them, even though it seems to fight me at every step the few times I've used it.
Over the last few years, I've been using a Mac for work. This is the first time in my career I've ever used one. I know people (some of whom I really respect) who rave about the brilliance of the Mac interface. Who absolutely adore the way the hardware is put together. Any time I've commented about something I don't like, I've been told it's that I "just don't understand the reasons why this is obviously the right way to do things."
The truth is, I have read about why some of the interface issues were chosen. I've heard the arguments on why I'm wrong. But, truth be told, they don't convince me. Amusingly enough, it's not the big things that give me grief, it's the little stuff.
There are a few things that a really minor and might even be reconfigurable. They are minor annoyances, that I trip over just often enough to annoy, but not so often that I am willing to take time to find a fix.
The next set of issues actually cause me some real trouble. BTW, there's no need to tell me that I just don't understand the superiority of the Mac way of doing things. The arguments about superior design really don't trump the loss of my productivity every time I trip over one.
In addition to the desktop machine, I also work with a Macbook. In addition to the other issues, there are a couple of things that only seem to happen on the Macbook.
Despite how it may sound, I'm not claiming that the Mac interface is horrible or badly designed. On the contrary, I see a lot of polish and good design in the interface. These problems are just mismatches between the world of the Mac and the way my brain works. That mismatch causes a definite loss of productivity. And, that explains why I'll probably never be a Mac person.
]]>I've used exceptions in multiple languages in the past decade. I've seen both good and bad usages of exceptions. I've also heard many of the arguments against exceptions over the years. (I actually touched on these years ago in the post Joel on Exceptions.) I asked if he could explain, mostly so I could hear which arguments he would use. I really expected the same old arguments. I was quite pleasantly surprised when Rob made some really interesting points in a direction I had not heard before.
If I understood him correctly, there were two major points to his argument.
These arguments were not like the normal complaints I've heard about exceptions. Rob's arguments were actually fairly well-thought-out and well-argued. During the discussion, we ended up attracting Piers Cawley into the discussion. The discussion turned pretty lively after that.
I don't think I presented my thoughts very well in that discussion. So, here goes. Let's start with the coupling argument. The simple example of Rob's argument looks something like this (in pseudo-code).
sub foo {
...
bar();
...
}
sub bar {
...
baz();
...
}
sub baz {
...
if( bad_stuff ) throw bad( "Shouldn't do this." );
...
}
Rob suggested that foo() is now coupled to baz() because it needs to be aware that a bad exception may be thrown. This would obviously be bad, because baz() is an implementation detail of bar() and foo() should only know about bar().
In fact, foo() is not coupled to baz() at all. Calling baz() has just expanded the interface of bar() to include the ability to throw a bad exception. In fact, that is no different than if the implementation of bar() were changed to throw a bad exception. This is really no different than having to change the calling signature of bar() to add a new parameter needed by baz(). The foo() function does not need to know what bar() does with the parameter (or exception), just that it is part of bar()'s interface.
What this argument does point out, however, is that the exceptions that may be thrown by a function are a hidden portion of the interface. Now, I'm sure some Java-fans will be getting smug about Java's checked exception system. Using this facility, the programmer must specify all of the exceptions that may be thrown by a method. Unfortunately, it's been my experience that system breaks down in maintenance as changes in lower-level code necessitate touching arbitrary amounts of calling code to fix up the throws clauses to deal with new exceptions.
It seems that many programmers eventually either change to unchecked exceptions or declare that all methods can throw an Exception or Throwable, so that anything is allowed. Better design skills or an architecture that converts exceptions to more generic forms as they move between layers can mitigate this issue. But, it does point out an implementation issue with exceptions. Exceptions seem to either result in a hidden interface or a lot of extra maintenance. Perhaps later implementations or better architectures will remove this issue.
The other argument that Rob made was that exceptions are basically global objects. Like globals, when you access the exception object you have little idea where in the code it was generated. Possibly the biggest problem with a real global is that it may be changed by literally any piece of code in the system. The only way to find out where the global is modified is to examine every piece of code in the system.
Rob's argument was that it is possible that you would need to examine a large amount of code to determine the context for the thrown exception, since the exception might have been thrown from a point far from where we catch it. Unlike a real global, exceptions are only created in code called by the point where we catch the exception. So, in theory, this limits the amount of code that would need to be examined. In some cases, however, this limited amount of code could still be a large section of the codebase.
Java exceptions reduce the effects of this issue to some extent by providing a stack dump in the exception. Depending on the system, that might be almost as bad as searching the code. Other systems support a method of chaining exceptions. In these kinds of systems, you catch an exception when you have more context to add and throw a new exception containing the old exception and further context information. Done correctly, this can reduce the noise of a stack dump type exception and increase the real, useful context available to deal with an exception.
One of Rob's arguments was exceptions are not actually necessary. He argued that when something goes wrong in code, there are basically two possible ways of handling the problem.
Piers made the very useful observation that exceptions give us another recovery method in between those. This allows the code to basically say I've got a problem I can't handle, someone handle this for me. If no code handles the exception, it basically degenerates to the second case. But, if higher-level code has more context and can actually handle an issue that the low-level code can not handle, an exception allows the code to turn the second case into the first.
So an exception can actually be looked as a a call for help from the lower-level code for someone else to deal with a problem.
To me, at least, Rob's arguments were interesting, but not compelling reasons to avoid exceptions. They also weren't strong enough to justify the claim that th design of exceptions are fundamentally and fatally flawed. I would also agree that certain implementations are less than ideal and that the best exception implementations only exist in the future.
]]>Fast forward a little over a year and he is assigned a project in school to do a report on a book. From what he says, the teacher has suggested that the kids do PowerPoint presentations for their reports. (Fifth-graders doing PowerPoint, the world is changing...) My son decides that he wants to do something a bit more entertaining than a basic slide presentation. So, he fires up the Scratch environment and starts building a little movie for his report. He asked for a little help on a few things, but mostly works at it steadily for several days on his own.
As the project nears completion, we discussed the problem that his teacher might not be willing (or able) to install the Scratch environment to run his movie. With a moment's thought, he decides to get his digital camera and record a video of the movie playing on his screen. He put both the Scratch program and the video on a thumb drive and brought them to school. His report was a success.
Even though I had thought he lost all interest in programming, he had actually learned what he wanted and kept those skills for later use. Unlike this old programmer, he was interested in programming for the end product and not the process.
If any of you out there decide to try to teach programming to kids, you might want to watch for a completely different mindset than we had when we were learning. Given that computers are everywhere in kids lives today (their phones have more power than the first computers I programmed), they probably see programming very differently than we do.
]]>Last year, I had the good fortune to go to SVG Open 2009. Despite following SVG for years and working with it where ever I could, I was still astounded at the creativity and power of the technology shown by many of the experts that were at the conference.
This year I had the opportunity to present at YAPC::NA on the topic Data Visualization with Perl and SVG. I expected a handful of attendees, and was surprised to be presenting to a packed room containing over 50 people. Apparently, there is quite a bit of interest in SVG.
There have also been a couple of changes in the world of SVG lately that should finally lay to rest the claims of SVG's demise.
* HTML 5 will include SVG support in-line with the HTML.
* IE 9 will finally begin supporting SVG natively.
As with many technologies, it looks like SVG has survived its first ten years and is about ready to break out into the mainstream.
]]>At last year's YAPC, I had the good fortune to attend a talk by Karen Pauley entitled Remote Controlled Volunteers. It's not the kind of talk I would normally look for, but I thought I would check it out anyway.
Boy, was I glad I did. Karen (currently The Perl Foundation president) has a lot of experience dealing with volunteers. Her observations of the connections between the open source community and difference kinds of volunteer organizations were right on target. Since my wife does a lot of work with volunteer organizations, I wanted to show her Karen's slides. I was never able to find them.
When I got a chance to talk to Karen at this year's YAPC::NA, I told her (again) how much I had enjoyed the previous talk and how disappointed I was that I had never been able to show it to others. Fortunately, she had polished the talk some more and presented it again. The paper Understanding Volunteers is the essence of the talk I was looking for, and she gave me the link.
If you ever have a need to understand what drives a particular open source or volunteer group, this paper is a real help. Open source projects are as much (or more) about the people as about the technologies. Many of us technologists forget that. Karen obviously doesn't.
]]>I would like to humbly suggest that this concept is (mostly) nonsense. The cause of the biggest complaints seems to be unfamiliarity with the language in question. Just like most of us would find ancient Egyptian or Middle English hard to read, it can be difficult to understand a programming language if you don't know the basic syntax. Even once you learn the syntax, there is a huge difference between understanding code written in the simplest form of the language and an idiomatic program written in the language.
I have programmed in Perl for a long, long time, well over a decade in fact. I have been told right to my face by fans of other languages that it is flatly impossible to write readable or maintainable code in Perl. It always comes as a shock to me, since I have written and maintained small programs and large systems in Perl for years. It would probably come as a bigger shock to the teams that worked on the Human Genome Project that used quite a bit of Perl (and were apparently able to read it). Not to mention quite a few companies that develop and run Perl code all of the time (Booking.com, the BBC, TicketMaster, and cPanel all come to mind.)
When pressed, I often find that the person making this claim has never used Perl, or possibly only wrote a handful of quick scripts. This is hardly enough experience to make a reasonable judgment.
I also have quite a bit of professional experience working in C and C++. I have been told by some that these languages are also write-only. Yet, there are millions of lines of code in each of these languages running everything from embedded systems up to large research projects, financial institutions, and video processing. Obviously someone finds this code at least somewhat readable or it would not be maintained.
The comment from Steve Gibson that kicked this off involved the Forth programming language. Now, Steve is an expert at computer security and (by all accounts) an excellent programmer. But, my experience maintaining and developing a very large Forth code-base over a period of eight years, is quite different than his opinion. Forth is very different, if you are used to most modern languages. But, that does not make it write only.
I particularly find this comment amusing from someone who writes Windows software in assembler. I believe that it is possible to write readable assembler, despite not having seen many examples. But, I'm willing to bet there are many out there who would declare assembler to be the ultimate write-only language.
When I was pursuing my degree in CS, I had the opportunity to learn and program a bit in Lisp. While the syntax is not my favorite, I found well-written Lisp to be mostly readable. I've had a similar experience with the small amount of Haskell I've done. The same goes for SmallTalk and so on.
While I'm not willing to suggest that a particular language is write-only, I can agree that some programs are effectively write-only. But that trait does not appear to be limited to a particular language. In every language I've worked in or even just read, I've seen examples of readable code and examples of garbage I hope to never have to maintain.
In many cases, the really bad examples come from people unfamiliar with the language or just inexperienced. The more readable code normally came from more seasoned programmers. The readable code also tended to become more readable as I became familiar with the idioms of the language or even the team that wrote the code.
Although it is probably not universally true, most of the people I have heard scream write only language have experience with only one or a few languages. Much like we Americans who only know one language and consider all of those foreign languages too hard, many of these one-language programmers are still focused on the One True Syntax and have not moved on to the concepts behind the syntax. (Forgive the stereotype, but it's a good analogy.)
In some cases, the opinion was not a case of single-language blinders, but just plain unfamiliarity.
Not everyone is going to like every programming language. That's a given. Some programming languages don't fit the way you think. But, claiming a language is write-only seems ridiculous. Most of the languages I don't like, I've programmed in long enough to be able to read them, even if I would prefer not to.
I also made an explicit decision after a discussion with another programmer years ago to have no opinion on any language I haven't used. Even if other people have really bad things to say about it. I don't know what it's like, so I have no opinion. I find this makes things easier if the job ever requires me to do a little maintenance on a language I don't know.
I've also found in the past that the languages that worked the most differently from the way I think were the most valuable to learn. Those languages were the ones that taught me the most fundamental aspects of programming, because I got to see familiar programming concepts in a completely new context.
Update: I realized that I should have made clear, that Gibson had made clear that his difficulty with reading Forth was due to unfamiliarity. He does have experience with other languages.
]]>Well, Matt Trout reminded me of it again today with his post Simple Debugging. So, I decided it's time to throw my thoughts into the wind.
When I was a very new programmer (a long, long time ago) I only used print statements to debug. This was partially due to a lack of tools and partially due to a lack of experience. I was introduced to a couple of debuggers, but was not really convinced.
Then I was introduced to a really good interactive debugger. It completely changed the way I worked, I could inspect data structures interactively, chase down long chains of links, and basically do the same sort of stuff I did with print, just faster. I could also set a breakpoint near where I thought things would happen and kind of explore the area to see what was wrong.
It didn't take me long to decide that this was definitely the way to work. Unfortunately, I couldn't rally use this wonderful tool at work where I was working on a background program that could not comfortably be placed under the debugger's control. (This was an old TSR program running under MSDOS.)
I moved on to another job and was working in an environment without an interactive debugger. More importantly, it was the server-side of a client server system. Suddenly, the concept of pausing the whole server to track down some bugs did not seem like such a great idea. Our client-side programmers often used an interactive debugger for their work and I could easily compare their experiences with mine.
I started noticing something interesting. A few people were really amazing with the debugger. They could zero in on a problem with a small umber of breakpoints and a little judicious inspection. Others spent enormous amounts of time single-stepping through the code, eventually finding a problem.
Over of the host side, I noticed something equivalent. There were a few people that wielded their print statements like a scalpel. They would examine code, use a few well-placed prints to uncover surprising behavior. Iterate a few times, and solve the problem. Others would scatter-shot print statements all over the code, printing out reams of data in the hopes of seeing something of interest.
With either tool, the key insight appeared to be the method chosen for placing either a print statement or a breakpoint. The effective programmers strove to understand the code and form a hypothesis of what problem might cause the current symptom. That person would use this hypothesis to determine where a print statement or breakpoint could be used to prove or disprove the hypothesis.
Those people would usually make effective progress toward finding and squashing the bug. The ones that also had the most experience or understanding of the code were particularly effective in this technique. Their initial guesses tended to be much closer to the target.
The other group used either print statements or single-stepping to explore the code. They seemed to almost be on a voyage of discovery. This is admirable, and can be an effective way of learning the code. But, this approach does not lend itself to rapid bug fixing.
In watching several people over a period of years, I've noticed that people using print statements often discover this approach on their own. This is caused by the simple fact that adding and removing print statements is time-consuming and boring. Good programmers soon learn to reduce this cycle with better analysis.
On the other hand, I've seen many developers using the interactive debugger single-step technique continue because it feels like they are making progress. They don't notice the boring stuff as much, because they appear to be moving. It often takes input from a mentor to get them to understand that a better breakpoint choice would make all the difference.
I tend to prefer the print statement approach. This has fit well with the kinds of programming I've done during my career. However, I'm not a fanatic about it. A good interactive debugger can be a joy to use on a code-base that I'm really familiar with. But, as with most of the debates in programming, the most important and powerful debugging tool in your arsenal is the one between your ears.
]]>Some truly amazing demos and research are being done in this area despite nay-sayers claiming that the technology is dead. Some of the biggest announcements leading up to the conference included two solutions for the one major browser that does not yet have native SVG support.
* SVG Web a JavaScript library for running SVG in any browser.
* Google Chrome Frame a plugin that gives IE support for SVG, canvas, HTML5, etc.
The presentations ranged from interesting uses of vector graphics to great tools for business or government to jaw-dropping effects that you would have had to see to believe. Some of the presentations are already available from the SVG Open website, with others hopefully following soon.
One of the most profound things I got out of the conference is the large number of people that have not been deterred by the pessimism that has seemed to surround SVG in recent years. These people are doing amazing work and getting things done, despite inconsistent browser implementations and one, big holdout among the major browsers. Everyone seemed excited that browser implementations are becoming more complete and consistent with every release. The HTML 5 specification is defining the ability to write SVG inline.
Truly, SVG is coming of age.
[1] Disclaimer: I was on the organizing committee.
]]>Since my experience has not been quite that good, I thought it was worth documenting what I have seen. Based on some of the responses I've seen to any negative comments I expect to be blasted if anyone actually reads this. But, if anyone else runs into these kinds of problems, you'll at least know one person has seen the same.
I have fairly simple needs from a version control system based on the last couple of decades of software development.
I don't have a particular agenda or approach I care about. I just want to be able to work with my code and have the VCS help me. At present, I don't have a major need for distributed version control, but it might be nice. For me, a VCS is a tool, not a religion.
As time has gone on, I have used several VCS or SCM systems. These include: RCS, CVS, Subversion, and ClearCase. (I also had to help support some people using SourceSafe long ago.) I've done branching and merging in all of those (except RCS), so I'm fairly conversant with the general issues.
A couple of years ago, I tried to use git and was badly frustrated, I could not get data committed. I wasn't able to follow my normal workflow. The tool forced me to completely change the way I was working. I immediately dropped it. I had a similar experience with another distributed VCS tool (I don't remember which one), and so I discounted the whole mess as a bad idea.
Later, a fellow Perl Monger started talking a lot at the local meetings about how git was working well for him, and even gave a talk on the subject. It seemed like git might be worth trying again. With the information from that talk and better online resources that had become available in the intervening time, I was able to use git for a few minor projects I was working on.
It turns out that my original problem had to do with the index feature. This had apparently been a problem for many people and the newer tutorials made a point of explaining this feature better. I eventually got used to the extra step of re-adding files I had changed (or using the -a switch).
I was becoming somewhat comfortable with the tool.
Because I was comfortable with my Subversion repositories and I had been told about the git svn tool, I was using Subversion as my remote repository. This allowed me to have it backed up with all of my other repositories and fit my comfort zone better than having the whole repository in the local directory. (I know a lot of people swear by that feature. But I remember disasters in the old RCS days when the repository was also stored with your sandbox. It was too easy to lose both your current work and all of the history with one mistake.)
Things seemed to be going along okay until one day when I decided to push some changes from my laptop to the Subversion repository and pull them to my working directory on my desktop.
At the time, I think that my desktop was up to date with the master branch. I had been doing some history rewriting clean up a few commits on a (local) branch. I merged a branch on my laptop to master and pushed the changes to the Subversion repository. A day or two later I pulled from the Subversion repository to my desktop machine. (The details are a little hazy since I expect the version control to keep up with what I've saved and when.) When I did, there were merge conflicts like crazy. Almost every file was conflicted somewhere. I tried to resolve the conflicts by hand and could not get everything back into a stable state.
This was quite surprising, because I've merged multi-month long branches in CVS (with much pain and suffering) as well as resolved merges in Subversion without much problem. Given the hype about how easy merges are with git, I was not expecting this.
Eventually, I came to the conclusion that the best thing to do was to blow away my working directly and start over with git svn with a new working directory. This was not a good feeling. Although I'm pretty sure there was nothing in my desktop working directory that I lost, this was not the kind of behavior I expected from a VCS.
With some research, I eventually convinced myself that I must have messed up somewhere in the history rewriting and that was the cause of my mistake. Maybe history rewriting and Subversion weren't compatible or something.
A few months later, I was working on another project. I was still using Subversion as the remote repository for working with git. Honestly, despite all of the assurances that everything that goes into git comes back out again, I was still more comfortable with Subversion for safety.
Once again I was working on the code from two different machines. I had just finished some relatively hairy work and pushed to the remote (Subversion) repository. A day or two later, I pulled on the other machine and BAM, I'm in conflict hell again. I tried to resolve the issues without a whole lot of success. The conflicts did not seem to match up with what I could see on either machine. This time I'm sure I hadn't done any history rewriting. (I wasn't using that feature after the previous disaster.)
After fighting with the mess (and pulling two or three more times), I eventually gave up and blew away my working directory and rebuilt it.
I checked for similar stories on-line and talked with my local expert to no avail.
The biggest problem I had with it was that the actions that blew up that day were identical to things I had been doing all along. I couldn't track down exactly what I'm doing wrong or even localize it to a sequence of steps that caused the problem. As a developer myself, I know that reporting a bug that randomly blows up after doing something that worked the last dozen times in a row was not going to be taken too seriously.
At this point, it's worth reminding you that I've been using version control tools for almost 20 years. I've recovered from disasters in almost every one that I've used. I have never been left in this situation before.
Despite a few bad experiences so far, most of my usage of git has done what I needed. I was still not completely comfortable with the new workflow. But the ability to add aliases and script new commands is quite addictive. I knew that my experiences had to be odd, otherwise people would be reporting them and dropping git like a hot rock.
I had a new project that I wanted to work on, so I decided to do things a little differently. This time, I worked entirely in git, no Subversion repository. I made a bare repository along side my Subversion (and CVS) repositories to give me a single spot to back up and began working on the new project.
Things went along fine for a month or so, until I needed to get ready for a conference. I had been working mostly on my desktop, but would need my laptop updated before I could go to the conference. I merged a couple of feature branches back to master and made certain everything was working fine. I pushed the master branch to the remote (git) repository. I went immediately to the laptop and did a pull. BAM, my working directory was suddenly a smoking crater with conflict shrapnel everywhere.
Unfortunately, this was a Catalyst project and I ran into a new kind of conflict hell. The Catalyst system uses a Perl ORM that creates class descriptions for data stored in a database. The main description of the classes are protected by an MD5 sum to show they haven't changed. Merging files with this sum in it are guaranteed to have conflicts. Unfortunately, I couldn't get the code to match up with either MD5. By this point, I had learned about the git reset command. But I still wasn't able to completely recover.
In any normal circumstance, I would probably have throw away any tool that causes me this much grief. I have not yet had the religious conversion that many seem to have where git is concerned.
The only saving grace is that despite the disasters, I've always been able to recover my code (if not my working directory). I also haven't been able to nail down the problem. My latest attempt to stop the problems is to stop using the git that comes with Ubuntu and update to the latest.
I'm not convinced that git is as wonderful as everyone says, but it does have features I like.
Unlike most of the people writing about git, I'm not a true believer. It's got some advantages over the systems I've used before. But, despite their flaws, I've never had either Subversion or CVS to leave me with a smoldering crater where my working directory was.
I'm sure that someone (if anyone actually reads this) will tell me that I'm doing something horribly wrong, or that I just don't understand the beautiful elegance of Linus's vision. Frankly, I don't care. Elegance of design doesn't matter if the implementation blows up. As Richard Feynman once said,
It doesn't matter how beautiful your theory is, it doesn't matter how smart you are. If it doesn't agree with experiment, it's wrong.
I use version control to support my code. If it doesn't, I will switch to a new system. I haven't given up on git yet (it hasn't yet lost any code I've committed), but one more explosion may the last.
]]>I finally got a reasonable release out on CPAN as SVG::Sparkline version 0.30. This version supports 6 different sparkline types: Area, Bar, Line, RangeArea, RangeBar, and Whisker. It also has better documentation in the form of a manual and a cookbook.
Proving once again that you never know who's watching, Jeff Schiller (of SVG fame) commented on my last post that I needed a demo. <grumble/> I had a little demo application that I was using to print a primitive gallery of sparklines for my own debugging purposes. I've cleaned up the output and put the Sparkline Gallery on-line.
I also found that the module did not have as high Kwalitee as I expected. So, I did some cleanup to improve that value. (If you haven't run across kwalitee before, you might think it's a misspelling. Actually, it's a part joke/part serious measure for Perl modules. Check out the article for details.
I would have had this version finished sooner, if another SVG project had not distracted me. David Dailey mentioned the idea of a Friendly Little Intermittent Clockfest. The last few times this subject came up, I was not tempted. For some reason, this time it bit me. Sometime soon, I'll probably be adding an SVG clock gallery to my site as well.
Every now and then the programming muse shows up and the ideas start flowing.<grin/>
Update: Thanks to a minor failure on my part the 0.30 release was missing the Manual. I've released 0.31 to fix that.
]]>In the process, I've been exploring a few other ideas that were not part of the original plan. I planned to develop the module as I usually do and then release it to CPAN. I released an early version (something I normally do not do) and got feedback from an interested developer within 24 hours. This was on a barely functional module.
Based on that email exchange and conversations with some local developers, this is turning into a much more robust and flexible module than originally intended. The interface has improved dramatically. There will be more types than originally planned. There are more ways to configure the look of the generated Sparklines. There is now a cookbook explaining how to generate different effects.
I've also put the module on Github as svg-sparkline. Although I've been playing with git off and on for a few months now, this is the first time I've tried to use with it seriously.
Unlike many of my past projects, this one requires more visual design thought than I normally need. In a way, this quick, little project has turned into a way to experiment with several new ideas and skills at once. We'll have to see how this one turns out.
Note:
* A Sparkline is an intense, word-sized graphic intended to convey useful information inline within text. The concept was proposed by Edward Tuft in his book Beautiful Evidence. For more on sparklines, see Tuft's article on the subject.
Any real system will require some amount of Chronistic coupling. The key design point is to decide how much. Choosing the wrong level of coupling will certainly impact how your system evolves in the future. Over the next few posts, I'm going to explore some of these levels of Chronistic coupling with some examples.
Once upon a time, people doing communication between two processes (or computers) regularly debated how the data should be transferred: ASCII or binary. (This was pre-Unicode.) The advocates of the binary approach argued that it was more efficient for two reasons:
When we transferred data at 1200 or 2400 bps these arguments were pretty convincing. Especially when communicating between processes on the same machine.
However, there were problems when communicating between machines that were not the same architecture. When crossing the architecture boundary, you had to do conversions anyway. Some places where the binary format might change include:
Soon, a sizable amount of effort could be applied to converting binary data from other machines to the native format. The worst part about this was the lack of information in the data stream to help troubleshoot problems. Normally, you found out that your decoding logic was wrong when some portion of the binary data stream gave ridiculous results, or when you got to the end of the stream and found you had too little or too much data.
Meanwhile, text-based protocols sent more data over the wire (which became less of a problem as networks became faster). But, where a text-based protocol really shines is in debugging the data stream. If the next number in the stream is 1000000 and you expected a 16-bit short int, it's easy to see there's a problem. In a binary stream, the first two bytes of a long int look the same as an actual short int, there's no way to tell (at the protocol level) that something is wrong.
There were still problems. There was the EBCDIC vs. ASCII issue, which has mostly gone away. There is also the line ending problem, (LF vs, CRLF vs. CR).
The biggest win for the text-based protocols was the success of TCP/IP protocols on the network. A large number of the protocols that run the Internet are basically text. For example, HTTP, SMTP, FTP, Telnet, and more are basically a series of text strings sent between the client and server.
The major solutions to the size issue are relatively straight-forward. First, the networks got faster, so the problem is less of an issue. In places where bandwidth is still a problem, we can compress the text stream (gzip) to reduce the number of bytes. Since the compression is something that can be used by everyone, it is been greatly optimized over the years giving more benefit to everyone.
As a result of the (possibly compressed) text-based protocols used on the net today, machines with very different architectures can communicate easily. Text protocols have a lower chronistic coupling than binary protocols. An email client written to work on 16-bit Windows 3.1 could send messages to a client on a 32-bit Windows XP system. A web page served from a 64-bit Linux box can be viewed comfortably on Mac OS X, Windows Vista, or a mobile phone. More importantly, these clients don't need to know if the web page was generated from a C++ program, Ruby, Java, Lisp, or even Forth. It just doesn't matter.
Our video and audio formats are still binary because of the large amount of data being transferred. We still have chronistic coupling issues there. If you don't have the right codec for the file, you are basically out of luck. Many of these codecs are tied directly to the architecture where they were written.
In this case, the trade-off for reduced size is still more important than the ease of porting to multiple architectures.
]]>Since that time, I have worked with other systems with similar functionality and have decided the problem was worse than I described five years ago. Serializing an object to disk with the intent of reading it in at a later date, couples the structure of the object from a past date to the structure of the object at a future date. If the object never changes form, that is not a problem. If the object structure needs to change, then the serialization process becomes more complicated. It has to take one of three forms:
Recently, I have begun calling this effect Chronistic Coupling. (I like Temporal Coupling better, but that name is already taken.) Although you might think of this as another manifestation of Data Coupling, I think the time element makes Chronistic Coupling stronger (and more subtle) than data coupling. Unlike simple data coupling, object serialization couples the object structure through time. The older object format reaches forward in time to effect how the new program can structure its data.
If we allow saving in old formats, we must be very careful not to introduce an anachronism. This would be an old-style object that is inconsistent with the old program. This can cause problems that are hard to troubleshoot. You have to be able to identify where the old data came from to determine the problem. (In one system I worked on, we augmented the version of the data set with an extra piece of data describing the version of the program that saved this data.)
There is a sort of seductive quality to the idea that we can serialize objects and reinstantiate them at another time. This pattern recurs many times in the field of programming. Although it seems like a really good idea to have the data completely encapsulated by the object by serializing and deserializing the data straight to storage and back, the reality is there are still tradeoffs.
The obvious issue is to be certain that the data we read in is consistent with the design of the object. Most serialization needs to be augmented with some form of validation.
A separate issue that people often don't notice is that changes in the responsibilities and structure of an object can be hampered by Chronistic Coupling. At the very least, the code needed to deserialize old objects becomes much more complicated. In the worst case, it may be necessary to keep older classes in the design for the sole purpose of allowing us to convert old object into new object.
Where things really start to go bad is when a substantial portion of an object hierarchy changes. The object you have serialized may not bear any resemblance to the new classes. If the new object hierarchy is different enough, you would have to parse the old serialized object into a neutral format that can be used to instantiate the new objects, Either that, or you don't make the design improvements, because the work is too great (for this release).
In this way, the old design reaches into the future to prevent changes to the design. Often, the only way to fix the problem is to abandon backwards compatibility. This may result in major problems for clients or the need to provide special utility software to convert old data to a new format.
I am not saying that object serialization should always be avoided. The purpose of coining the term Chronistic Coupling is to give name to a cost that you may not realize that you are paying. In some cases, it might be better to store data in an object-neutral format and build new objects to represent the data rather than store the objects themselves. The unfortunate part of this is that there is no magical way to convert your objects to and from this simpler format.
No software can exist without some forms of coupling. However, some of the best minds in our field remind us to reduce coupling where we can. If you decide to use object serialization, remember that you are increasing coupling in the time dimension. It is important to consider whether or not this increased coupling is worth the cost.
]]>