Category Archives: Software Engineering

Brook’s law, why software engineering is not programming

1 Reply

Part 2 of Managing Software Projects

Do you know the difference between programming and software engineering?

Not a Jeopardy question, the difference affects tens of millions of people who work directly on software. Programming is writing instructions for a computer, something everyone should learn. Software engineering occurs when two or more people work on a project, which introduces a major difference: communication. Communication is also something everyone should learn. Writing good instructions is clearly important but the communication between two or ten or a hundred programmers quickly becomes the critical factor in producing a good product.

No it's this way

The important thing to know here is Brooks’ law. Fred Brooks ran software development for IBM during the 60s and 70s; thousands of smart people writing millions of lines of code. For large projects like operating systems he tried to scale up by adding more programmers, just as a construction project would add more workers or a factory would add more assembly lines. But it didn’t work.  He concluded that the more programmers he added the slower things went.

Brooks formulated his law of communication as “adding programmers to a late project makes it later.”

Here is the problem. For every programmer there is a communication link between them and every other programmer working on the project or n(n-1)/2 links. For two programmers you have one communication link; three, three links. So far, not so bad, sit those three people in the same room and they might avoid too many misunderstandings. Keep going up, with seven programmers you’ve got 21 links, about half of everyone’s time is spent coordinating with others. A sixteen programmer team? 256 links!

Clearly the project cannot be built in half the time by doubling the number of workers.

Very quickly the number of communication paths increases faster than the number of programmers,  Worse it’s not just programmers, it’s everyone who needs to intimately know the software: QA staff, the development manager, the product manager. This limits the number of people who can work on a single project.

bricks_smallBut wait, plenty of companies employ thousands of programmers, how does Lockheed Martin build something like an artillery targeting system for the army?  One piece at a time, with vast amounts of planning, hundreds of people dedicated to project communication, working for decades and probably 100% over budget. Much of this effort goes into splitting the project into pieces that a small team can handle and defining detailed interfaces to minimize the communication they need to do with each other.

So that’s the difference, communication is the key skill of software engineering. Software engineers are often not known for their communication skills – or maybe we should say programmers are not known for their communication skills but a well-functioning software engineering team is one that communicates well.

Project Failure

1 Reply

Part 1 of Managing Software Projects

Your project sucks.

It’s months late, over budget, full of bugs, or maybe it’s a total bust.  Worse, if you are managing the project you don’t even know how bad it is because you can’t spend your time crawling through the depths of the code.

At least you’re in good company. 70% of software projects fail to meet their budgets and deadlines and 1/4 of projects fail completely.

If failure is common, what can we learn?

First, let’s take off those rose-colored glasses that we put on every time we work on our résumé and look through our past jobs. “How many of your software projects have failed?”  It’s likely that not many come to mind, certainly not 70%. Most software operations tend to forget their failures or they never really kill off projects,

letting them sit around in a zombie state sucking morale from employees’ brains. Or, large projects are often partial failures once you measure the amount of maintenance required to run them;  a drag on the company rather than an engine of growth.

It may also be complicated, “These apps work, these features don’t, I don’t even know if that part works or not.” Especially for the project you are working on right now, it’s useful to think about whether different modules or features are successful or not. After all, the goal is to apply some bolt cutters to the parts that are a weight chained to your ankle and let the other parts of your project soar.

So it’s not enough to ask, “How many of your software projects have failed?”.  We’ll also ask “How did these projects fail?” Failure can take a lot of forms and it’s hard to measure something that people struggle to quantify and struggle to get good data on.

The largest source of failure, product failure, involves a lot more than software management. Getting the right product to customers for the right price, selling them on it, that’s the goal of most businesses and that’s the elephant’s graveyard of failures. Again, features or modules are often in this category. A set of features are developed for customers at great expense and then never rolled out or never purchased. Good prioritization can help with this problem, it’s a good compass, but when your customers are on the other side of the river a compass is not much use.

However, for now we’re concerned with software project failure—things that cause software projects for a good product to go awry, to fail completely, or to fail partially. First there are projects that got canceled. Those are clearly failures, right? Something was started, got some way along—a month, six months, two years, ten years—and then someone finally realizes that it’s a disaster and the company needs to stop working on it. Often this is because it’s gone way over budget or way over deadline. As the small problems add up over months or years, it’s easier to see, “Oh, this has gone really wrong.” “We were supposed to complete this in a year, it’s been two years, and it’s still not done.” Particularly this happens on projects for the government where there’s a set initial budget, things go late, extra funds are procured, but eventually things go so bad that the whole project ends up getting canceled.

Other projects get built and never rolled out, canceled in effect by the users. There isn’t enough checking in as the project’s being built, so when “suddenly” it’s time to deliver the project people realize, ‘Hey, this doesn’t do what we need,’ or ‘It does most of what we need but this last 10% is so critical that we’re going to ditch the whole thing; we can’t make the transition to that new software.’  All of these are failures due to lack of communication with users.

Another case is the cobbled-together software in use by customers that everyone agrees desperately needs to be redone. Similarly you often find software where the maintenance costs are too high, it’s not extensible enough, it’s breaking too often, it’s pretty clear  that new software needs to be built. A new software project is launched. Version 1 was at least a partial failure, thus we have Version 2: the complete rewrite.  Or 3.  Version 4: the complete rewrite? I think I saw that one already, it was much better with that one actor, you know, the guy with the long hair and the Linux Rules T-shirt?

When you add up these different kinds of software project failures you get that headline failure rate of 70%. People have studied this for decades and many studies come to the same conclusion.

Why is software hard?  Next up Software Engineering…

Partial test coverage

1 Reply

Software, it’s a mean scary world full of monsters just waiting to get you.

By which I mean everything is nice and cozy when coding away on a cool new feature within the four walls of your company. But there is a whole world of bugs and performance problems and security holes and confused users out there just waiting to pounce.  Just let that code into the wild they whisper.  Give it a basket of goodies and a red hood and send it out the door.

Which is why we write tests.

Tests are windows that let us see the monsters lurking outside.  That monster has a big mob of users, better not let this feature out the door until it can run extra fast.  Those hackers like our basket of goodies, lets give this feature some more layers of protection … and a flamethrower.

One of the projects I work on has about 10% test coverage.  When tests break or need upgrading someone (clearly of the non-test writing persuasion) will ask, “why should we spend time adapting the tests to the new code?”

Because, it may just be one window, I can only see part of the back yard but at least I can see a few things.  Sure, you can always go out the door and look around; maybe there’s nothing out there. With at least one window, if anything really big is lurking about, you’re sure to see part of it.

Is that a tail?

Managing Software Projects

2 Replies

With software “eating the world” more and more people find themselves managing software projects.  While many people have experience programming and others have experience managing teams, software engineering has its own set of challenges.  Creating software that solves a problem, is easy to use, doesn’t break all the time, doesn’t die under load, is easily changed and adapted, that new programmers can learn quickly and for a reasonable price… is hard.  In this blog series I’ll cover the forces that drive software and what you can do to keep your project in the fast lane instead of  broken down on the side of the road.

The general outline.