2.1 Our Setting

We want to convey some of the atmosphere in which large software environments are grown, in a dynamic and evolutionary way, often starting from very small beginnings - and always surprising the original authors when they see in what unexpected ways others can and will modify their products in whole new directions. Most of our narrative will follow this process step by step, but occasionally we will turn away from the process to the players: the developers writing the software. We have chosen one representative of each of the three target groups mentioned in our preface, from natural science, business and computer science.

The setting is an undergraduate lounge, where three friends occasionally hang out after dinner, and sometimes tell each other about the projects they are working on. Tonight, Alice talks with great animation to her friends Bob and Carol. Alice is an astrophysics major, Bob is preparing for business school, and Carol majors in computer science.

Alice:

Guess what! Today I was asked to choose a junior project, for me to work on for half a year. Many of the choices offered seemed to be interesting, but for me the most exciting opportunity was to work on the overhaul of a laboratory for interactions between stars.

Bob:

What are the interactions that are so interesting?

Alice:

Imagine this, the current software package allows you to create a star cluster and to let it evolve for billions of years, and then you can fly through the whole four-dimensional history in space and time to watch all the collisions and close encounters between normal stars and black holes and white dwarfs and you name it!

Carol:

If that package already exists, what then is so exciting about an overhaul?

Alice:

Yes, the package exists, but every large software package tends to grow and to become overweight. As you both know, this is true in business-driven software projects, but it is even more true in science settings, where the value of clean software engineering is underrated even more than in more profit-oriented areas. As a result, by far the most reasonable and efficient way to extend older packages is first to do a thorough overhaul.

Bob:

I see. You mean that rewriting a package is worth the time, presumably because you have already figured out the physics and you have similarly built up extensive experience with hooking everything together in various ways in software.

Alice:

Exactly. Rewriting a package takes far less time than writing it in the first place - if we want to keep the same functionality. In practice, it may take longer than we think, since we will for sure find new extensions and more powerful ways to include more physics. As long as we don't get carried away, and keep our science goals in sight, this extra time is well spent and will lead to greater productivity.

Carol:

I wonder, though, whether a complete overhaul is desirable. I have just learned about a notion called refactoring. The idea is to continuously refine and clean up code while you go along.

Alice:

Yes, that would be better. In fact, I already had a brief chat with my supervisor, and he mentioned just that. He said that this was the last really major overhaul he hoped to do for his software environment. The main reason for the planned overhaul is to make it flexible enough that the system from now on can grow more organically.

Bob:

The overhaul that will be the end of all overhauls!

Carol:

Well, maybe. I've heard a lot of hype about programming, in the few years that I have been exposed to it. But the basic idea sounds good. And even if you will have to overhaul in the future, a cleaner and more modular code will surely be easier to understand and disentangle and hence to overhaul.

Bob:

May I ask a critical question? You have half a year to get your feet wet, doing a real piece of scientific research. Would it really be prudent to spend that time overhauling someone else's code?

Alice:

I asked that question, too. My supervisor told me that a through-going attempt to improve a large software environment in a fundamental way from the bottom up is guaranteed to lead to new science. Instead of overhauling, a better term might be fermenting. You will reap the benefits of all the years of experience that have gone into building the software, from working with the science to the figuring out of the architecture of the software environment. Those who write the original code won't have the time to do a complete rewrite; they have become too engrossed in teaching and administration. But they will have time to share their experience, and they will gladly do so when they see someone seriously working on improvements.

Carol:

In other words, during this coming half year you might find yourself engaging in actual research projects, as a form of spin-off of the overhauling, or fermenting as you just called it?

Alice:

Exactly.

Bob:

You know what? Perhaps this is a silly thing to suggest, but I suddenly got an idea. It seems that Alice today has started what amounts to an infinite task. She will have her hands full at it, even if she could clone herself into several people working simultaneously, and she is not expected to reach anywhere near completion in half a year. At the same time, she is expected to start absolutely from start. If she wouldn't do so, it wouldn't be a complete overhaul. Here is my proposal: how about all three of us pitching in, a couple times a week, after dinner, using the time we tend to spend here anyway?

Carol:

To keep Alice honest?

Bob:

Exactly! Of course, she may well get way ahead of us into all kind of arcane astrophysics applications, but even so, if we plod behind her, asking her questions about each and every decision she has made from the start, we will probably keep her more honest than any astrophysicist could - simply because we know less about astrophysics than any astrophysicist! And besides, for me too it would be a form of fun and profit. I intend to focus on the software industry when I go to business school, and I might as well get some real sense of what is brewing in the kitchen, when people write non-trivial software systems.

Carol:

Hmm, you have a point. Obviously, something similar holds for me too, in that I can hone my freshly learned theoretical knowledge on realistic astrophysics problems. What do you think, Alice, are we rudely intruding upon your new project?

Alice:

No, on the contrary! As long as I keep my actual project separated, as Bob stressed, I am more than happy to discuss the basics with you both during after-dinner sessions, as long as you have the stamina and interest to play with orbital dynamics of stars and star systems. And I'm sure we will all three learn from the experience: I would be very surprised if you didn't inject new ideas I hadn't thought about, or notice old ideas of mine that can be improved.

Bob:

Okay, we have a deal! Let's get started right away, and get back here tomorrow, same time, same place.

Carol:

Okay, but let's say almost the same place: next door is the computer center, where we will be able to find a big enough screen so that the three of us can gather around it, to start writing our first star-moving program.

Alice:

An $N$-body code, that is what it is called. Okay, I'm game too. See you both tomorrow!