Previous ToC Up Next

0. Preface

0.1. The ACS Initiative

This book is the first volume in the Development Series of the Maya Open Lab project, for constructing a laboratory for simulations of dense stellar systems in astrophysics. The Maya project in turn is the first concrete example of projects developed in an initiative titled The Art of Computational Science (ACS). The aim of the larger ACS series, of which the Maya Series will be a part, is to provide real-world guides to designing, developing, using, and extending computational laboratories.

In many areas of science, computer simulations of complex physical systems cannot be performed with off-the-shelf software packages. Instead, computational scientists have to design and build their own software environment, just as experimental scientists have to design and build their own laboratories, before being able to use them. For a long time, however, the know-how needed to construct computational laboratories has remained only a form of tacit knowledge.

Unlike explicit knowledge that can be found in manuals, this type of implicit knowledge has been alive in conversations among experts, and has been passed down in that way only as a form of oral tradition. This kind of knowledge has not been written down anywhere in sufficient detail to allow it to be passed down without direct personal instructions, or indirect osmosis through personal participation in a joint project.

We have started the ACS series with the aim of making explicit the implicit knowledge of experts of scientific simulations. Besides offering detailed explanations of the structure of the computer codes used, in an `open source' style, we provide a deeper layer of knowledge. Besides the what and how for any computer code, we also provide the why: the motivation for writing it the way it was written, within the context in which it was conceived. This will give the user more appreciation for the background of the structure chosen, and most importantly, this will give the user the ability to easily modify and extend the codes presented, without finding oneself at odds with the original style and aim.

The umbrella series within ACS is called Open Knowledge, to indicate the fact that we aim at far more than providing an `open source' project. In that series we plan to publish what we have learned about the ACS approach, insofar as it is generally applicable, to any area in computational science.

0.2. The Maya Series, and the Kali Code

The Maya series will contain the first such fully worked-out example of building a computational lab. The first several volumes in this series will focus on one particular topic, the gravitational N-body problem, the oldest unsolved problem in mathematical physics, and an area of active research in astrophysics, with applications ranging from star formation to collisions between normal stars, neutron stars and black holes, with implication for the growth of structure in the universe on all scales, from that of clusters of galaxies down to the formation of planets and their satellites.

One reason to choose this topic is to provide an example of a computational lab where the underlying physics is very simple, just Newton's classical law of gravity. The complexity lies exclusively in the emergent properties that show up in the paradoxical thermodynamics of self-gravitating systems, where negative specific heat and other conundrums drive each system far out of equilibrium in ways that are theoretically still ill understood, yet can be modeled accurately in computational labs.

Given the simplicity of at least the underlying physics, we hope that the venerable N-body problem can function as a paradigm for the exploration of new principles and methods for a novel generation of computational laboratories. With this idea in mind, we are trying to present these first few books in sufficient detail for researchers from other areas of natural science to be able to follow and appreciate the approach to virtual lab building presented here.

The aim of the Maya series is to develop, in many small steps, the Kali code, a state-of-the-art code that can be used for research in dense stellar systems.

The name Maya seemed fitting for two reasons, one connected with Middle America and one with India. The Maya culture was very good at accurate calculations in astronomy. And the word maya in Sanskrit has the following meaning, according to the Encyclopedia Britannica: ``Maya originally denoted the power of wizardry with which a god can make human beings believe in what turns out to be an illusion.'' Indeed, a simulation of the heavens is something virtual, an illusion of sorts, and a considerable feat of wizardry.

We borrowed the name Kali from the Sanskrit kali, meaning dark, as in the kali yuga, the dark ages we are currently in according to Hindu mythology. The same word also occurs in the name Kali, for the Hindu Goddess who is depicted as black. The term dark seemed appropriate for our project of focusing on forms of tacit knowledge that have not been brought to light, so far, and perhaps cannot be presented in a bright, logical series of statements. Instead, we expect our dialogues to carry the many less formal and less bright shades of meaning, that pervade any craft.

Future volumes in this series are expected to cover wider and wider areas of science. Within the Maya series, we will move from stellar dynamics, associated with the gravitational N-body problem, first to stellar evolution and stellar hydrodynamics. It is our hope that this series will be followed by additional series for other areas of simulations in astrophysics and physics in general. And before too long, we hope to attract chemists and biologists, and well as scientists from other areas. If they find our approach useful, and like to try it out in their own field of expertise, we will welcome them to join this series, and to provide additional volumes discussing computational labs for their specialty.

The current volume is the first one in the Development Series of the Maya Open Lab project. In parallel, we are also writing a School Series for the same Maya project, as a more educational counterpart to, and introduction for, the full Development Series. In the School Series, also in dialogue form, we introduce the basic notions of physical forces and resultant motions, the associated mathematical description through differential equations, and numerical techniques for finding general solutions to those equations. Here, too, we welcome others to join us with similar educational approaches, applied to other topics.

0.3. A Narrative in Dialogue Form

The format we have chosen for our book series, is sufficiently unusual that it deserves some clarification. Most scientific text books summarize the state of the art of a field `after the dust has settled.' Students are presented with a clean path of logic that bears little resemblance to the historical process of trial and error that underlies any scientific discovery, large or small. As a result, a graduate student facing his or her first major piece of independent research is in for a big shock, suddenly realizing how hard it is to derive anything new with little or no guidance.

In practice, the blow is softened in many cases by conversations with a range of helpers, from fellow students, somewhat more senior postdocs, as well as thesis advisers and other faculty members. However, not everyone is so lucky as to find sufficiently helpful advice that way, and in a world of increasing specialization, the advice of even the most well intended helpers may not suffice for the problem at hand. This is especially true in a rapidly evolving area such as computational science, where the most basic tools, from languages to packages to the very approach to programming, have changed so much even in the last ten years.

Nor is browsing in a book store is likely to turn up much that is helpful. While there are many books that discuss algorithms and programming languages, those are necessary but far from sufficient ingredients for building a computational lab. What is needed is a practical form of know-how, of how to approach the whole process of lab building, from developing an overarching vision to connecting it to nitty-gritty details. To present this form of know-how, via case studies, is the central aim of our book series.

Given this wide-ranging challenge, a traditional approach of just summarizing our basic points would most likely be insufficient to let students apply our method to their own specific problem case. Instead, our approach is based on giving a hands-on feel for the many pitfalls and diversions on the road to the design and development of large and complex software systems. And the only real way to get such direct experience is to make mistakes, many of them, and then to learn from them, systematically.

Even the best researchers spent most of their time making mistakes; it is the fuel for the engine of exploration. Our aim in this series is to share this approach with our readers, providing them with a sample of the types of mistakes we have made in the the quarter century that we have been working in computational science. The best way to share this atmosphere of trial and error, we think, is through a dialogue between researchers who resemble ourselves, in their way of formulating plans, noticing problems, adjusting their aims, finding ways around stumbling blocks, in short, dealing with the joys and frustrations of every-day research.

One of us (J.M.) first applied this approach in a short book called Pasokon Butsuri Jitchi Shidou (Practical Computer Physics) [1999, Kyouritsu Shuppan, Tokyo, Japan]. Since we were both happy with the result, we decided to continue this style in the ACS series.

We could of course have written a fictional dialogue between the two of us, distilling our experience of working together for much of our professional life: since 1988 we have been coauthors on some fifty scientific publications, and together we have organized two symposia sponsored by the International Astronomical Union and edited the corresponding proceedings, as well as a number of workshops. However, such an literal approach would have been too narrow, and we decided it would be much more fun to use composite characters, drawing from history and imagination.

0.4. ACS versions

We use the name The Art of Computational Science not only for our book series, but more generally for the software environment for which the books provide the narrative. The environment includes the collection of computer codes discussed in the books, together with the infrastructure to make it all work together seamlessly. This implies extensive comments provided in the codes themselves, as well as manual pages.

Our plan is to make successive stable versions of this software environment available, starting with ACS 1.0, which contains a small but self-sufficient core of simple N-body programs and accompanying documentation and narrative. These versions can be freely downloaded from our web site "http://www.ArtCompSci.org". They include all completed and partly completed volumes in our book series. Text, code, and everything else is presented as open source software under the conditions of the MIT license:

    Copyright (c) 2004 -- present, Piet Hut & Jun Makino

    Permission is hereby granted, free of charge, to any
    person obtaining a copy of this software and associated
    documentation files (the "Software"), to deal in the
    Software without restriction, including without
    limitation the rights to use, copy, modify, merge,
    publish, distribute, sublicense, and/or sell copies of
    the Software, and to permit persons to whom the Software
    is furnished to do so, subject to the following
    conditions:

    The above copyright notice and this permission notice
    shall be included in all copies or substantial portions
    of the Software.

    THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF
    ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED
    TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A
    PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT
    SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
    CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
    OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR
    IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
    DEALINGS IN THE SOFTWARE.

0.5. Background

The quickest way to obtain more background information about dense stellar systems is to consult the web site "http://www.manybody.org/modest.html" of the MODEST initiative (for MOdeling DEnse STellar systems). There you can find out about its many workshops and working groups, as well as teaching activities, including N-body Schools where we regularly present the current ACS material. You can also find literature references there, under working group 8.

Two specific recent references, close to the material presented in the Maya series, are:

0.6. Historical Note

The current version of this introduction was rewritten in December 2006, to take into account the name change from Kali series to Maya series, and the introduction of a School series in addition to the Development series that was started in 2004.

0.7. Acknowledgments

Besides thanking our home institutes, the Institute for Advanced Study in Princeton and the University of Tokyo, we want to convey our special gratitude to the Yukawa Institute of Theoretical Physics in Kyoto, where we have produced a substantial part of our ACS material, including its basic infrastructure, during extended visits made possible by the kind invitations to both of us by Professor Masao Ninomiya. In addition, we thank the Observatory of Strasbourg, where the kind invitation of Professor Christian Boily allowed us to make a rapid beginning with the current volume.

Finally, it is our pleasure to thank Douglas Heggie, Stephan Kolassa, Ernest Mamikonyan, Bill Paxton, Michele Trenti and John Tromp for their comments on the manuscript.

Piet Hut and Jun Makino

Kyoto, June 2004 (revised in December 2006)
Previous ToC Up Next