Software tools I'm using today

The title of this article is a bit misleading, as my primary intent is to document my current approach to producing numerical simulation code -- mostly the libraries/classes involved. However, I'll also cover my "workflow", which is laughably simple.

My research focuses on developing neural networks and neuron simulation software so that I might gain a bit of insight into how nervous systems perform computations. This tends to be computationally intensive work. The individual simulations aren't very huge, but I often need to run tens of thousands of them to systematically explore various combinations of parameters. The simulations themselves are also somewhat fluid, in that there's often a number of different structural variations in the models to explore, too. Once upon a time, I tried to write all my code with graphical user interfaces, first using custom libraries I wrote for Turbo C under DOS, then for X Windows, and most recently for MATLAB. The problem with GUIs is that virtually every change in the simulator produces changes that propagate through the GUI. This is incredibly time consuming. And, while MATLAB is a great environment for numerical analysis and visualization, interfacing it to C or C++ code involves too much glue code. Eventually the process of changing the glue and GUI dominates any simulation-oriented coding.

So, I've stopped integrating the simulators into a GUI, and now write stand-alone code, mostly in C++ using Emacs, the g++ compiler, and RCS (Revision Control System). I don't even use a debugger, just output to cerr, because the few bugs I produce are usually the type that a debugger isn't very helpful for (OK, I'm just too lazy to learn gdb). Here are some libraries/classes I've found useful lately:

• ParamContainer, a class that's basically a wrapper around a program's command line parameters. It makes it pretty simple to define, parse, and retrieve command line options, including short and long keywords, required and optional parameters, and parameters with and without arguments. All with just a few lines of code.
• TinyXml, a set of classes for parsing XML files. I've decided that XML is the best way to go for producing files to contain simulation parameters, outputs, etc. I can incorporate documentation into the XML files, both by the very nature of the XML and by adding comments. It's very easy to format output as simple XML. TinyXml makes it straightforward to pull even fairly complex structured data from XML (you may sense a common theme here).
• In a related note, I've hacked together a set of classes that "automatically" (with the addition of only two lines of code in each .cpp file) collect RCS version information (or any arbitrary information that can be written in a string) for each source file used to build a program and will format it as XML comments for output. I now write these comments into every simulation output file I produce, so I know exactly what code was used to produce the results (assuming I bother to check in the files after each change is debugged). This is a pale shadow of my Logos system (which doesn't exist right now). For example, the beginning of an output file might look like:

   
  <?xml version="1.0" standalone=no?> 
  <!-- State output file --> 
  <!-- This can be used as input; the "Statistics" element --> 
  <!-- is ignored when the state is read by the simulator (but --> 
  <!-- can be read by functions written for MATLAB) --> 
  <!-- $Id: SourceVersions.cpp,v 1.2 2005/02/18 13:41:04 stiber Exp stiber $ --> 
  <!-- $Id: KIIgrowth-prog.cpp,v 1.5 2005/02/18 20:45:20 stiber Exp stiber $ --> 
  <!-- $Id: KIIgrowth-simulator.cpp,v 1.7 2005/02/18 20:45:54 stiber Exp stiber $ --> 
  <!-- $Id: InputGenerator.cpp,v 1.1 2005/02/18 20:46:58 stiber Exp $ --> 
  <!-- $Id: Matrix.cpp,v 1.3 2005/02/18 13:40:02 stiber Exp stiber $ --> 
  <!-- $Id: VectorMatrix.cpp,v 1.4 2005/02/18 13:41:42 stiber Exp stiber $ --> 
  <!-- $Id: CompleteMatrix.cpp,v 1.3 2005/02/18 13:38:53 stiber Exp stiber $ --> 
  <!-- $Id: SparseMatrix.cpp,v 1.3 2005/02/18 13:41:35 stiber Exp stiber $ --> 
  <!-- $Id: MatrixFactory.cpp,v 1.3 2005/02/18 13:40:11 stiber Exp stiber $ --> 
  <!-- $Id: RNG.cpp,v 1.2 2005/02/18 13:40:44 stiber Exp stiber $ --> 
  <!-- $Id: norm.cpp,v 1.2 2005/02/18 13:41:51 stiber Exp stiber $ --> 
  <!-- $Id: DistanceList.cpp,v 1.4 2005/02/18 19:39:01 stiber Exp $ --> 
  <SimState> 
     <!-- Mitral unit states --> 
     <Layer name="mitral"> 
        <StateQueue name="x" stages="3"> 
           <Matrix type="complete" rows="1" columns="25" 
                    multiplier="1.0"> 
     0.968226 1.10523 1.28773 0.684952 0.401885 0.344824 -1.56298 0.989861 
  

  and so on.

[+/-] show/hide the rest of this post