Simulation::Automate - A Simulation Automation Tool
The set of modules is called Simulation::Automate.
The tool itself is called SynSim, the command synsim.
use Simulation::Automate;
&synsim();
SynSim is a generic template-driven simulation automation tool. It works with any simulator that accepts text input files and generates text output (and even those that don't. See EXAMPLES for special cases). It executes thousands of simulations with different input files automatically, and processes the results. Postprocessing facilities include basic statistical analysis and automatic generation of PostScript plots with Gnuplot. SynSim is entirely modular, making it easy to add your own analysis and postprocessing routines.
tar -xvf Simulation-Automate-0.9.4.tar.gz
cd Simulation-Automate-0.9.4 perl Makefile.PL
make
make test
make install
make localinstall
or
perl -e "use Simulation::Automate;&Simulation::Automate::localinstall();"
make setup or
perl -e "use Simulation::Automate;&Simulation::Automate::setup();"
The archive structure is as follows:
README
Makefile.PL
Automate.pm
Automate/
Main.pm
PostProcLib.pm
Analysis.pm
Dictionary.pm
PostProcessors.pm
eg/ synsim synsim.data ErrorFlags.data Histogram.data SweepVar.data Expressions.data gnuplot.data SOURCES/ bufsim3.cc MersenneTwister.h TEMPLATES/ DEVTYPES/ SIMTYPES/ bufsim3.templ
SynSim must be configured for use with your simulator. This is done by providing template and source files, creating (or modifying) datafiles and (optionally) customizing some modules for postprocessing the simulation results. All files must be put in a particilar directory structure:
You can use "make setup" to create a SynSim directory structure. If you want to create it manually, this is the structure:
YourProject/ synsim YourDataFile.data [SOURCES/] TEMPLATES/ [DEVTYPES/] SIMTYPES/ YourSimTempl.templ
[Simulation/SynSim/] [Dictionary.pm] [PostProcessors.pm]
The synsim script contains the 2 lines from the SYNOPSIS. The local Simulation/Automate modules are only required if you want to customize the postprocessing (highly recommended). =head2 Source files
Copy all files which are needed "read-only" by your simulator (e.g. header files, library files) to SOURCES/. This directory is optional.
Template files are files in which simulation variables will be substituted by their values to create the input file for your simulator. SynSim can create an input file by combining two different template files, generally called device templates and simulation templates. This is useful in case you want to run different types of simulations on different devices, e.g. DC analysis, transient simulations, small-signal and noise analysis on 4 different types of operation amplifiers. In total, this requires 16 different input files, but only 8 different template files (4 for the simulation type, 4 for the device types).
Put the template files in TEMPLATES/SIMTYPES and TEMPLATES/DEVTYPES.
There must be at least one template file in SIMTYPES; files in DEVTYPES are optional. SynSim will check both directories for files as defined in the datafile. If a matching file is found in DEVTYPES, it will be prepended to the simulation template from SIMTYPES. This is useful if the datafile defines multiple simulation types on a particular device (See DATAFILE DESCRIPTION for more information).
NOTE:
SynSim creates a run directory ath the same level as the SOURCES and TEMPLATES directories. All commands (compilations etc.) are executed in that directory. As a consequence, paths to source files (e.g. header files) should be "../SOURCES/sourcefilename".
The datafile is the input file for synsim. It contains the list of simulation variables and their values to be substituted in the template files, as well as a number of configuration variables (See DATAFILE DESCRIPTION for more information).
The PostProcessing.pm module contains routines to perform postprocessing on the simulation results. A number of generic routines are provided, as well as a library of functions to make it easier to develop your own postprocessing routines. See POSTPROCESSING for a full description).
The Dictionary.pm module contains descriptions of the parameters used in the simulation. These descriptions are used by the postprocessing routines to make the simulation results more readable. See DICTIONARY for a full description).
The datafile defines which simulations to run, with which parameter values to use, and how to run the simulation. By convention, it has the extension .data.
The datafile is a case-sensitive text file with following syntax:
The main purpose of the datafile is to provide a list of all variables and their values to be substituted in the template files. The lists of values for the variables can be used in two different ways:
A simulation will be performed for every possible combination of the values for all parameters.
Example:
_PAR1 = 1,2 _PAR2 = 3,4,5
defines 6 simulations: (_PAR1,_PAR2)=(1,3),(1,4),(1,5),(2,3),(2,4),(2,5)
Simulation results for all values in ','-separated list are stored in a separate files.
If more than one ';'-separated list exists, they must have the same number of items. The values of all parameters at the same position in the list will be used.
Example:
_PAR1 = 0;1;2 _PAR2 = 3;4;5
defines 3 simulations: (_PAR1,_PAR2)=(0,3);(1,4);(2,5)
Values from ';'-separated lists are processed one after another while are values for all others parameters are kept constant. In other words, the ';'-separated list is the innermost of all nested loops.
Simulation results for all values in the ';'-separated list are stored in a common file. For this reason, ';'-separated lists are preferred as sweep variables (X-axis values), whereas ','-separated lists are more suited for parameters (sets of curves).
Example: consider simulation of packet loss vs number of buffers with 3 types of buffer and 2 different traffic distributions.
_NBUFS = 4;8;16;32;64;128 _BUFTYPE = 1,2,3 _TRAFDIST = 1,2
This will produces 6 files, each file containing the simulation results for all values of _NBUFS. A plot of this simulation would show a set of 6 curves, with _NBUFS as X-axis variable.
A number of variables are provided to configure SynSim's behaviour:
The SynSim datafile has support for expressions, i.e. it is possible to express the value list of a variable in terms of the values of other variables.
Example:
# average packet length for IP dist
_MEANPL = ((_AGGREGATE==0)?2784:9120)
# average gap width
_MEANGW= int(_MEANPL*(1/_LOAD-1))
# average load
_LOAD = 0.1;0.2;0.3;0.4;0.5;0.6;0.7;0.8;0.9
# aggregate
_AGGREGATE = 0,12000
The variables used in the expressions must be defined in the datafile, although not upfront. Using circular references will not work. The expression syntax is Perl syntax, so any Perl function can be used. Due to the binding rules, it is necessary to enclose expressions using the ternary operator ?: with brackets (see example).
The next sections (DICTIONARY and POSTPROCESSING) are optional. For instructions on how to run SynSim, go to RUNNING SYNSIM.
The Dictionary.pm module contains descriptions of the parameters used in the simulation. These descriptions are used by the postprocessing routines to make the simulation results more readable. The dictionary is stored in an associative array called make_nice. The description of the variable is stored in a field called 'title'; Descriptions of values are stored in fields indexed by the values.
Following example illustrates the syntax:
# Translate the parameter names and values into something meaningful
%Dictionary::make_nice=(
_BUFTYPE => {
title=>'Buffer type',
0=>'Adjustable',
1=>'Fixed-length',
2=>'Multi-exit',
},
_YOURVAR1 => {
title=>'Your description for variable 1',
},
_YOURVAR2 => {
title=>'Your description for variable 2',
'val1' => 'First value of _YOURVAR2',
'val3' => 'Second value of _YOURVAR2',
},
);
Postprocessing of the simulation results is handled by routines in the PostProcessors.pm module. This module uses the PostProcLib.pm and Analysis.pm.
Routines to perform analysis on the simulation results in the PostProcessors module. In general you will have to create your own routines, but the version of PostProcessors.pm in the distribution contains a number of more or less generic postprocessing routines:
SWEEPVAR as X-axis and all other variables as paramters. This routine is completely generic. The SWEEPVAR value list must be semicolon-separated.
Creates a plot using SWEEPVAR as X-axis and all other variables as paramters. Calculates average and 95% confidence intervals for _POP_SAMPLE samples and plots error flags. This routine is almost fully generic, apart from the 95% interval which is fixed for now. See eg/ErrorFlags.data for an example datafile. The SWEEPVAR value list must be comma-separated (!). The number of samples of the population _POP_SAMPLE is a simulation variable rather than a configuration variable. As a consequence, it must be specified as a list, e.g.
_POP_SAMPLE = 1..20
and not
_POP_SAMPLE = 20
Creates a histogram of the simulation results. This requires the simulator to produce raw data for the histograms in a tabular format. The configuration variable OUTPUT_FILTER_PATTERN can be used to 'grep' the simulator output. When specifying logscale X or XY for the plot, the histogram bins will be logarithmic. See eg/Histogram.data for an example.
The number of bins in the histogram must be specified via SWEEPVAR, e.g.
SWEEPVAR : _NBINS
_NBINS = 20
In a lot of cases you will want to create your own postprocessing routines. To make this easier, a library of functions is at your disposal. This library resides in the PostProcLib.pm module.
Following functions are exported:
&prepare_plot # does what it says. see example below
&gnuplot # idem. Just pipes the first argument string to gnuplot. The option -persist can be added to keep the plot window after gnuplot exits.
&gnuplot_combined # See example, most useful to create nice plots. Looks for all files matching ${simtempl}-${anatempl}-*.res, and creates a line in the gnuplot script based on a template you provide.
Following variables are exported:
%simdata # contains all simulation variables and their value lists
@results # an array of all results for a sweep (i.e. a var with a ';'-sep. value list
$sweepvar # SWEEPVAR
$normvar # NORMVAR
$sweepvals # string containing all names and values of parameters for the sweep, joined with '-'
$datacol # DATACOL
$count # cfr. OUTPUT FILES section
$simtempl # SIMTYPE
$anatempl # ANALYSIS_TEMPLATE
$dirname # name of run directory. cfr. OUTPUT FILES section
$last # indicates end of a sweep
$verylast # indicates end of all simulations
$sweepvartitle # title for SWEEPVAR (from Dictionary.pm)
$title # TITLE
$legend # plot legend (uses Dictionary.pm)
$legendtitle # plot legend title (uses Dictionary.pm)
$xlabel # XLABEL
$ylabel # YLABEL
$logscale # LOGSCALE
$plot # corresponds to -p flag
$interactive # corresponds to -i flag
An example of how all this is used:
sub YourRoutine {
## Unless you want to dig really deep into the code, start all your routines like this:
## Get all arguments, to whit: $datafilename,$count,$dataref,$flagsref,$returnvalue
my @args=@_;
## But don't bother with these, following function does all the processing for you:
&prepare_plot(@args);
## this makes all above-listed variables available
## Define your own variables.
## As every variable can have a list of values,
## $simdata{'_YOURVAR1'} is an array reference.
my $yourvar=${$simdata{'_YOURVAR1'}}[0];
my @sweepvarvals=@{$simdata{$sweepvar}};
## $verylast indicates the end of all simulations
if($verylast==0) {
## what to do for all simulations
## example: parse SynSim .res file and put into final files for gnuplot
open(HEAD,">${simtempl}-${anatempl}-${sweepvals}.res");
open(IN,"<${simtempl}_C$count.res");
while(<IN>) {
/\#/ && !/Parameters|$sweepvar/ && do {
## do something with $_
print HEAD $_
};
}
close IN;
close HEAD;
my $i=0;
foreach my $sweepvarval ( @sweepvarvals ) {
open(RES,">>${simtempl}-${anatempl}-${sweepvals}.res");
print RES "$sweepvarval\t$results[$i]";
close RES;
$i++;
}
## $last indicates the end of a sweep
if($last) {
## $interactive corresponds to the -i flag
if($interactive) {
## do something, typically plot intermediate results
my $gnuplotscript=<<"ENDS";
# your gnuplot script here
ENDS
&gnuplot($gnuplotscript);
} # if interactive
} # if last
} else {
## On the very last run, collect the results into one nice plot
## You must provide a template line for gnuplot. Next line is a good working example.
## This line will be eval()'ed by the &gnuplot_combined() routine.
## This means the variables $filename and $legend are defined in the scope of this routine.
## Don't locally scoped put variables in there, use the substitution trick as below or some other way.
#this is very critical. The quotes really matter!
# as a rule, quotes inside gnuplot commands must be escaped
my $plotlinetempl=q["\'$filename\' using (\$1*1):(\$_DATACOL) title \"$legend\" with lines"];
$plotlinetempl=~s/_DATACOL/$datacol/; ##this is a trick, you might try to eval() the previous line or something. TIMTOWDI :-)
my $firstplotline=<<"ENDH";
# header for your gnuplot script here
ENDH
&gnuplot_combined($firstplotline,$plotlinetempl);
}
} #END of YourRoutine()
A module for basic statistical analysis is also available (Analysis.pm). Currently, the module provides 2 routines:
To calculate average, standard deviation, min. and max. of a set of values.
Arguments:
$file: name of the results file. The routine requires the data to be in whitespace-separated columns.
$par: Determines if the data will be differentiated before processing ($par='DIFF') or not (any other value for $par). Differentiation is defined as subtracting the previous value in the array form the current value. A '0' is prepended to the array to avoid an undefined first point.
$datacol: column to use for data
$title: optional, a title for the histogram
$log: optional, log of values before calculating histogram or not ('LOG' or '')
Use: my $file="your_results_file.res"; my $par='YOURPAR'; my $datacol=2; my %stats=%{&calc_statistics($file,[$par, $datacol])};
my $avg=$stats{$par}{AVG}; # average
my $stdev=$stats{$par}{STDEV}; # standard deviation
my $min=$stats{$par}{MIN}; # min. value in set
my $max=$stats{$par}{MAX}; # max. value in set
To build histograms. There are 3 extra arguments:
$nbins: number of bins in the histogram $min: force the value of the smallest bin (optional) $max: force the value of the largest bin (optional)
use: my $par='DATA'; my %hists=%{&build_histograms("your_results_file.res",[$par,$datacol],$title,$log,$nbins,$min,$max)};
NOTE: Because the extra arguments are last, the $title and $log arguments can not be omitted. If not needed, supply ''.
The SynSim script must be executed in a subdirectory of the SynSim directory which contains the TEMPLATES subdir and the datafile (like the Example directory in the distribution).
The command line is as follows:
./synsim [-h -i -p -w -v -N -f datafile]
The synsim script supports following command line options:
none: defaults to -f synsim.data -f [filename]: 'file input'. Expects a file containing info about simulation and device type. -p : plot. This enables generation of postscript plots via gnuplot. A postprocessing routine is required to generate the plots. -i : interactive. Enables generation of a plot on the screen after every iteration. Assumes -p. A postprocessing routine is required to generate the plots. -v : 'verbose'. Sends simulator output to STDOUT, otherwise to the [rundir]/simlog file -w : 'warn'. Show warnings about undefined variables -N : 'No simulations'. Perform only postprocessing. -h, -? : short help message
SynSim creates a run directory {SIMTYPE}-[datafile without .data]. It copies all necessary template files and source files to this directory; all output files are generated in this directory.
SynSim generates following files:
Output files for all simulation runs.
The names of these files are are {SIMTYPE}_C[counter]_[simulation number].out
counter is increased with every new combination of variables in ','-separated lists
simulation number is the position of the value in the ';'-separated list.
Combined output file for all values in a ';'-separated list.
The names of these files are are {SIMTYPE}_C[counter]_.out
counter is increased with every new combination of variables in ','-separated lists.
Only the lines matching /OUTPUT_FILTER_PATTERN/ (treated as a Perl regular expression) are put in this file.
Combined output file for all values in a ';'-separated list, with a header detailing all values for all variables.
The names of these files are are {SIMTYPE}_C[counter].res,
counter is increased with every new combination of variables in ','-separated lists.
Only the lines in the .out files matching /OUTPUT_FILTER_PATTERN/ (treated as a Perl regular expression) are put in this file.
Separate input files for every item in a ';'-separated list.
The names of these files are are {SIMTYPE}_[simulation number].{EXT}
simulation number is the position of the value in the list.
These files are overwritten for every combination of variables in ','-separated lists.
Here are some examples of how to use SynSim for different types of simulators.
Normal use: spice -b circuit.sp > circuit.out
With SynSim:
Copy circuit.sp to TEMPLATES/SIMTYPE/circuit.templ Replace all variable values with SynSim variable names.
e.g. a MOS device line in SPICE:
M1 VD VG VS VB nch w=10u l=10u
becomes
M1 VD VG VS VB _MODEL w=_WIDTH l=_LENGTH
TITLE: MOS drain current vs. length SIMTYPE : circuit COMMAND : spice -b inputfile > outputfile
# Required for postprocessing OUTPUT_FILTER_PATTERN : id # keep only the drain current on the output file ANALYSIS_TEMPLATE : SweepVar # default template for simple sweep SWEEPVAR : _L # we sweep the length, the other variables are parameters DATACOL: 2 # first col is the name
_L = 1u;2u;5u;10u;20u;50u _W = 10u,100u _MODEL = nch
There are more possible keywords, cf. DATAFILE DESCRIPTION.
./synsim -p -i -v -f IDvsL.data
-p to create plots -i means interactive, so the plots are displayed during simulation -v for verbose output -f because the filename is not the default name
SynSim will run 12 SPICE simulations and produce 1 plot with all results.
All results are stored in the run directory, in this case:
circuit-IDvsL
Normal use: simplesim -a50 -b100 -c0.7
Output is saved in out.txt.
With SynSim:
As simplesim does not take an input file, we create a wrapper simplesim.templ in TEMPLATES/SIMTYPE. This file is actually a template for a simple perl script:
system("simplesim -a_VAR1 -b_VAR2 -c_VAR3");
system("cp out.txt $ARGV[0]");
TITLE: simplesim test SIMTYPE : simplesim COMMAND : perl inputfile outputfile
./synsim -f test.data
SynSim will run without any messages and produce no plots.
All results are stored in the run directory, in this case:
simplesim-test
Normal use: Modify values for #if and #ifdef constants in the header file; then compile and run.
e.g.:
vi bufsim3.h g++ -o bufsim3 bufsim3.cc ./bufsim3 > outputfile
With SynSim:
As bufsim3 does not take an input file, we create a wrapper bufsim3.templ in TEMPLATES/SIMTYPE. This file is actually a template for a perl script that writes the header file, compiles and runs the code:
open(HEADER,">bufsim3.h"); print HEADER <<"ENDH"; #define NBUFS _NBUFS #define NPACKETS _NPACK #AGGREGATE _AGGREGATE ENDH close HEADER;
system("g++ -o bufsim3 bufsim3.cc");
system("./bufsim3 $ARGV[0]");
TITLE: bufsim3 test (_NBUFS, _NPACK) # will be substituted by the values SIMTYPE : bufsim3 COMMAND : perl inputfile outputfile
./synsim -w -v -f Aggregate.data
SynSim will run verbose and flag all variables not defined in the datafile.
All results are stored in the run directory, in this case:
bufsim3-Aggregate
Normal use: spectre circuit.scs -raw circuit.raw
With SynSim:
The .raw file is a binary file, so it should not be touched. SynSim creates output files with extension .out, and combines these with the headers etc. (cf. OUTPUT FILES). By keeping the extension .raw, the simulator output files will not be touched.
In the datafile:
TITLE: Spectre simulation with SPF output EXT: .scs COMMAND: spectre inputfile -raw outputfile.raw > outputfile
outputfile, but not outputfile.raw.
This module is still Alpha, a lot of work remains to be done to make it more user-friendly. The main tasks:
Wim Vanderbauwhede <wim\x40motherearth.org>
Copyright (c) 2000,2002-2003 Wim Vanderbauwhede. All rights reserved. This program is free software; you can redistribute it and/or modify it under the same terms as Perl itself.