The stats backend (coNCePTuaL User’s Guide)

The stats backend outputs various statistics about a coNCePTuaL program. It can be used to help verify a program’s correctness or merely to search for interesting patterns within a complex communication pattern. The following is some sample output from a program compiled with stats:

Execution parameters
--------------------
  Number of processors:          16
  Random-number seed:   -1727114895
  Command line:         ncptl --backend=stats hycom.ncptl --xdim=4 --ydim=4 --tasks=16 --iter=10
  Timestamp:            Wed Jan  4 17:32:04 2006

Message traffic
---------------
  Total messages sent:           710
  Total bytes sent:          1454080
  Unique message sizes sent: 2048

Per-processor message traffic
-----------------------------
  Processors sending a total of 61440 bytes:    0
  Processors sending a total of 81920 bytes:    1-3, 12-15
  Processors sending a total of 102400 bytes:   4-11
  Processors receiving a total of 61440 bytes:  1-3, 13-15
  Processors receiving a total of 81920 bytes:  5-7, 9-11
  Processors receiving a total of 122880 bytes: 12
  Processors receiving a total of 143360 bytes: 4, 8
  Processors receiving a total of 184320 bytes: 0
  Processors sending a total of 30 messages:    0
  Processors sending a total of 40 messages:    1-3, 12-15
  Processors sending a total of 50 messages:    4-11
  Processors receiving a total of 30 messages:  1-3, 13-15
  Processors receiving a total of 40 messages:  5-7, 9-11
  Processors receiving a total of 60 messages:  12
  Processors receiving a total of 70 messages:  4, 8
  Processors receiving a total of 90 messages:  0

Processor SEND-event peers
--------------------------
  Processors posting SEND events to offsets {-12, -4, +1, +3}:    12
  Processors posting SEND events to offsets {-8, -4, +1, +3, +4}: 8
  Processors posting SEND events to offsets {-4, -3, -1}:         15
  Processors posting SEND events to offsets {-4, -3, -1, +4}:     7, 11
  Processors posting SEND events to offsets {-4, -2, -1, +1}:     14
  Processors posting SEND events to offsets {-4, -2, -1, +1, +4}: 6, 10
  Processors posting SEND events to offsets {-4, -1, +1}:         13
  Processors posting SEND events to offsets {-4, -1, +1, +4}:     5, 9
  Processors posting SEND events to offsets {-4, +1, +3, +4}:     4
  Processors posting SEND events to offsets {-3, -1, +4}:         3
  Processors posting SEND events to offsets {-2, -1, +1, +4}:     2
  Processors posting SEND events to offsets {-1, +1, +4}:         1
  Processors posting SEND events to offsets {+1, +3, +4}:         0

Network bisection crossings
---------------------------
  Bisection messages:    100
  Bisection bytes:    204800

Event tallies
-------------
  Total number of LOG events:       10
  Total number of NEWSTMT events:   48
  Total number of RECEIVE events:  710
  Total number of RESET events:     10
  Total number of SEND events:     710
  Total number of WAIT_ALL events: 480

Per-processor event sets
------------------------
  Processors executing only {LOG, NEWSTMT, RECEIVE, RESET, SEND, WAIT_ALL}: 0
  Processors executing only {NEWSTMT, RECEIVE, SEND, WAIT_ALL}:             1-15

Per-processor event tallies
---------------------------
  Processors executing 10 LOG events:      0
  Processors executing 3 NEWSTMT events:   0-15
  Processors executing 30 RECEIVE events:  1-3, 13-15
  Processors executing 40 RECEIVE events:  5-7, 9-11
  Processors executing 60 RECEIVE events:  12
  Processors executing 70 RECEIVE events:  4, 8
  Processors executing 90 RECEIVE events:  0
  Processors executing 10 RESET events:    0
  Processors executing 30 SEND events:     0
  Processors executing 40 SEND events:     1-3, 12-15
  Processors executing 50 SEND events:     4-11
  Processors executing 30 WAIT_ALL events: 0-15

stats is derived from interpret (see The interpret backend) and therefore supports the interpret backend’s options as well as the standard options described in Running coNCePTuaL programs. However, because stats does not produce log files, the --logfile option is absent. stats additionally supports the following three command-line options:

The --expand-lists option tells the stats backend whether it should output sets of numbers as comma-separated ranges (e.g., ‘1-3, 12-15’) or as individual numbers (e.g., ‘1, 2, 3, 12, 13, 14, 15’). The --format option specifies the output format. By default, it outputs human-readable text, as shown in the above example. However, --format=excelcsv outputs the data in comma-separated value format suitable for loading directly into Microsoft Excel. A more general form of computer-parseable output is specified with the ‘sep’ sub-option, which takes a separator string as a sub-option. For example, --format=sep:"#" produces output like the following:

The difference between --format=excelcsv and --format=sep:"," is that the former employs a number of bizarre special cases when quoting strings so as to coerce Excel into properly processing the file. The latter simply inserts a comma between columns with no special string processing other than preceding the characters ‘"’ and ‘\’ with ‘\’.

The stats backend issues a ‘Column separator sep appears within a field’ warning if the separator string appears in any of the output fields. (Note that it will always appear in the ‘Command line’ line because the --format line is specified on the command line.) The idea is to warn the user that automatic parsing of the output (e.g., using awk) may need to be careful when processing lines containing such fields.

As can be seen from the sample output, stats outputs a lot of information. The --exclude option—which can be specified repeatedly on the command line—provides the user with fine-grained control over which output to suppress. For example, --exclude="Total number of NEWSTMT events" tells stats not to output the line with that key. Similarly, --exclude="Processor SEND-event peers" eliminates an entire category of information.

3.3.7 The stats backend

3.3.7 The `stats` backend