3.4 Running coNCePTuaL programs

coNCePTuaL programs can be run like any other program built with the same compiler and communication library. For example if a program myprog was built with coNCePTuaL’s C+ MPI backend, the program might be run with a command like mpirun -np nodes myprog or prun -Nnodes myprog or pdsh -w node_list myprog. The important point is that job launching is external to coNCePTuaL. A coNCePTuaL program is oblivious to whether it is being run with a single thread on each multiprocessor node or with one thread on each CPU, for example. However, coNCePTuaL log files do include the host name in the prologue comments (see Log-file format) so job-launching parameters can potentially be inferred from those.

coNCePTuaL programs automatically support a “help” option. This is usually specified as --help or -?, depending on which option-parsing library configure configured in. (See configure.) The output of running myprog --help most likely looks something like this:

Usage: myprog [OPTION...]
  -C, --comment=<string>      Additional commentary to write to the log
                              file, @FILE to import commentary from FILE,
                              or !COMMAND to import commentary from COMMAND
                              (may be specified repeatedly)
  -L, --logfile=<string>      Log-file template [default: "a.out-%p.log"]
  -N, --no-trap=<string>      List of signals that should not be trapped
                              [default: ""]
  -S, --seed=<number>         Seed for the random-number generator
                              [default: 0]

Help options:
  -?, --help                  Show this help message
  --usage                     Display brief usage message

Although a coNCePTuaL program can specify its own command-line options (see Command-line arguments), a few are provided by default. In addition to --help these include --comment, --logfile, --no-trap, and --seed.

--comment

--comment makes it possible to add arbitrary commentary to a log file. This is useful for incorporating information that coNCePTuaL would be unable to (or simply does not currently) determine on its own, for example, --comment="Last experiment before upgrading the network device driver". Two special cases are supported:

1. If the comment string begins with ‘@’ then the remainder of the string is treated as a filename. Each line of the corresponding file is treated as a separate comment string. Hence, if the file sysdesc.txt contains the lines ‘Using FooBarNet’ and ‘Quux is enabled’, then specifying --comment= sysdesc.txt is similar to specifying both --comment="Using FooBarNet" and --comment="Quux is enabled".
2. If the comment string begins with ‘!’ then the remainder of the string is treated as a shell command. The command is executed and each line of its output is treated as a separate comment string. For example, --comment='!lspci | grep -i net' executes lspci, extracts only those lines containing the string ‘net’, and makes log-file comments out of the result. Note that --comment= '!command' differs from --comment= "`command`" in that the former causes command to be executed individually by each process in the program while the latter executes command only once and only before launching the program. Also note that ‘!’ must be escaped in csh and derivitive shells (i.e., --comment= '\!command').

As an example, the command line arguments --comment="This is a simple comment." --comment=!lspci --comment=@/proc/version --comment="This is another simple comment." produce log-file lines like the following:

# User comment 1: This is a simple comment.
# Output of 'lspci', line 1: 00:00.0 Host bridge: Intel Corp. 82860 860 (Wombat) Chipset Host Bridge (MCH) (rev 04)
# Output of 'lspci', line 2: 00:01.0 PCI bridge: Intel Corp. 82850 850 (Tehama) Chipset AGP Bridge (rev 04)
# Output of 'lspci', line 3: 00:02.0 PCI bridge: Intel Corp. 82860 860 (Wombat) Chipset AGP Bridge (rev 04)
# Output of 'lspci', line 4: 00:1e.0 PCI bridge: Intel Corp. 82801BA/CA/DB PCI Bridge (rev 04)
# Output of 'lspci', line 5: 00:1f.0 ISA bridge: Intel Corp. 82801BA ISA Bridge (LPC) (rev 04)
# Output of 'lspci', line 6: 00:1f.1 IDE interface: Intel Corp. 82801BA IDE U100 (rev 04)
# Output of 'lspci', line 7: 00:1f.2 USB Controller: Intel Corp. 82801BA/BAM USB (Hub  (rev 04)
# Output of 'lspci', line 8: 00:1f.3 SMBus: Intel Corp. 82801BA/BAM SMBus (rev 04)
# Output of 'lspci', line 9: 00:1f.4 USB Controller: Intel Corp. 82801BA/BAM USB (Hub  (rev 04)
# Output of 'lspci', line 10: 00:1f.5 Multimedia audio controller: Intel Corp. 82801BA/BAM AC'97 Audio (rev 04)
# Output of 'lspci', line 11: 01:00.0 VGA compatible controller: nVidia Corporation NV11 [GeForce2 MXR] (rev b2)
# Output of 'lspci', line 12: 02:1f.0 PCI bridge: Intel Corp. 82806AA PCI64 Hub PCI Bridge (rev 03)
# Output of 'lspci', line 13: 03:00.0 PIC: Intel Corp. 82806AA PCI64 Hub Advanced Programmable Interrupt Controller (rev 01)
# Output of 'lspci', line 14: 04:0b.0 Ethernet controller: 3Com Corporation 3c905C-TX/TX-M [Tornado] (rev 78)
# Output of 'lspci', line 15: 04:0d.0 Multimedia audio controller: Creative Labs SB Live! EMU10k1 (rev 08)
# Output of 'lspci', line 16: 04:0d.1 Input device controller: Creative Labs SB Live! MIDI/Game Port (rev 08)
# Contents of /proc/version, line 1: Linux version 2.4.20-28.7 (bhcompile@porky.devel.redhat.com) (gcc version 2.96 20000731 (Red Hat Linux 7.3 2.96-126)) #1 Thu Dec 18 11:31:59 EST 2003
# User comment 2: This is another simple comment.

To facilitate log-file parsing, all colons in the name of an ‘@’-file or ‘!’-command are written as periods. This affects only the display, not the file to read or command to execute.

Be careful when using shell backquotes with --comment (e.g., as in --comment="`who`"). Different shells have different ways of handling newlines output by a backquoted command. In some shells (e.g.,  bash), the coNCePTuaL program sees the entire output with embedded newlines as a single argument; in others (e.g.,  tcsh), each line of output constitutes a separate command-line argument. Because coNCePTuaL programs currently ignore arguments that do not begin with ‘--’, the comment written to the log file in the latter case terminates at the first newline. In virtually all shells, the double quotes around the backquoted command are needed to prevent the shell from splitting arguments at word boundaries. As a consequence, --comment=`who` logs only the first word output by the who command.

--logfile

--logfile specifies a template for naming log files. Each task maintains a log file based on the template name but with ‘%p’ replaced with the processor number, ‘%r’ replaced with the run number (the smallest nonnegative integer that produces a filename that does not already exist), and ‘%%’ replaced with a literal “%” character. The program outputs an error message and aborts if the log-file template does not contain at least one ‘%p’. The only exception is that an empty template (i.e., --logfile="") inhibits the production of log files entirely.

Like C’s printf() function, a numeric field width can occur between the ‘%’ and the conversion specifier (the ‘p’ or ‘r’ in the case of --logfile). The field is padded on the left with spaces to the given width. More practically, if the field width begins with the number ‘0’ the field is padded on the left with zeroes to the given width. For example, specifying --logfile="mydata-%03p.log" on the command line produces log files named mydata-000.log, mydata-001.log, mydata-002.log, mydata-003.log, and so forth.

--no-trap

--no-trap specifies a list of signals or ranges of signals that should not be trapped. For example, --no-trap= 10-12,17 prevents signals 10, 11, 12, and 17 from being trapped.⁸ Signals can also be referred to by name, with or without a ‘SIG’ prefix. Also, names and numbers can be freely mixed. Hence, --no-trap= 10-12,INT,17,SIGSTOP,SIGCONT is a valid argument to a coNCePTuaL program. Because signal reception can adversely affect performance, coNCePTuaL’s default behavior is to terminate the program on receipt of a signal. However, some signals may be necessary for the underlying communication layer’s proper operation. --no-trap enables such signals to pass through coNCePTuaL untouched. (Some signals, however, are needed by coNCePTuaL or by a particular backend and are always trapped.)

--seed

--seed (which selects a different default value on each run) is used in any program that utilizes the RANDOM TASK construct (see Binding variables) or that sends message WITH VERIFICATION (see Message specifications).

If the LOGS statement (see Writing to a log file) is used anywhere in a coNCePTuaL program, then all processes write a log file. This is done because coNCePTuaL log files—even those that contain no measurement data—include a wealth of important information in prologue comments, as described Interpreting coNCePTuaL log files. As a consequence, a program run with thousands of processes produces thousands of log files. If process 0 is the only process that logs actual data, the ncptl-logmerge script (see ncptl-logmerge) can merge these log files into a single, more manageable, file. For situations in which it is unreasonable for every process to write a log file (e.g., if the filesystem is unable to handle large numbers of simultaneous file creations, the NCPTL_LOG_ONLY environment variable lets the user limit the set of processes that produce log files. NCPTL_LOG_ONLY accepts a comma-separated list of dash-separated process ranges such as ‘0-3,12-16,24,25,32-48’. Only processes included in the list produce log files.