ncptl-logmerge manual page

NAME

ncptl-logmerge - Merge coNCePTuaL log files

SYNOPSIS

ncptl-logmerge --usage | --help | --man

ncptl-logmerge [ --output=filename] [ --simplify] filename...

DESCRIPTION

A coNCePTuaL program produces one log file per process. For large numbers of processes the result can be unwieldy. ncptl-logmerge combines a large set of log files into a single, merged file which can later be expanded back into its constituent log files. There are a number of restrictions on the input to ncptl-logmerge; see the Restrictions section for details.

The merged output file does not modify lines which are identical in all of the input files. Lines which do differ across input files are prefixed with the processors and processor ranges in which they appeared.

As an example, the following text was extracted from a set of 186 coNCePTuaL log files (from a 186-processor run):

# Microsecond timer type: PAPI_get_real_usec()
    # Average microsecond timer overhead: <1 microsecond
    #[0-4,6-12,14-16,18-52,54-78,80-94,96-101,103-121,123-140,142-169,
      171-185]# Microsecond timer increment: 1 +/- 0 microseconds
      (ideal: 1 +/- 0)
    #[5]# Microsecond timer increment: 1.00229 +/- 0.15854
      microseconds (ideal: 1 +/- 0)
    #[13]# Microsecond timer increment: 1.00228 +/- 0.158442
      microseconds (ideal: 1 +/- 0)
    #[17,79]# Microsecond timer increment: 1.00228 +/- 0.158392
      microseconds (ideal: 1 +/- 0)
    #[53]# Microsecond timer increment: 1.00228 +/- 0.158409
      microseconds (ideal: 1 +/- 0)
    #[102]# Microsecond timer increment: 1.00228 +/- 0.158458
      microseconds (ideal: 1 +/- 0)
    #[95,122]# Microsecond timer increment: 1.00228 +/- 0.158474
      microseconds (ideal: 1 +/- 0)
    #[141]# Microsecond timer increment: 1.00228 +/- 0.158491
      microseconds (ideal: 1 +/- 0)
    #[170]# Microsecond timer increment: 1.00228 +/- 0.158524
      microseconds (ideal: 1 +/- 0)

All of the input files contained the same Microsecond timer type and Average microsecond timer overhead lines. However, the measured Microsecond timer increment varied across input files. While many of the processors observed an increment of 1 +/- 0, processor 5 was alone in observing 1.00229 +/- 0.15854; processor 13 was alone in observing 1.00228 +/- 0.158442; and, both processor 17 and processor 79 observed 1.00228 +/- 0.158392 as the timer increment.

ncptl-logmerge can also be instructed to output only the lines which differ across files. Common lines are not output. This feature is useful for discovering misconfigured nodes in a large computer system. For example, on one computer system on which coNCePTuaL was run, five processors were running at a higher clock rate than the remainder which naturally affected performance. ncptl-logmerge can be used to help identify such outliers.

OPTIONS

ncptl-logmerge accepts the following command-line options:

-u

Output the Synopsis section then exit the program.

-h

Output the Synopsis section and the Options section then exit the program.

-m

Output a complete Unix man (“manual”) page for ncptl-logmerge then exit the program.

-o

ncptl-logmerge normally writes to the standard output device. The --output option redirects ncptl-logmerge’s output to a file.

-s

Simplify the output by including only lines which differ across input files. No data is output, only prologue and epilogue comments. --simplify can be specified up to four times on the command line:

once

Omit all comments and all lines which are identical across all input files.

twice

Lines which differ across all output files (e.g., Processor (0<=P<tasks)) are also omitted.

three times

The amount of output is further reduced by rounding to two significant digits all numbers appearing in all input files. Doing so makes 1.10644 +/- 0.593714 match 1.12511 +/- 0.58829, for example. (Both are converted to 1.1 +/- 0.59.)

four times

Lists of processors are replaced by the list size. For example, #[22,67,86,430] becomes #[4].

Note that --simplify is intended as a diagnostic tool; files output using --simplify cannot be un-merged to recover the original input files.

In addition to the preceding options ncptl-logmerge requires a list of log files to merge. If a directory is specified, all of the files immediately under that directory are used. (Note that ncptl-logmerge does not descend into subdirectories, however.) Files containing lists of filenames can be specified with a leading at sign (“@”). For example, @filelist.txt means to read a list of filenames from @filelist.txt. Filenames beginning with an at sign can be specified by doubling the at sign on the command line.

DIAGNOSTICS

filename does not look like an unmerged coNCePTuaL log file

ncptl-logmerge accepts as input only log files produced directly by a coNCePTuaL program. It is not a general-purpose file combiner nor does it accept its own output as input. Unrecognized input files cause ncptl-logmerge to abort with the preceding error message.

No process rank found in filename

ncptl-logmerge needs to map filenames to process ranks to indicate which ranks produced which lines of output. If an input file does not contain a Rank (0<=P<tasks) comment, ncptl-logmerge aborts with the preceding error message.

EXAMPLES

Merge a set of coNCePTuaL log files:

ncptl-logmerge mybenchmark-[0-9]*.log > mybenchmark-all.log

The following command is equivalent to the preceding one:

ncptl-logmerge mybenchmark-[0-9]*.log --output=mybenchmark-all.log

Show only “interesting” differences among the input files:

ncptl-logmerge --simplify --simplify mybenchmark-[0-9]*.log

For convenience, one can abbreviate --simplify --simplify --simplify --simplify to -s  -s  -s  -s or even -ssss:

ncptl-logmerge -ssss mybenchmark-[0-9]*.log

RESTRICTIONS

The log files passed to ncptl-logmerge are subject to the following restrictions:

• All files must be produced by the same run of the same coNCePTuaL program.
• None of the files can have been previously merged by ncptl-logmerge (i.e., ncptl-logmerge can’t read its own output).
• Only the first filename passed to ncptl-logmerge is allowed to contain data. Data from all other files is discarded with a warning message.

BUGS

ncptl-logmerge is not a particularly robust script. Specifically, it is confused when input files contain different numbers of comment lines. For example, if one input file includes more environment variables than another or issued a warning about a timer where another input file didn’t, ncptl-logmerge will erroneously report all subsequent lines as being mismatched across input files.

SEE ALSO

ncptl-logunmerge(1), ncptl-logextract(1), the coNCePTuaL User’s Guide

AUTHOR

Scott Pakin, pakin@lanl.gov