1.1 Motivation

A frequently reinvented wheel among network researchers is a suite of programs that test a network’s performance. A problem with having umpteen versions of performance tests is that it leads to a variety in the way results are reported; colloquially, apples are often compared to oranges. Consider a bandwidth test. Does a bandwidth test run for a fixed number of iterations or a fixed length of time? Is bandwidth measured as ping-pong bandwidth (i.e., 2 * message length / round-trip time) or unidirectional throughput (N messages in one direction followed by a single acknowledgement message)? Is the acknowledgement message of minimal length or as long as the entire message? Does its length contribute to the total bandwidth? Is data sent unidirectionally or in both directions at once? How many warmup messages (if any) are sent before the timing loop? Is there a delay after the warmup messages (to give the network a chance to reclaim any scarce resources)? Are receives nonblocking (possibly allowing overlap in the NIC) or blocking?

The motivation behind creating coNCePTuaL, a simple specification language designed for describing network benchmarks, is that it enables a benchmark to be described sufficiently tersely as to fit easily in a report or research paper, facilitating peer review of the experimental setup and timing measurements. Because coNCePTuaL code is simple to write, network tests can be developed and deployed with low turnaround times—useful when the results of one test suggest a following test that should be written. Because coNCePTuaL is special-purpose its run-time system can perform the following functions, which benchmark writers often neglect to implement:

• logging information about the environment under which the benchmark ran: operating system, CPU architecture and clock speed, timer type and resolution, etc.
• aborting a program if it takes longer than a predetermined length of time to complete
• writing measurement data and descriptive statistics to a variety of output formats, including the input formats of various graph-plotting programs

coNCePTuaL is not limited to network peformance tests, however. It can also be used for network verification. That is, coNCePTuaL programs can be used to locate failed links or to determine the frequency of bit errors—even those that may sneak past the network’s CRC hardware.

In addition, because coNCePTuaL is a very high-level language, the coNCePTuaL compiler’s backend has a great deal of potential. It would be possible for the backend to produce a variety of target formats such as Fortran + MPI, Perl + sockets, C + a network vendor’s low-level messaging layer, and so forth. It could directly manipulate a network simulator. It could feed into a graphics program to produce a space-time diagram of a coNCePTuaL program. The possibilities are endless.