Dissertation: Data-Driven Analysis and Modeling of Packet Processing Systems

Full text: PDF

Abstract

The experiment—the pickax of the scientist—is the instrument of choice to mine scientific knowledge. The defining feature of scientific experiments is their reproducibility. Other researchers can recreate experiments to verify or falsify the results of the original experiment. The domain of computer networks rarely reproduces experiments, mostly due to a lack of proper documentation of the original experiment. This thesis proposes a measurement methodology that embeds the concept of reproducibility into experiment design, documentation, and execution.

Our main target of investigation is a two-node setup that allows the benchmarking of packet processing devices. To perform experiments in this setup, we propose an experiment workflow that relies on full automation for experiment configuration, execution, and evaluation, thereby creating repeatable experiments. By releasing the experiment artifacts, including the experiment scripts, the measured data, and their analysis, other researchers can either try to replicate the results of the original experiment or even create their own experimental setup to reproduce the original results.

Further, we present measurement tools that allow the effective investigation of the packet processing devices. An essential prerequisite of network experiments is the precise definition and replay of the network traffic at the input of the investigated device. Therefore, we analyze differ- ent packet generators and develop suitable packet generation strategies. To enable an accurate delay analysis, we use hardware-generated timestamps.

The measurement methodology and the measurement tools are applied to two different domains: high-performance packet processing applications and wireless networked control applications. The former domain typically relies on specialized packet processing frameworks. We analyze the frameworks themselves and other applications based on them, such as software routers or intrusion prevention systems. In contrast to traditional kernel-based network processing, these frameworks offer a ninefold increase in throughput that almost scales linearly with the number of available processing cores. At the same time, service quality rises. The latency and especially the worst-case behavior improve when relying on such a high-performance framework. The requirements and the conditions of the second application domain differ significantly. Wireless links are lossy and sensitive to interference, poor preconditions for control applications where low latency and loss rates are crucial. An additional challenge is the execution of repeatable measurements due to the highly sensitive network links. We present an approach using a shielded environment for repeatable wireless network experiments. Furthermore, we create our own platform for control systems consisting of a robot and a benchmarking suite to perform realistic measurements for this domain.

Models can help quantify the observations of the presented measurements and describe the behavior of packet processing systems. The thesis presents a novel modeling technique to predict the performance of packet processing devices. It analyzes the components of packet processing devices, such as the CPU or system buses, and their individual performance. The performance of the components is then combined to predict the overall performance of the packet processing device. We check the validity of our models by applying them to the performed measurements.