RFC 2544 Benchmarking Framework Based on MoonGen
Benchmarks
The framework has been applied to different devices under test (DuT):
- MikroTik Router
- Linux Router
- FreeBSD router
MikroTik Router
- Generated PDF report
- Mikrotik Cloud Core Router CCR1036-8G-2S+
- Tilera Tile-Gx36 @ 1.2GHz
- RouterOS
- 25 firewall rules were configured.
Linux Router
- Generated PDF report
- 1 x Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
- Memory: 16 GB
- Mainboard: X9SCL/X9SCM
- NICs: 2x Intel 82599EB (2x X520-SR1), 4x Intel 82576
- Grml Debian Linux 3.7.9-1
- iptables was used to implement the 25 firewall rules recommended by RFC 2544.
FreeBSD Router
- Generated PDF report
- 1 x Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
- Memory: 16 GB
- Mainboard: X9SCL/X9SCM
- NICs: 2x Intel 82599EB (2x X520-SR1), 4x Intel 82576
- FreeBSD 10.1
Comparison of Throughput and Latency in one Table
| Frame Size [bytes] | Throughput [Mpps] | Throughput [Mbit/s] | Latency average [mus] | Latency std. dev. [mus] |
|---|---|---|---|---|
| Linux (with firewall) | ||||
| 64 | 1.025 | 626 | 55.7 | 32.0 |
| 128 | 1.018 | 1095 | 63.5 | 106.7 |
| 256 | 0.934 | 1875 | 64.6 | 36.5 |
| 512 | 1.011 | 3909 | 50.3 | 30.3 |
| 1024 | 1.009 | 7658 | 47.2 | 28.5 |
| 1280 | 0.992 | 9375 | 55.9 | 29.9 |
| 1518 | 0.893 | 10000 | 1052.0 | 41.3 |
| FreeBSD (without firewall) | ||||
| 64 | 1.283 | 782 | 6.3 | 1.2 |
| 128 | 1.310 | 1408 | 6.9 | 2.8 |
| 256 | 1.326 | 2659 | 7.5 | 2.7 |
| 512 | 1.375 | 5315 | 29.1 | 28.5 |
| 1024 | 1.316 | 10000 | 660.5 | 42.6 |
| 1280 | 1.057 | 10000 | 188.9 | 42.7 |
| 1518 | 0.893 | 10000 | 124.3 | 42.7 |
| MikroTik (with firewall) | ||||
| 64 | 14.204 | 9545 | 60.5 | 30.5 |
| 128 | 8.446 | 10000 | 504.9 | 33.1 |
| 256 | 4.529 | 10000 | 969.7 | 33.1 |
| 512 | 2.350 | 10000 | 303.2 | 34.1 |
| 1024 | 1.197 | 10000 | 249.6 | 21.1 |
| 1280 | 0.962 | 10000 | 141.6 | 21.0 |
| 1518 | 0.813 | 10000 | 315.5 | 21.0 |
Discussion
The follwing authors have used these and other benchmarking data for discussion of router benchmarking challanges. The authors try to extend the state of the art in order to adapt to current technical progress, including grown capabilities of benchmarking devices and changed router internal processing paths in software data planes.
Framework source code
The source code is available on GitHub and has been merged to the upstream MoonGen Repository. The framework was initially developed in context of a student projects done by Patrik Werneck [1]. It is based on the high speed packet generator MoonGen. MoonGen is the first software packet generator that allows precise latency measurments and traffic pattern generation. The code is not considered production ready but part of ongoing research activity. The implemented tests go beyond RFC 2544 [2] requirements as they include in depth latency tests.
References
[1] Patrick Werneck: "Network Device Benchmarking Framework", Bachelor Thesis, Advisors: P. Emmerich, D. Raumer, S. Gallenmüller, 2015
[2] S. Bradner, J. McQuaid "RFC 2544: Benchmarking Methodology for Network Interconnect Devices", 1999
Chair of Network Architectures and Services (I8)
Technical University of Munich
Boltzmannstraße 3
85748 Garching
Tel.: +49 89 289 - 18032
Fax: +49 89 289 - 18033