Thursday, May 19, 2011

Home network benchmarking with iperf

Having recently purchased a new NAS device, as well as upgraded network cables in my house to cat6 (1Gbps), I started wondering about just how fast and healthy my network setup was. Are all cables in order? Are any of my switches a bottleneck? Any particular slow endpoints?
performance challenged
There are of course commercial software to assist in this, but after having compiled and loaded up a bunch of Linux software onto my small NAS, it was only natural to give "iperf" a try. It's a small open source client-server utility you start in listening mode on one device, and in transmission mode on another, and have it transfer a bunch of random data over TCP. You can obtain iperf from sourceforge, from your Linux package manager or just Google for precompiled binaries if you are lazy.

Simple unidirectional test
The process is simple, log onto the receiving device and run iperf in listening mode, in the following case it's my Synology NAS:

NAS> iperf -s -p 5555
Server listening on TCP port 5555
TCP window size: 85.3 kByte  (default)

Then also run iperf, though this time in client mode targeting the server you just started:

casper@laptop:~/$ iperf -c -p 5555
Client connecting to, TCP port 5555
TCP window size: 19.0 kByte  (default)
[  3] local port 52742 connected with port 5555
[ ID] Interval       Transfer     Bandwidth
[  3] 0.0-10.3 sec   26.2MBytes   21.4 Mbit/sec

Note that the above result is quite slow, as a consequence of being on congested WiFi and benchmarking the slowest node in my household, the NAS.

This gives you a bps metric for how fast the client could transfer data to the server. To benchmark both upstream and downstream, you must reverse the client-server pair and repeat the process.

Complex bidirectional test case
By running tests between each and every possible node, it becomes possible to draw some interesting conclusions about the setup in general. The wired setup in my household looks roughly like this (without smartphones, TV, PS3, IP phone etc.):

With Gigabit connections between all major nodes, I expected to see throughput up around 90% of the theoretical maximum bandwidth. What I saw was hardly consistent however, as the following graph shows:

Note that solid edges signify physical cat6 cables, dashed edges logical connections. Edges are labeled with directional throughput in Mbps and nodes contains summerized throughput on the form in/downtream and out/uptream.

Lessons learned
Infrastructure nodes like router and switch would appear to have performed optimal, and there is no hinting of faulty cabling. On several occasions, one can observe throughput beyond 940Mbps, or 94% of the theoretical maximum.
I was a little surprised to see that the overall best performer, was actually my 3 year old sub-notebook. The network stack (hardware+software) is obviously very well implemented on the laptop. The PC's, particularly one of them, would probably benefit from a new dedicated Gigabit adapter rather than relying on the one built into the motherboard.
However, the greatest surprise was to see how the NAS struggled, particularly when receiving data and when having to cross 2 switches. In all fairness, the slow CPU of the NAS (1.6GHz ARM) could very likely be key to the poor performance of iperf - loopback tests on all endpoints incl. the NAS suggests this to be the case.

There are of course many other parameters one could care about; ping time, jitter, packet loss, UDP packets, jumbo frames etc. but I was mostly just concerned about TCP bandwidth throughput. I've learned that perhaps I should flash my DD-WRT router back with the original firmware, or perhaps try another open source image like Tomato, in order to see whether I can mitigate some of the observed bandwidth loss. It also seems abundantly clear, that the network system of my NAS device is more optimized for delivering data than receiving it.

I'd like to do a plugin for Synology's management interface, having it run native iperf against a Java implementation hosted by the client as an applet. Currently looking into whether this is possible. I fear it's too low level for Java, as this was one of the reasons I changed to C# for dealing with raw sockets some years ago.


Thomas Schjøtler Nielsen said...

Emne: Ang. Synology DS211+

Hej hr. Bang

Hvordan får jeg ændret bootstrap.xsh, hvis jeg er logget på DS211+ via SSH (telnet).
Jeg tænker på fejlmeddelelsen:
Error: CPU not Marvell Kirkwood, probably wrong bootstrap.xsh

Unknown said...

Hej Thomas. Det skriver jeg lidt om her []. Problemet er at scriptet checker mod den lidt ældre mv6281 chip, hvor DS211+ har mv6282. Så enten ændrer du "Feroceon-KW" til "Feroceon" i den udpakkede (og kører igen), eller også henter du bare et .xsh script hvor denne tilretning allerede er lavet [].

Thomas Schjøtler Nielsen said...

Takker Casper. Jeg er ved at opstarte den java baserede PS3 Media Server - takker for din fine guide :)

Unknown said...

Lyder interesant. Bruger du Cacao fra ipkg pakkerne?

Thomas Schjøtler Nielsen said...

Nej det har jeg ikke prøvet. Hvad gør cacao af forskel?

Unknown said...

"cacao" er en Java implementation man kan installere på Synology.

Thomas Schjøtler Nielsen said...

Jeg prøvede cacao men det hjalp ikke. Kender du til PS3 Media Server?

Unknown said...

Ja, men den kan du ikke køre på Synology p.g.a. den bruger "rigtig" Java (Oracle, til x86 CPU). Der findes dog noget andet DLNA software til Synology som jeg ikke kan huske navnet på. Jeg bruger bare Media Station fra Synology. Den kan ikke transcode on-the-fly men det vil intet software kunne på en lille 1.6Ghz ARM CPU!

Thomas Schjøtler Nielsen said...

Tænker du på Twonky Media Server?

Der er en gut, som skriver herinde, at han fik det til at virke:

Jeg har fulgt vejledningen, men når aldrig så langt som de andre brugere i tråden.

Unknown said...

Cool. PS3's DLNA funktion er jo ret ringe (bl.a. ingen WMA Pro codec), hvorfor jeg istedet bruger den DLNA funktion der er indbygget i mine fjernsyn. Men jeg tror desværre stadig ikke du kan få transcoding til at virke på så lille en CPU som der er i disse små Synology enheder. Har du overvejet istedet at konvertere dine video filer? Hvor hopper kæden af for dig, for jeg kunne da godt tænke mig at høre om du kan få det til at virke?

Thomas Schjøtler Nielsen said...

Jeg starter(prøver) PMS ved at taste sh Den står stille et kort øjeblik, og så får jeg en hel masse meddelelser.

Den står stille et øjeblik ved meddelelsen:
"Toolkit error: Cannot load AWT toolkit:".

Den går videre og står i stampe ved:
"- End of configuration.

GUI environment not available
Switching to console mode"

Nu skulle den skrive en masse om, at den starter PMS op. Der sker dog aldrig.

Thomas Schjøtler Nielsen said...

Hvis jeg prøver at starte PMS med kommandoen: jamvm -jar PMS.jar får jeg samme meddelelser som ved sh

Unknown said...

Tja fejlen skyldes jo nok at der ikke er noget java.awt (grafik) library installeret. De små Java-ports der findes til ARM enheder, har typisk ikke noget UI lag med.
Så hvis ingen af de 2 Java pakker der findes til ipkg, virker, kan du prøve at smide parametren -Djava.awt.headless=true på, som instruerer JVM'en i at starte headless (uden at søge efter grafiske libs).

Thomas Schjøtler Nielsen said...


Jeg opgav PS3 Media Server, og valgte at følge en længere til denne mediaserver:

Den fungerer fint - bortset fra at .ISO filer ikke bliver vist, men det er vist ikke understøttet endnu?

Thomas Schjøtler Nielsen said...

Jeg droppede PS3 Media Server. Jeg forsøgte med et andet javabaseret medieserver, Serviio, og det virker ganske fint. Dog er .ISO ikke understøttet.

Du kan finde en guide her:

Unknown said...

Meget interesant, tak for linket!