OS/Hardware interaction


Modern OSes handle network IO and buffering quite efficiently, thanks
So there's no real reason to perform lots of buffering in an application!
Syscalls are cheap, but not at high throughput rates
You'll want to try and read/write large chunks
At high throughput (say, 20k FDs, about 5k IO events a loop, about 100 loops a second) you're looking at a good 100,000 syscalls a second
Memory bandwidth can become an issue at high speed
Think of all the RAM-RAM copying!
If your network server requires oodles and oodles of RAM, it may become susceptible to paging
Squid is a good example - processes can grow in excess of 500mb due to its in-memory index