This page describes how to setup and operate an ethernet link between 2 computers, without need for a hub, switch...

Hardware setup

 Fast ethernet (10/100 base-T) : the cable uses two twisted pairs in each direction (receive and transmit). Hence a crossed cable, similar in the principle to those used for null-modem links, is required. This kind of cable is generally green.
 Gigabit ethernet : all four twisted pairs are used for both receive and transmit, hence a crossed cable is not necessary (and will not work).

Software setup

 Linux : gateway and DNS must be deactivated for this interface. A special set of IP addresses is reserved for local networks : 192.168.x.y, where 0<=x<255 and 0<y<256. Using any of these addresses makes it possible to use both local connections and an internet one using another interface. Performance can be dramatically improved by tuning the following parameters :

• the MTU (Maximum Transfer Unit). The default is 1500. It is advised to push it to its maximum value of 9000 ("jumbo frames") :

• % ifconfig eth0 mtu 9000

• the socket buffer size. The default is 64 kB. In theory, the optimum buffer size should be close to <bandwidth>*<round-trip delay>, which gives in our case about 12 kB. Hence for big files no change should be necessary. Nevertheless if needed there are several ways of changing it. It can be done dynamically in NIC benchmark tools, special versions of FTP client/servers like Autobuf, or using sysctl with (for a 256kB buffer)

• % sysctl -w net.core.rmem_max=262144
• % sysctl -w net.core.wmem_max=262144
• % sysctl -w net.core.rmem_default=262144
• % sysctl -w net.core.wmem_default=262144

A helpful document which explains the basics of IP networking on Linux can be found here. A PDF guide to Gigabit performance-tuning can be found here.

 Windows : not tested yet.

Performances

 Fast Ethernet (10/100 base-T) : as expected, using ncftp, we measure a transfer rate of 12 MB/s, that is exactly the maximum theoretical one.
 Gigabit Ethernet (1000 base-T) : using ncftp on RAM-disk files (both sides), we measure a (modest) transfer rate of 30 MB/s with a pair of SysKonnect SK-9821 cards, between 2 PCs equipped with Athlon Thunderbirds @1.33GHz, PC2100 DDR-SDRAMMs and KT266PRO motherboards. Up to 40% of CPU usage is taken by the transfer operation. The transfer rate appears to be mostly limited by the PCI bandwith, which is measured at 56 MB/s using SysKonnect diagnostic tools. This number is stable and does not appear to depend much on things like other PCI cards, IRQs or BIOS configuration.

Moving to other PCs, a dual PentiumIII and an Athlon with this time an Abit KG7-lite motherboard (AMD 761 northbridge), we record PCI bus bandwidths of 86 and 125 MB/s respectively. And indeed, using the optimized software setup described above, we manage to get a sustained transfer rate of 63MB/s between both machines (37 MB/s without optimizing). The measured latency is about 0.10 ms. And this time, CPU usage does not exceed 20%, which correlates with the fact that a smaller fraction of packets is lost (and has to be resent) : PCI buses are now more able to keep-up with the transfer data-rate.

The conclusion of this ? Gigabit ethernet stresses a lot the PCI bus, and for machines equipped with a 32bit@33MHz PCI bus - as most PCs are -, the bottleneck can be the chipset, not the interface card or the network !!