The 10GbE Upgrade Path: NICs, Switches and Reality

A single modern SSD outruns gigabit Ethernet by five to ten times — which means on a 1GbE network, every NAS transfer, VM migration and backup window is measuring the wire, not the storage anyone paid for. The 10GbE upgrade stopped being expensive years ago; what remains is knowing which parts to buy and where the gains actually land.

What changes, concretely

Gigabit moves ∼112MB/s flat out. Ten gigabit moves ∼1.1GB/s — a large VM image in minutes instead of half an hour, backup windows shrunk to fit their nights, NAS shares that feel like local disks for real work. The catch worth stating: you get those numbers when storage on both ends can feed them. An SSD pool saturates the link; a tired RAID of spinners improves modestly. Upgrade the network when storage is ready to use it — on flash-era servers, it long since is.

The NICs: a solved problem

Used enterprise SFP+ cards are abundant, cheap and superbly supported. Intel X520 (82599): the veteran — driver support everywhere, ESXi through 8 included, the default recommendation. Intel X710: newer, more offloads, similarly universal. Mellanox ConnectX-3/4: excellent Linux citizens, often the cheapest port-for-port (check ESXi-version fit for the exact model before buying for VMware). On HPE/Dell refurbished servers, the FlexibleLOM/rNDC variants of these same silicon families fill the dedicated slot without spending a PCIe one. Prefer SFP+ models over 10GBASE-T for heat, latency and link-type flexibility — the cabling article's argument, inherited whole.

Switches: where the price broke

Three sane tiers. Used enterprise (Arista 7050, Brocade/Ruckus ICX 6610-class, Dell force10 lineage): many SFP+ ports for scrap prices, datacenter features included, fan noise included too — closet hardware. Prosumer quiet (MikroTik CRS3xx family, QNAP/TP-Link 8-to-12-port SFP+ units): office-tolerable, enough ports for a small estate, modest cost. The two-node zero-switch answer: a DAC straight between server and NAS — two NICs, one cable, full speed, no switch at all; the honored first step of half the labs in existence.

Jumbo frames and other tuning

MTU 9000 helps most where CPUs are weakest, and current NICs with offloads deliver line rate at standard 1500 anyway. The engineering answer: it is an all-or-nothing setting per network segment, and a single mismatched device produces mysterious stalls that cost more hours than the frames save. Default to 1500 everywhere; deploy 9000 only on a dedicated, fully-controlled storage VLAN where every port is yours to verify. Then measure with iperf3 before and after — the honest benchmark that keeps tuning honest.

Migration order that works

Storage boxes and hypervisors first (they feel it most), a DAC or small SFP+ switch as the fabric, VLAN for storage/migration traffic if the switch speaks it, clients later or never — desktops rarely need more than gigabit. Total damage for a three-machine core done used: dinner money per node — for the single most tangible performance upgrade an existing fleet can receive.