10GbE

fastfive.jpg
I live my life a quarter gig at a time.

After a half-price offer expired with Virgin Media, my 1Gb broadband price went from around €42.50 per month to €85, so I called them up to negotiate a better price. Those new customer offers were my target, but I was also willing to switch provider for a much cheaper 1Gb offer.

Siro (fibre delivered via the ESB network) has arrived in my area in the last month or so, and I figured I was in a reasonably good negotiating position. After an hour or so of waiting in queues and chatting to people, I ended up talking myself into signing up for Virgin's recently added 5Gb package.

Just to be clear, they did not talk me into it – there was no upsell. They gave me a reasonable offer for 1Gb broadband, and I pushed for 5Gb. It was also still about 25% cheaper than what I had been paying for 1Gb.

#Fast 5

Fast cars have never been my thing, but fast Internet? From my teens I have always wanted the fastest Internet, even if it didn't really make much sense. A holdover of those early years sipping 5kB/s over dialup – an unquenchable desire for more speed. Just like Vin Diesel. Maybe. I don't really remember what those movies were about.

In any case, even before I signed up, I knew I was giving myself a small problem: I do not have a single piece of equipment that supports 5Gb networking. Everything I have tops out at 1Gb. My thinking was that this was fine, I would just do some planning and slowly upgrade piecemeal over the next few months. Obviously 1Gb had been more than enough for everything, I had only really called up to get a cheaper price.

Unfortunately as soon as the router arrived, my 1Gb equipment became ancient technology shackling me to the mortal realm.

I didn't realise it, but the 5Gb package is not delivered via the same inconveniently located coax endpoint as the 1Gb. Since that coax was originally intended to deliver cable television, it always ends up being in a living room near wherever a TV might go, which is not necessarily the best place for your router to live.

The new broadband is delivered via a new fibre optic line being routed from inside the apartment building, and so two days later I ended up with a sliver of fibre optic going directly to my office.

This changed things. Instead of running CAT6 through the walls myself at some unspecified date in the future, the termination point was now right where I needed it. The only thing holding me back was... everything else.

#10Gb is not necessarily RJ45

Since 1GB had served all my needs up until this point, I actually wasn't very clear on what happened beyond that. The last time real networking hardware was relevant to me – rather than the abstract concept of a network you get via AWS – was the late 2000s, and 1GbE was good enough for the rack at the time.

I was pretty sure that the next step change from 1Gb ethernet was 10Gb ethernet, but there is a halfway house, in fact more of a quarter-way house, between them: 2.5Gb.

Why 2.5Gb? Well, 2.5Gb still runs on CAT5e, which was cheap and widely available for much of the 2000s and 2010s. When your network cards were maxing out at 1Gb anyway, what was the point in paying more for CAT6? Why future-proof for a 10Gb world when your network cards are a mix of 100Mb and 1Gb and your Internet connection is maybe ~25Mb? I still had heaps of old CAT5E cables I cleared out at the start of this year.

Today, the networking hardware to handle 2.5Gb is actually more widely available in consumer devices, though not at all ubiquitous. The most recent M4 Mac mini can be upgraded from a 1Gb port to 10Gb, but I don't recall seeing an option like that on any machine I've bought for nearly two decades? Even my 2008 unibody MacBook, practically a museum piece, had gigabit ethernet as standard!

So 2.5Gb is a big speed increase over 1Gb for a moderate price increase, and will work if you've got CAT5E cables in the walls for some reason.

10Gb is different.

Not only does 10Gb need at least CAT6 cabling, it actually requires a decent amount of energy to operate, which means more heat, more expensive components, and potentially active cooling (i.e., fans).

At 10Gb, networking enters a transitional phase where you can use either copper cabling or fibre optic, with the latter being amazing for low-latency, long-distance, low-interference transmission of, like, hundreds of gigabytes per second. As you might imagine, fibre optic, being a way to transmit photons instead of electrons, requires a different connector than good old 8-pin RJ45. But to help with the transition, there is a clever solution: a hot-swappable modular interface called SFP+, basically an adapter for different types of network interfaces.

Depending on your hardware, you can plug optical or copper cables into ports that support these transceivers.

Presumably, none of this is new if you're a network engineer, but it was new to me and pretty confusing.

Wherever you look on Amazon, you mostly find switches that are a mix of 2.5Gb RJ45, 10Gb SFP+, and only occasionally 10Gb RJ45. If you want to go from an eight-port gigabit switch –which you can buy for €20 – to an eight-port 10Gb switch, you are talking about actively cooled hardware costing anything from €500 up.

I ended up reading a bunch of stuff on ServeTheHome.com, which seems to have great breadth and depth into this particular niche of not-quite-home-not-nearly-enterprise hardware.

One of my main takeaways was that it's a lot easier to stick to a reasonable budget if you're not set on more recognisable brands like Zyxel or Netgear. In my case, I decided getting 10Gb on my primary home server was probably enough, with other equipment running on 2.5Gb, and the rest via WiFi.

Winging their way to me are:

  • A MokerLink Ethernet Switch with 2x10G SFP, 4 x 2.5G Base-T Ports
  • A Binardat 10G SFP+ PCIe Network Adapter, to handily eclipse the capacity of the two gigabit network ports in my aging-but-durable HP Proliant Microserver Gen8.
  • A 10Gtek 10G SFP+ to SFP+ Direct Attach Copper cable
  • A QSFPTEK 10GBASE-T SFP+ RJ45 Transceiver
  • A UGREEN USB C Ethernet Adapter, 2.5Gbps

I mean, even getting 2.5Gb to most of the machines would be a huge step up – this is a 5Gb connection, so two of those machines could go flat out together, which is wild.

UGREEN is the only brand I'd even heard of beforehand, and even that's a pretty new one.

I'm interested to see how it all fits together. So far, this hardware is just pending orders waiting to be delivered, but at least some of my Christmas holiday period is going to be spent wiring this stuff up – I'm looking forward to it!


There is currently no comments system. If you'd like to share an opinion either with me or about this post, please feel free to do so with me either via email (ross@duggan.ie) on Mastodon (@duggan@mastodon.ie) or even on Hacker News.

If you're a network engineer, or just an enthusiast who spots something wrong, please do let me know!