Cisco Provides 400G Roadmap

Over the past few months Cisco has released a series of blogs discussing 400G topics but has avoided discussing anything product related. That changed this week, as Cisco announced its 400G roadmap.

400G may seem like bandwidth overkill considering we’re still in the transition to 100G, but I believe this to be an important technology cycle. More and more things are being connected to the network and generating massive amounts of data, and that’s driving network traffic to new heights.

Cisco is introducing two new product families:

 

  • Nexus 3400-S series includes a pair of 12.8Tbps high density 400G switches optimized for low power and latency, making them ideal for web-scale companies. The Nexus 3432D-S is a 1RU fixed form factor switch, and the Nexus 3408-S is a 4RU semi-modular switch. Both use the QSFP-DD optics for 400G connectivity.
  • Nexus 9300GX series are leaf and spine switches that fit into Cisco’s ACI (Application Centric Infrastructure) solution. The Nexus 9316D-GX is a 1RU switch with 16 400G ports and a total capacity of 6.4Tbps. The Nexus 93600CD-GX is also 1RU with 28 ports of 100G and eight ports of 400G. The differentiator here lies in fabric telemetry (versus box level) with full flow data that isn’t sampled. The latter is important, as when sampling is introduced, the data is incomplete and key insights can be missed. Cisco claims it’s the only 400G switch with SRv6 capabilities (segment routing for IPv6). I couldn’t find another manufacturer that offers this, so it appears it is correct about that. This feature set is optimized more for enterprises and service providers, particularly those with ACI deployments.

All of the switches are fully backwards compatible and can operate at 10G, 25G, 40G, 50G, 100G, and 400G, giving customers the ability to run them at whatever speed they need now and then upgrade to 100G or 400G when the need arises in the future. The switches are loaded with features that make the network easier to manage and secure, including policies, segmentation, whitelisting, packet visibility, smart buffering, and low latency.

 

Let’s Talk Optics

Cisco is choosing to support the QSFP-DD optical connectors across all its products instead of the OSFP optics. There’s currently a debate in the industry regarding whether OSFP or QSFP-DD optics should be used. Without going into too much technical detail, QSFP-DDs are smaller and provide better backward compatibility. This means that 400G switches that use QSFP-DDs can seamlessly drop into existing environments.

 

OSFPs are slightly larger optics and are designed with a better forward-looking roadmap. Deploying OSFP-capable switches does take a bit more work and creates some disruption in the data center. Juniper has announced support for QSFP-DD, as has Arista, which has a separate product for customers interested in OSFP.

 

I believe OSFPs to be technically superior as they are the next generation of optics, but QSFP-DD is likely to see greater uptake because of its non-disruptive nature. This leads to a bit of manifest destiny, as most of the ecosystem providers (network interface cards, cabling vendors, etc.) I have talked to also believe QSFP-DD will be the way for most organizations and are planning to build their products with that assumption… which only helps to fulfill on the assumption that more business will use them.

 

Cisco also plans to bring its BiDi (BiDirectional) optical technology to 400G. For those not familiar with BiDi, it’s a proprietary Cisco technology that was first used to let customers bring 40G speeds to the access layer using the same 10G cabling plant, saving big dollars. Cabling can be very expensive, particularly optical cables which can run about $2,000 per port. The BiDi optics are the same size as the QSFP-DD but operate over a quarter of the fiber strands. It’s unlikely that another switch manufacturer could have pulled this off; but Cisco is Cisco and has a large number of third parties that support BiDi, so it’s been very popular in its customer base.

 

The other difference with Cisco versus the competition is that it is using a mix of merchant and custom silicon. Juniper and Arista have built their products using Broadcom’s Tomahawk 3 chip, and during my pre-briefing with Cisco, executives stated emphatically that it was not using Broadcom. I suspect the 3400-S products will use an alternate merchant silicon product and the 9300-GX either solely Cisco or Cisco plus merchant. Over the years Cisco has used its own silicon to jump into markets faster than its competition.

 

The Big Picture

400G network is critically important, as the world is becoming more network centric. Web-scale providers need larger, faster network fabrics as they compete on speed. Service providers are prepping for 5G deployments that will drive up bandwidth demands on the core network. With enterprises, this opens the door to new applications such as 3D holographic telepresence and virtual reality. Additionally, artificial intelligence and machine learning is dependent on storage, network, and compute working in lockstep. If one or two are significantly faster than the other, AI/ML won’t work optimally. Over the past few years GPU speeds have grown exponentially and storage is shifting to NVMe, which is even faster than flash. 400G ensures that the three legs of the AI/ML stool are all in lock step.

 

Cisco is starting field trials in December of 2018, with the new switches due to be generally available in the first half of 2019 calendar year -- just in time for Cisco Live in early June. Cisco offered no pricing information at this time, but I’m expecting the price to be somewhere in the 1.5 – 2x range of its 100G solutions.