Introduction To Network Topologies For IP – Part 2

In the first part of this article, dB Broadcast looked at the 'big switch' and three-tier tree network topologies as applied to IP-based broadcast systems. Limitations on scalability in particular were identified. In this part, another architecture is considered with an examination of the spine-leaf system of network design.

Clos crossbar networks (after researcher Charles Clos), originated in telecoms networks in the 1950s. The big advantage of this architecture in telecom applications was that multiple switching paths allowed telephone calls always to be connected, and not blocked by another call.

In the context of data centre applications, two-tier Clos networks, which are commonly referred to as spine-leaf networks (Figure 2), are inherently non-blocking. Hosts are connected to leaf switches (analogous to the access switches in a tree network) and each leaf switch is connected to every spine switch in a full mesh topology. As a result, the spine layer acts as a network backbone, providing interconnection between all the leaf switches.

Equal-Cost-Multi-Path (ECMP) Layer 3 routing or Layer 2 switching allows all links to remain forwarding (non-blocking), meaning that multiple paths are available, minimising bottlenecks and oversubscription. In Layer 2 leaf-spine designs, Transparent Interconnection of Lots of Links (TRILL) or Shortest Path Bridging (SPB) commonly take the place of the spanning tree. In Layer 3 designs, the path is explicitly routed, often using a shortest path routing algorithm such Open Shortest Path First (OSPF).

In both Layer 2 and Layer 3 implementations, whatever path is used, the payload hops to a single spine and then to another leaf, making latency predictable (the exception being where both hosts are connected to same leaf switch, where no hops are needed). As the forward and reverse paths always have the same number of hops, the network has far less asymmetry than a three-tier tree network.

Spine-leaf networks are highly scalable. If the network is danger of becoming oversubscribed, an additional spine can be added with links to every leaf. Likewise, if port capacity becomes an issue, an additional leaf can be added with links to every spine.

There are some disadvantages with the spine-leaf topology. The first is the large amount of cabling needed to connect every spine with every leaf. This becomes an increasing issue as the network is scaled up. This does mean that careful consideration should be given to the location of the spine switches within the data centre, to minimise cable lengths.

If a layer 3 design is used, then spine-leaf VLANs are limited to each individual leaf switch, making leaf to leaf VLAN access not possible, even within the same location. However, "VLAN Everywhere" is not restricted if a layer 2 design is used. Alternatively, the use of Virtual Extensible LAN (VXLAN) provides a virtual layer 2 overlay over layer 3, making leaf to leaf VLAN mobility then possible.

The inherent non-blocking characteristics of the spine-leaf topology along with its built-in scalability make it likely that it will eventually become the norm in high-bandwidth IP video applications.

dB Broadcast has accumulated unique knowledge in designing and implementing IP solutions for broadcasting, and has systems experience with technology from several different switch manufacturers, including solutions from Arista, Cisco and Evertz. Over the last three years, dB Broadcast has completed a major programme for Bloomberg Television, and for the next two years will be working with BBC Wales. This landmark project features a spine-leaf network deploying SMPTE ST 2110, and will deliver the first BBC facility to use IP technology across both its production and broadcast operations.

This article is also available in the June edition of Broadcast Film & Video.