Control Planes Are More Than Signalling

Earlier this year I joined the technical advisory council for the Magma project, which has given me an excuse to dig deeper on the workings of mobile networks. Once I got my head around the Magma architecture, I noticed that its control plane looks much more like an SDN system than a traditional telco network implementation. This post looks at the different types of control planes and why centralized control is so significant. Thanks to Amar Padmanabhan and Jen Rexford for their helpful input.

The idea that the data plane and control plane are separable is almost as old as networking. In the Internet, the data plane was defined as part of the original architecture and (aside from IPv6) has barely changed since. Meanwhile, the control plane has continued to evolve largely independently, with new routing protocols being added over time as a notable example. There have also been several efforts to formalize this separation, of which the Software-Defined Networking (SDN) movement was the most successful. (Notable prior efforts included Forces and GSMP).  While the original Internet architecture separated control and data conceptually, SDN enabled them to be separated into different physical devices, with protocols defined (e.g., OpenFlow) to allow communication between the control and data plane. The implications of this were significant for reasons discussed below.

This separation is not limited to data networking but is found in the telephone network as well. SS7, for example, forms part of the control plane of the telephone network, and is largely independent of the data plane. More recently, 3GPP formally defined “Control and User Plane Separation” (CUPS). As in SDN, this separation was intended to allow different devices to implement the different planes.  There is no real difference between “user plane” and “data plane”, but there are important differences between how control planes are implemented in typical 3GPP systems versus how SDN systems implement them. And this difference is highlighted by the architecture of Magma, which takes an SDN approach to building a mobile packet core.

Why has Magma adopted an SDN approach? One of the main objectives of Magma is to deliver a mobile core that can be managed cost-effectively and scalably. SDN has been successful not only because it provided a clear point of separation between control and data planes, but because it introduced a central point of control. This is true for Magma as well.

Consider three applications of SDN: traffic engineering (as in Google’s B4); network virtualization (as pioneered by Nicira); and SD-WAN. Traffic engineering needs a central view of traffic demands to achieve optimal path placement. Network virtualization needs a central API so that networks can be provisioned and managed by other software without knowing where all the network components are located. And SD-WAN simplifies operations by managing a network as a whole rather than provisioning individual boxes, branches, and tunnels. In the same way, if mobile and wireless networks are to be managed efficiently at scale, we need centralized management for configuration and monitoring, and central APIs to integrate with other software systems. All of these capabilities come when we build a centralized control plane.

The Power of Logically Centralized Management

Consider the figure below of a typical SDN system. The management plane sits above the control plane and provides the single point of entry for both the input of configuration information and for the querying of status information such as metrics, alerts, etc. When configuration information is provided to the management plane via the northbound API, that updates the desired state. To provide a single API to manage an entire network, rather than managing a series of boxes one by one, it was necessary to create this centralized management model. This is one of the salient features of SDN, and it was adopted in Magma to simplify the operation of a mobile core network. It also provides the API for integration with other software systems for logging, alerts, metrics, etc.

The desired state model used here is a common pattern in cloud native systems (here is Joe Beda giving my favourite explanation of the desired state model in Kubernetes). The control plane’s main job is to continuously reconcile the actual state of the system with the desired state, in the face of configuration changes, failures of components, or other events such as the arrival and roaming of UEs. 

Note that this is a logical view of the architecture, not a physical one. The control plane is logically centralized in the picture, but it is actually implemented in a hierarchical and distributed manner for reasons of availability, fault isolation, and scale. Some parts of the control plane run centrally (in the orchestrator), while others are distributed out to the access gateways (so they are close to the distributed data plane and scale out accordingly). This is another established SDN design pattern: a hierarchical control plane that combines a centralized component and a distributed component. OVN (Open Virtual Network) is another example of an SDN system built with this approach.  This hierarchy provides scaling and reliability benefits while retaining the centralized management that is essential for simplified operations and integration with other systems via a central, network-wide API.

An important benefit of SDN control planes is that they leverage standard distributed systems techniques (and, in some cases, existing code) to achieve reliability, fault tolerance, scalability, etc. This saves us from continually reinventing techniques to achieve those goals in every network protocol. This paves the way for adding new capabilities (such as traffic engineering) without having to repeatedly solve problems like reliable dissemination of state among all the network components. 

Separation of Control and Data Alone Is Not Enough

In telecommunications networks, the control plane often refers to signalling protocols used to set up the user plane channel. As an example, before a mobile device (user equipment or UE) can start exchanging data with the Internet, it needs to authenticate itself, a user profile needs to be looked up, and the device needs to be assigned an IP address. All of these are control plane functions. Once they are complete, the UE can send and receive data via the user plane. If the UE roams to a different base station, some more signalling needs to take place to set up the user plane state so the UE can send and receive data via the new base station. The separation of control and user (or data) planes is illustrated below.

With the formal definition of CUPS, some devices may be dedicated to user plane functions (e.g. moving packets between a UE and the Internet) while others may be dedicated to control plane functions (e.g. authentication of UEs). This separation is a helpful principle for building a network but it does not, on its own, deliver the benefits of SDN. The benefits come from a more expansive view of the control plane.

Software-Defined Networking: A Different Model

In SDN, the control plane is logically centralized, as discussed above. Its job is to make changes to the data plane to ensure that the desired state–as specified via the northbound API–is actually realized. It is this centralization that enables a network operator to manage a network as a whole, rather than on a box-by-box basis. The creators of SDN realized that only by centralizing the control plane would it become easier to manage and reason about the behavior of networks. Troubleshooting a problem, for example, now starts with querying a central API rather than trying to figure out which boxes to investigate.

Note that the control plane is constantly gathering information from the data plane, e.g., learning about the new location of a UE. This is referred to as “discovered state”. When there is a discrepancy between the discovered state and the desired state, the control plane has the responsibility to decide how to resolve that discrepancy, e.g. by pushing some new information to some part of the data plane. This is an important sense in which a control plane is more than signalling: the control plane is constantly responding to changes in the network, and not only events such as a new UE trying to establish connectivity. 

SDN transformed networking because it enabled completely new operational paradigms for networks. But separation of control and data was only the first step; it was the development of logically centralized control planes, using distributed implementation techniques to achieve reliability and scale, that made SDN successful. Now we see this approach being used to deliver the benefits of SDN to mobile networks: scalable, resilient systems with simplified operations and the power of network-wide management. This trend of mobile networks leveraging the best practices of cloud native systems is one we hope will continue.

The set of published use cases for Magma keeps growing. For a few hours last month, there was a published “proof” that P = NP. Now we’re back to not knowing. And the gold medal in the women’s Olympic cycling road race went to a postdoc mathematician who apparently did her own research on heat acclimatisation