Private 5G: As Easy As Wi-Fi*
*Some Restrictions May Apply
As we’re working towards finishing our Private 5G book, we’ve been completing the Hands-On Experience Appendix. That necessitated a bit more, er, hands-on work than we normally would do, which provided the impetus for this week’s post.
Our Private 5G book is informed by our experience designing and implementing an open source Kubernetes-based edge cloud that hosts—among other edge workloads—a managed 5G connectivity service. Edge applications can take advantage of local breakout, meaning they can communicate directly with IoT devices (and the like) without their packets ever leaving the enterprise. This local Connectivity-as-a-Service is then offered as a managed cloud service (rather than a traditional Telco service), including an API and Dashboard that makes it easy to monitor and control connectivity at runtime. The hope is that Private 5G will be as easy to deploy and use as Wi-Fi is today. (For the reader prepared to argue that Wi-Fi is sufficient for all edge use cases, we leave the 5G vs. Wi-Fi debate for another time.)
It should come as no surprise that designing/implementing Private 5G is not exactly the same as deploying/operating Private 5G, and since the main goal of the Appendix is to help readers with the latter, we decided to take the system we had built out for a test drive. But there’s an important qualifier before we get to that. The system we’re talking about, Aether, is not a collection of isolated components that leaves the dirty work of operationalization to someone else. Aether includes all the integration glue needed to bring up an operational system in support of live traffic—a topic we’ve covered in other posts and written an entire book about—but that doesn’t mean it’s easy for ivory tower architects like Bruce and myself to bring up Aether without a little friction. Some of the challenges were our missteps, but some point to the inherent difficulty in the Telco-to-Cloud transition Aether is trying to catalyze.
Step one is getting your hands on a 5G small cell radio, and they aren’t exactly available today at Best Buy. We’re using the Bridgestone Indoor 5G Sub-6G Small Cell from Sercomm. We also have experience with Sercomm’s 4G counterpart (which is less expensive and easier to find). You’ll also need a starter set of UEs, and while several smartphones support CBRS (e.g., iPhone 11, Google Pixel 4, or newer), our recommendation is to include a 5G dongle that can be attached to a Raspberry Pi. Acquiring the 5G hardware is still a problem today, but that’s probably a short-term situation. The other bit of hardware you’ll need is a server (or VM) to run Aether on, but the requirements aren’t too steep (Quad-Core, 12GB RAM, running Ubuntu 20.04 or 22.04). Note that the approach I’m describing uses CBRS spectrum that is allocated in the US; other countries are in different stages of establishing similar allocations.
Step two is where 5G is the most unfamiliar to anyone who has installed a Wi-Fi AP: Configuring the small cell radio. There are three parts to this. The first part is setting the RF-related parameters, which I am wholly unqualified to do. Their names are cryptic (e.g., FreqSsb, Arfcn), their settings seemingly arbitrary (e.g., 3609120, 643356), and the formulas to compute them… not exactly intuitive:
These (and other) parameters are related to the control the operator has over how the available frequency band is used, which is part of the value 5G brings. I clearly have more to learn, but fortunately, the out-of-the-box defaults work. The second part is connecting the small cell to the local network, which is straightforward, complicated only by the fact that the radio has two 802.3 ports: one known as WAN (but labeled 2.5G on the Sercomm 5G small cell) and the other know as LAN (but labeled 1G on the Sercomm 5G small cell). The WAN port is how the small cell connects to the Internet (indirectly via the Mobile Core, which we’ll get to in a moment). The LAN port is for connecting the radio to a management network, which is worth mentioning because you’ll eventually need to learn TR-069/TR-098 (in place of SNMP/MIB, respectively), since you’ll technically be managing on-prem Telco equipment instead of IETF-spec’ed Internet devices. There’s also an O1 Management interface, which is the O-RAN approach to managing RAN elements, but I have not yet had an opportunity to use it. It’s probably better to have too many programmatic interfaces than too few, but I was able to do everything I needed to through the dashboard, which is enough to get started. The third part is configuring the Spectrum Allocation Server (SAS), which is responsible for managing access to the three tiers of the CBRS spectrum. You’ll need to familiarize yourself with the SAS requirements and get credentials from a SAS provider (we use Google) if you want to get past the “turn it on and see if it boots up” stage. (You’ll also need to connect a GPS antenna, which the radio needs so it can tell the SAS its precise location.)
Step three is interesting because it’s related to how you assemble a system out of building-block components. As I discussed previously, the mobile cellular network defines a global naming scheme that makes it possible for any two RAN-connected devices to communicate with each other. You need to configure both the small cell radio and the Mobile Core software stack so they know how to plug into that global network. This means defining the Mobile Country Code (MCC) and Mobile Network Code (MNC) that you plan to use. This MCC/MNC pair forms a Public Land Mobile Network (PLMN) code, where we’ve used two different ids in different settings: 315010 constructed from MCC=315 (US) and MNC=010 (CBRS), and 00101 constructed from MCC=001 (TEST) and MNC=01 (TEST). And since you’ll technically be the MNO responsible for the Private 5G network you bring up, you’ll also need to burn the SIM cards that are to be inserted into all the UEs. The SIM cards include a unique identifier (called an IMSI), which is a 15-digit number with the PLMN code as its prefix. (You can buy a 5G SIM writer on Amazon, where one product description reads: PLS Kindly Note: The cards be provided to professional engineers, PLS be professional, you need have knowledge about sim cards, if you don't have, PLS do not buy it!)
Finally, in step four, you’ll be back in familiar IP-land, but your ability to juggle IP subnets, Linux bridges, and iptable rules will be taxed to the max. I won’t go through all the details, and your mileage will vary depending on how deeply you want to integrate the RAN into your enterprise network, but by my count, there are as many as seven subnets in play. This is in part because the Mobile Core is implemented in Kubernetes (with its own set of intra-cluster and service-visible addresses), in part because the backhaul that connects the small cell radios to the Mobile Core is an overlay network (for example, running on top of your local enterprise network), and in part because the forwarding plane of the Mobile Core—the User Plane Function (UPF) running as a Kubernetes-hosted microservice—is itself an IP router that forwards packets between the RAN and the rest of the Internet. You’ll certainly find that having access to diagnostic tools such as ping, traceroute, and tcpdump to be essential (which is one reason we recommend connecting at least one Pi+Dongle UE).
I’m pretty sure Wi-Fi configuration was never this complicated. To some extent, this may be due to where the line is drawn between the customer and the provider: Telcos have strived to keep the end-system they sell subscribers simple, but have accepted operational complexity in the network devices (such as base stations) that they manage. In contrast, anyone who purchases a Wi-Fi AP from a vendor assumes it will be straightforward to install. One would expect that, with time, small cells deployed in enterprises (and maybe even homes) can be pre-configured before they are shipped or auto-configured after they are installed, but our goal with the book is to demystify 5G, including all the configuration steps. If you’re an enterprise system admin (or a hobbyist who wants to try out the technology at home) you will need to know about all of this. That’s why we wrote the book! It’s also why it’s important to have access to open source implementations of all this technology.
As noted above, our Private 5G book is close to completion. Not only does the appendix now describe our own hands-on experience, we also just received a very kind foreword from networking pioneer Jennifer Rexford. Congratulations to Jen on her recent ascension to the position of Provost at Princeton. The final step before we can commit the book to print is a last round of bug-scrubbing. Once again, we are offering a free copy of the book to anyone who submits two or more accepted bugfixes, and a thank-you in the Acknowledgments to everyone who submits a fix.
You can find us on Mastodon here.