Service Mesh: SDN for Layer 7
In our most recent post we made note of the common pattern of centralized policy input coupled with distributed implementation of policies, and this week’s issue follows that thread into the world of Service Meshes.
I remember when I first heard about Service Meshes in 2017, and wondering what the big deal was. Building cloud applications as a graph of microservices was commonplace, and Telcos were hard at work inventing yet other ways to chain together virtualized network functions. Service graphs, service chains, service meshes… how many ways do we really need to talk about composing complex systems from a collection of smaller components?
It wasn’t until I recognized a familiar pattern that I got it: A Service Mesh is just SDN at Layer 7. That’s probably what happens when SDN is the hammer you keep hitting nails with, but I’ve come to believe there is value in that perspective. The figure highlights the similarities between the two scenarios, both of which include a centralized controller that issues directives to a distributed set of connectors (physical/virtual switches in one case, and a sidecar container in the other case)—based on a combination of policy intents from above and monitoring data reported from below. The primary difference is that the SDN controller on the left is controlling L2/3 connectivity and the Service Mesh on the right is controlling L7 connectivity.
Comparisons like this often break down at some point, but for me, identifying the differences between the two cases also helped me understand the opportunities in this space. In short, these two cases can be viewed as two ends of a spectrum, with each making a different performance-vs-expressibility design choice for the “connector” elements. Sidecars can run arbitrary code, and so implement any imaginable service connectivity policy, but the biggest knock on sidecars is that bouncing all traffic through an intermediate container results in a non-trivial performance hit. Physical L2/L3 switches have forwarding rates measured in terabits-per-second, but support limited/fixed functionality (e.g., coarse-grained ACLs). P4-programmable forwarding pipelines present an opportunity to offload some sidecar functionality to the switching fabric, but the best opportunity to find a best-of-both-worlds design point is in virtual switches and SmartNICs. Note also that the functionality of a sidecar generally needs to be close enough to the relevant service to see the actual RPC messages, which tends to rule out network devices that will only see encrypted traffic between hosts.
All of this brings us to a topic that is attracting a lot of attention, which is to optimize service meshes using a combination of eBPF (extended Berkeley Packet Filter) and XDP (eXpress Data Path). When used together, they provide a way to program generalized Match-Action rules in the OS kernel (as part of a virtual switch), or alternatively, on a SmartNIC. That eBPF/XDP can be viewed as an alternative implementation of OpenFlow/P4-inspired flow rules is not a coincidence; there is something fundamental about Match-Action rules as an abstraction for programming (and controlling) end-to-end connectivity. Having identified this commonality, the differences are again helpful: eBPF/XDP allows (mostly) general code, while OpenFlow defines a fixed set of Match-Actions and P4 is a restricted language for expressing the same. This is necessary when the Action must execute within a fixed cycle budget, as is the case for a switch-based forwarding pipeline. It also enables formal verification of the data plane, a promising opportunity being pursued by the research community.
It turns out I wasn’t the only person to make the connection between SDN and Service Meshes—here is Bruce’s version from two years ago, and VMware’s service mesh product clearly has parallels to their other SDN offerings. In my experience, there is enormous value in recognizing commonality and defining unifying abstractions across seemingly disparate implementation artifacts. Unifying abstractions are the basis for building better systems. Acknowledging the power of a centralized policy engine (e.g., the role of the SDN controller) is one such abstraction (which we also noted in our recent security post). The fundamental nature of Match-Action rules as a way to specify forwarding behavior (e.g., the role of OpenFlow) is another. Recognizing that Envoy sidecars, eBPF/XDP kernel modules, and P4-programmed pipelines can be viewed as three implementation choices for programmable forwarding engines used to build end-to-end service connectivity is an intriguing opportunity that deserves more attention. Successful platforms build on the abstractions that have proven useful in the past. And that is a key tenet of the Systems Approach.
One of our favourite talks about Service Mesh is by Thomas Graf introducing the Cilium project, an eBPF implementation of Service Mesh. It’s also hard to improve on the fermented fish analogy provided by Louis Ryan of the Istio Project. There is so much going on in this space that we’ll be sure to come back to it in another post.
In other news, we did a podcast recently with Ben Pfaff (of Open vSwitch fame) which was (for us at least) very interesting, but also led to an unexpected and delightful spin-off episode featuring a small amount of Systems Approach and a lot of Mark Twain. It was the most fun podcast we’ve heard in a while.
Hi Larry!
Great to see you looking into popular stuff with a CS fundamentals approach.
Looking service meshes as a forward/policy system is a somewhat limiting view though. We are using a LinkerD based mesh and our main benefit is the excellent observability it provides out of the box.
Our services receive large input data such as documents or sound files, and perform some computationally expensive tasks (preprocessing, extraction, conversion, machine learning, etc) amongst regular database/cache or other service accesses.
Even our ML researchers can take a look at the visualization LinkerD dashboard generates and quickly see which transaction processed through which paths, where were the high latencies or high error rate links etc in a very large cloud system.
Now, all of these can be custom built, but it is a lot of work, especially when you have hundreds of services built by teams where their main competence is not necessarily the network/cloud.
This also wouldn't work well in anything below L7 as the information becomes less accessible to people, and cannot capture what is going on in today's multiplexed protocols (http2, grpc, etc).
Service mesh is just at the right level to capture meaningful information and integrate easily with other tracing/metrics systems.
LinkerD's L7 proxy performance is more than enough for us, even on a beefed up GPU machine, service can only process a few hundred transactions per sec anyway and we can scale vertically instead. If you are looking for 10k/sec+ transactions with minimum machine cost, micro service architecture is not for you anyway.
There are other benefits for sure (policies, upgrading connections to https, providing a in-cluster CA, etc etc). But to me, observability out-of-box dwarfs everything else.
Best,
Gurer Ozen