A Conversation on the Past, Present and Future of Networking

Earlier this month, Bruce and Larry sat down over Zoom to discuss their shared history researching and developing networking, how "Computer Networks: A Systems Approach" came to be written, and their thoughts on the future of Networking. The interview is on YouTube and the transcript follows.

Bruce Davie: so I'm here today with Larry Peterson. I'm drinking coffee as is my normal habit. I'm guessing it's bit late in the day for you for coffee.

Larry Peterson: Well it isn't too late for a brew of some kind.

Bruce Davie: All right, well, if you want to go and pop up and get a beer, feel free. So, for those of you who don't know, this is Larry Peterson, my co-author on "Computer Networks: A Systems Approach" and Larry and I have known each other since I think early 1990s, when we collaborated on a networking project.

I guess before I go into giving more background about you. I want to know that your cat since the cat theme been showing up a lot

Larry Peterson: He's going to be interrupting here at some point. This is Toby. If you can see him. Yeah, so we live in the desert, which is full of coyotes. Toby's used up a couple of his lives right outside in our backyard. There's a wall on the other side of that is open desert. And when he was a few months old a pack of coyotes came in the yard in and got him and took him over the wall. My wife went out and started yelling at the coyotes. We think he has some coyote DNA now.

Bruce Davie: So I have a lot of literally warm memories of visiting you in Arizona. Also remember working with you on a joint research paper sometime in the early 90s and I was locked in my house during an ice storm in New Jersey, and it was literally unsafe to go outside, because there was so much ice on the roads. You were probably soaking up the sun in Arizona. But it was one of those early experiences, similar to what we're going through now where we actually could leverage networking to collaborate from a pretty remote distance.

So I think you've actually been in networking longer than I have. And your ageing very well, by the way. Can you maybe tell me a little bit about your early experiences with networking?

Larry Peterson: Yeah, let's see. Well, I was a grad student at Purdue. I remember the transition from the ARPANET to the Internet and my advisor actually handed me a nine track track tape for our VAX. That had this thing called TCP/IP on it so, yeah, that was my first exposure to it.

I was just thinking about that as you were talking about having been connected from home. You also getting connected from your university back in the day when I first moved from being a grad student to the University of Arizona, and at that time, except for a few DARPAnet Internet sites, most universities weren't on the Internet. And so the first thing I did was get us on a 9600 baud leased line network called Cypress that connected to the internet. And that was the first time a lot of universities got connected to the Internet in some way other than login in to CSnet to get your email.

That was a long time ago - 9600 baud.

Bruce Davie: Okay. And so, I think you and I worked on this project to build what was at the time a high speed network, that was supposed to operate at about one gigabit per second, which, you know, was pretty high speed back then in the 90s, and I remember after having done that for a bit, that was how I managed to get my winter trips out to Arizona to work with you and your grad students.

Larry Peterson: We have a thermal networking lab. I don't know if you remember

Bruce Davie: I do actually, yeah.

Larry Peterson: We had a can of freon that we were spraying on the board that you built - the ATM NIC.

Bruce Davie: I remember having a piece of hardware that was working perfectly in the cold weather in New Jersey, but stopped working in the heat of Arizona.

Larry Peterson: We had to cool it down to keep it running.

Bruce Davie: Yeah. And so I think it was in about 1995 that you were starting work on the book that would eventually become "Computer Networks: A Systems Approach" - and realized maybe that you'd bitten off more than you can chew. And that's my recollection of how I came to be your co-author.

Larry Peterson: Yeah. I definitely bit off an awful lot. And yeah, I think we met at a conference where we started to collaborated on the gigabit testbed at the time and figured out a way to divvy up the work and get it done. That was great.

Bruce Davie: Yeah, and honestly for me that was one of the great early experiences, having to learn networking well enough that I could explain it to somebody else. And I remember working very long hours in that period to get the book out because, as I think we both discovered, it's a ridiculous amount of work to write a book. And that book in the first edition was like 600 pages or something.

And and so I always would be slaving away late at night and on the weekends thinking it's going to be so great when this is done and people can read this book and actually learn from us without us necessarily having ever met them in person.

And here we are 25 years later and the book's heading into its sixth edition. So I guess that was a good investment.

Larry Peterson: Yeah, absolutely. You know, I still find, till you can explain it to someone, you don't really understand it yourself - and even in what I do now, working with a lot of engineers and open source software. They build wonderful systems but it's when they reduce it to the tutorials and write it up that they refine their thinking and explain it in a way - and that influences how they build the system as well.

Bruce Davie: Yeah, I guess the other thing I figured out is you can get away with a few mistakes when you're explaining something to a person one-on-one. But if you put a mistake into print, you're absolutely going to find out about it. I made some absolute clangers when I was first learning about security, but they were very quickly pointed out to us. Once the book was in print.

Larry Peterson: The good thing is that there are plenty of people to tell you where you went wrong.

Bruce Davie: And then fortunately, you can reprint. Not like we're doing brain surgery.

Larry Peterson: Well, in terms of getting content right now - we used to do editions. I don't remember - once every three years or so. Maybe it was even more than that. But since we've gotten the source to the book and put it online, we now are doing agile book writing so every time I find a mistake or I figure out something I could say just a little more clearly. I'm able to get it up on the web, make the edits, compile everything, and it's up on the Web in a matter of minutes.

Bruce Davie: Yes. I mean, it's great that we've got the entire book online now and I think with both of us having had some experience with open source software, I think I had this realization in about 2012 that open source was the right model for a textbook, as well as for software and it took a couple of years to persuade the publisher of that viewpoint, but I think we've definitely got some benefits, now, having a the book available freely online.

Larry Peterson: You know, Bruce, I think it's time to get the cat out of my lap here.

Bruce Davie: Okay, sure.

[Larry exits, returns with a beer]

Larry Peterson: Cheers

Bruce Davie: You really made me jealous. It's a little early for the beer here in Australia.

Bruce Davie: Alright, so I guess I wanted to ask you what you're working on these days. I mean, aside from, you know, sixth edition of our book, what networking projects you're working on.

Larry Peterson: Yeah, well I officially retired with emeritus from Princeton a few years ago and I've been with the Open Networking Foundation since then. They're kind of the heart of the SDN movement, you know, they were spun up out of Stanford, Berkeley and Scott Shenker & Nick Mckeown were principals in that. The ONF is doing open source software and it's really trying to push the edge of what's possible with SDN (software defined networking). The place that we're pushing it right now is into the cellular network, into 5G. So it's been interesting. I had to learn a lot about 5G or just about the cellular network. If you go back and look at our early editions of the book we never really talked about it. It was just a little bit of an aside, and so it's an entire new space. There's an entire network that's been out there, just as long as the Internet has and has evolved in its own ways, independent of everything that we know of the Internet and the IETF. You kind of think of the access network as part of the last mile, but it's really going to become a more integral part of of the Internet. Maybe you could argue it will be the most important part of the Internet going forward.

Bruce Davie: Yeah, probably at this point some pretty significant percentage of people accessing the Internet are doing it over the cellular network if not yet over 5G.

Larry Peterson: Yeah, yeah. And it's not just the cells that are part of it, of the 5G part of it, which is hugely important, but it's the way the Internet has flattened, and so you and I are not connected directly to each other over the Internet. We are both connected to our nearest cloud site. And so it's basically that hop from my on-ramp to the cloud, and your on-ramp to the cloud, and the cloud is doing its own thing behind the scenes with its own backbones. And so the only piece left is the access network. And that's increasingly cellular and 5G.

Bruce Davie: Yeah, it's an interesting point there. I do want to touch on how things have been going since we're both, like most of the world now, we are really dependent on aspects of networking to get us through this period of lockdown. And, so here we are today on a Zoom conference and as you say, it's not like there's a direct internet connection from me to you. Actually we're using a cloud service, which apparently is working extraordinarily well. And so I guess you and I have been involved in trying to get networks to work since a point in time when they really didn't work so well. You know, we certainly weren't very successful in doing video conferencing, you know, 25 years ago. What do you think has changed in the 25 years to make video conferencing so effective.

Larry Peterson: Well, I mean certainly having the bandwidth to do it. This has helped a lot. But I think it's probably mostly just because of the coding speeds, the capabilities of the edge devices. We just never thought we could run the encoding fast enough to really do anything practical.

Bruce Davie: Yeah, again you're talking about things we had to learn for the book, like, understanding how video coding works. I think that continues to get better. And that there were people who understood coding and there were people who understood networking. There were very few people who understood both. And I think that's one of the things which I think is really extraordinary today even compared to five years ago. That the way to do video and audio coding in a way that sort of adapts to the things that don't necessarily go perfectly on the internet.

You know, there's a lot of latency between you and me right now. It's just unavoidable, given the speed of light, and that latency varies and yet we're able to converse reasonably comfortably.

And I think a lot of the things that are making that work are: When the packets get delayed or dropped this adaptive coding is going on to make the experience as a user pretty seamless.

Larry Peterson: Yeah, and it's optimized for voice. A lot of people using Zoom for a lot of different things right now so, you've seen concerts and music and music teachers teaching their students over Zoom. The voice codecs don't work so well for that, but I just learned myself because I was trying to use Zoom like that with my son, and there's a setting on Zoom - I forget what it's called, but you get out of the voice codec. And you can actually get a much wider range of audio out of it.

Bruce Davie: Ah right, I didn't even realize that but, yeah, coding is a very advanced science now and coding for voice is a different thing.

Larry Peterson: Incredibly optimized. I don't know an awful lot of it.

Bruce Davie: Another area where the Internet's been absolutely priceless in lockdown has been supporting entertainment video which again is a different set of tradeoffs versus video conferencing.

Bruce Davie: I actually remember Dave Sincoskie, who we both worked with back in my first job at Bellcore, wrote a paper about whether video on demand was technically possible. And I think he would have written this right around the time that you and I met in 1990 or so. And it was a very bold paper, suggesting that maybe Video on Demand could work across the Internet, of course, today, we absolutely take that for granted. But it's not that long ago that Netflix was shipping us DVDs in the mail, because that was a more efficient way to get content to us then streaming it over the Internet. Today it works incredibly well.

Larry Peterson: Yeah, I shouldn't confess this but I was an expert witness once in the federal district court trial of the folks that ripped Divx if you remember that, and I testified as to how bad congestion was on the Internet and there wasn't any real practical way that any harm could be done because who could possibly transmit videos over the Internet.

Bruce Davie: Yeah. And this is a story I've heard about the DVDs versus streaming going back to a textbook that probably you and I both learned from, where there was an exercise where you can take a magnetic tape and have it carried around by a St. Bernard dog. And the idea was that the dog carrying the tape was actually moving data faster than most networks. And I think that was the idea of Netflix - it actually was faster to take a DVD and stick it on a truck in terms of the bits per second, rather than actually streaming the bits over the network.

Clearly that equation changed in the last few years, and these days, obviously, video streaming over the Internet is just extraordinarily good. It's partly because the bandwidths have gotten better. But again, there's some pretty interesting stuff with the relationship between coding and congestion control again in the video streaming world.

Larry Peterson: Yeah, well there's obviously a lot going on there. It makes you wonder what will be next, and we're now in a video mindset, with video streaming and conferencing. But, you know, will that become passée and will there be something else that overtakes it in importance?

Networks have latency and sometimes not totally predictable latency. If you're going to do mission-critical things maybe it's not the bandwidth, maybe the density of the devices communicating, their mission-critical, real-time latency requirements.

Bruce Davie: The issue of latency comes up a lot in things not related to entertainment, although I guess gaming is one where latency comes up a lot - you definitely don't want to be firing your trigger and having a half-second delay before the missile fires or whatever. You know, I used to be very much into gaming when I was a student before I met you and I realized that I should give up - let's say I had a bit of an issue. So I just stopped playing video games at about age 21 and I've never gone back because I'm afraid I would just lose all that time again. But obviously, you know, gaming is one of those areas where the desire for low latency comes up. You hear stories too, I don't know if you get this in your house, but you hear stories about competition for the bandwidth between the different members of the family, the kids playing games...

Larry Peterson: I've experienced that - one of my kids was in Japan for five years and came back and moved back with his folks for six months, setup his gaming system and yo, the network.

Bruce Davie: So, one of the things that you and I worked on was quality of service. I'm still ambivalent about how that really played out. But, you know, clearly, that was one of the use cases - you could control the allocation of bandwidth between, say, the important work that dad's doing on his computer from home and the less important work that the kids are doing on the game, or maybe vice versa, depending on your priorities.

I feel like an enormous amount of work went into trying to build mechanisms to get QoS into the Internet. And today, by and large, they're not being being used, but I don't know, maybe it's different in the part of the network where you work?

Larry Peterson: Well, yeah, it is. And I've always been one of those people that believe, maybe if you just give it a little bit of time the network will get fast enough and the endpoints can adapt. And we certainly saw that with Netflix, they really changed the equation in terms of the ends adapting instead of trying to do QoS in the network. I still have to be completely convinced because all my biases point me the other way. But I think in the cellular network might be a different game just because of what a scarce resource it is. And again, as you start to try to mix bandwidth with predictable latency. And the scheduling is where all the intellectual property, the secret sauce and magic happens.

Bruce Davie: One of the topics that's come up in terms of how well the Internet's been doing during lockdown is the fact that we've seen people engineering it to deal with quite unpredictable things, and the sense that putting in those QoS mechanism was all about trying to make things really predictable. And what a lot of people figured out was, you know, it's just easier to provision extra capacity. And it's kind of an interesting comparison between how the internet is run versus almost any other system that we deal with. You know, there isn't tons of extra capacity sitting out there in the interstate highway system, or in the hospital system. There is not excess capacity because, you know, it just feels too expensive, but yet the internet has tended to be built with this kind of excess capacity. With the results that it's there for the really, really busy hour, which I think, you know, once upon a time, that would have been probably people watching movies on Saturday night. A year ago, that would have been busy hour and now, probably, we're in the busy hour right now on the internet. It's when people are doing all this Zoom conferencing during the workday is probably now becoming the busy hour.

Larry Peterson: You know, I think that's probably right. That reminds me. What happened with Slack yesterday? Have you heard?

Bruce Davie: No, I haven't. I'm actually eagerly awaiting the post mortem on that because one of the things that's been impressive - all these cloud services that we all depend on - Zoom included - scale out incredibly well.

And I think this is one of those things where it's more in the realm of how software development has changed over 30 years that people build systems now to have unbelievable ability to scale. Seemingly Zoom has gone with barely a hiccup in terms of how well it's adapted to all of that additional load and clearly something didn't perform yesterday in Slack.

Larry Peterson: I depend on Zoom and I depend on Slack and I really felt it when Slack failed me. It will be interesting, but you're absolutely right. And I don't know: this is networking, it's cloud. It's cloud; it's networking. If you can't get more capacity of the network, you just replicate space and you put more of the Internet at the edge where you live. It's what the cloud is doing for us.

Bruce Davie: Yeah. I have to think that somewhere in Slack, there was some component yesterday that became a bottleneck. And, generally speaking, you build these sort of cloud native systems to not have any single bottleneck. And clearly something yesterday became a bottleneck.

Larry Peterson: Probably what became the bottleneck was some operator who made some configuration change.

Bruce Davie: Yeah, that's a good bet actually. I mean, as we know, a lot of the things that go wrong with networks tend to be where humans get involved and make ill-advised configuration changes.

I think that actually is one of the things that's been very interesting in the SDN world. How we've moved to a world where more and more of networking is software talking to software. If you think of all the things that humans tend to do wrong in networking - it's a very, very complicated configuration. I have to get it exactly right. And if I type one character wrong, I can bring down a big chunk of the network.

And so the more that you can have that task done by a piece of software - obviously software has bugs, too - but you can at least have things be more predictable and take out those sources of human error.

I remember being at a talk by one of the people from Facebook talking about the fact that he was absolutely confident that robots would do a better job of running these networks than humans. So I think that is one of the things that's happened with networking over the time that we've been involved in it - there is this idea that it's gone from being something where manual configuration done by human experts is increasingly being replaced by automation, done by software.

Larry Peterson: Yeah, it's interesting. It shifts the problem. So one way I think about SDN is it's transforming networking problems into a distributed systems problem and in that setting, one of the things that I keep coming back to is, it's not so much the the congestion control algorithm that's so important. It's not how you get reliability that's so important. It is how you operationalize the network and it comes down to how you configure it and how you track changes in the configuration.

That's where the center of gravity is in many networks that get built this way.

Bruce Davie: The other thing I was thinking about, and you kind of reminded me talking about Slack, is that we've been involved with the Internet since before there was a World Wide Web. And you remember that there was a sort of period of people trying out lots of different ways of moving information across the Internet. We used to do things like FTP and there were a few other things like Gopher, a bit more esoteric, and then at some point HTTP just became the dominant way to get stuff across the Internet and clearly we've benefited enormously from the success of that. But I think one of the things that I feel the early Internet architects and pioneers really got right was the idea that the Internet should support any service, even all the ones we haven't thought of yet. In contrast, for example, from how the telephone network was built to do phone calls and the cable TV network was built to do television, the internet was built to do anything. And, you know, going all the way back to email, there have been all these ideas that nobody had at the time that turned out to be incredibly successful.

I think Slack is one of those examples - who would have thought that messaging could really still be an area of innovation in 2020? What do you think about how well the Internet's done for enabling innovation?

Larry Peterson: Well, you're absolutely right that it was a general purpose thing and that's what I want - as I learned more and more about the evolution of the cellular network and you look as transitions from 2G to 3G to 4G and so on, every one of those was just adding maybe a little bit more degree of freedom to what we consider a specialized network to do. And that's actually where 5G is making the most important change because it now says it's just part of the cloud. They don't say it that way but that's really finally coming to the realization that it's no different than anything else. It's part of the cloud. And then at that point, then you can start to bring all these other best practices and building new functionality into play.

Bruce Davie: So I wanted to just talk a little bit about where we think things might go in the future. Once every year, roughly, I write some kind of predictions post and I'm feeling pretty good about the one I wrote this year because I actually went back and looked at Bob Metcalfe's prediction that the Internet was going to collapse, which was, I think, sometime in the 90s - about 96 or so. You know, Bob Metcalfe is an absolute legend in terms of his contributions to networking, but on that particular occasion he made a very bold prediction, which was spectacularly wrong.

So I wrote this post a few months back, sort of predicting that the Internet would continue to be resilient and continue to evolve to meet emerging needs, which maybe wasn't such a bold prediction. But it turned out to be a bit bolder than I realized, given the amount of stress on the Internet. So I'm feeling good - I am happy to be right and then I'm really happy that the Internet's working right now.

So, anything you would care to predict in terms of what we might see in the next few years of networking?

Larry Peterson: Well, I can do a stretch prediction. Not so much because I'm pretty sure it's going to happen, but I would like to see it happen. And that is, it goes back to SDN a little bit, which is people are now programming the network data plane and maybe you know about languages like P4. And what interests me about that is that the network device becomes commodity, just like the PC became a commodity.

And you get to program it with a pretty general purpose programming language. P4 has specialized a little bit, but it's a lot like C, and now, anyone can do it. And it really, really shifts control the network. I mean, right now we're in the transition where we're trying to get control moved from the vendors to the operators, but it really belongs from the operators, all the way to the users - not your average unwashed masses users - people that have innovated and written interesting code and useful applications on the x86. It really is going to open the door for them and, this is the kind of "in the limit" prediction. I would never put a date on this, but eventually we're going to have different kinds of processors. Some of them are really good at AI. Some of them are really good at shuffling packets from input to output, some of them are really good at polygons. And they're just going to be a bunch of processors out there and they're all going to be programmable, and that's just going to completely change the way we think about the network.

That's so far in the future. I'll never be held accountable for it, but...

Bruce Davie: One of the things that I feel fairly good about is that most people these days don't think too much about what the network's doing. I feel like we've gone from the point where, once you had to be some kind of wizard to even get connected, to one where it's just a service. It's just sort of a commodity that we pretty much take for granted, except of course when it goes down.

One thing that I've found interesting when I moved to Australia was that we've got a much more competitive market for Internet service providers here than, say, in the US, because of the way they designed the network here. So when I decided to get broadband service in my home, I had a choice of about 75 different Internet service providers.

Because of the fact that there's one company that has the right to bring a physical cable into my house, but then, on the order of 75 different companies have the layer three service that runs on top of that. And that's been really interesting to see how that's played out, in terms of the operators having more control over what kind of service they give you.

They can do things like just offer a larger amount of capacity, they can offer you things like - I wanted to make sure that I could could not have double NAT done to my IP address, which is a pretty exotic thing to care about. The ISP sent me an email saying "we're introducing carrier grade NAT. If this bothers you let us know." And I sent them a note saying "yes this will bother me, it'll stop a few things that I depend on from working." And it's like, wow, whoever thought of an Internet service provider caring enough to ask you about carrier grade NAT.

Larry Peterson: That's fantastic.

Bruce Davie: You know, the set of people who believe that's a good feature was very small, but I've got to tell you, that's one of the ways which my ISP won my heart. That they didn't just go and introduce it and not tell me; they told me it was coming and asked me if I cared.

Bruce Davie: And then a simple matter of an email explaining why I cared to get that feature not turned on for me.

Larry Peterson: I'm envious.

Bruce Davie: Yeah - I always like to talk about how great the broadband is in Australia, because almost every Australian thinks it's terrible. And it's just because there have been some aspects in which it hasn't been what we were originally promised. But that aspect of competition between the ISP. It's so different than what I had to experience living in the US where I think I had a choice of two ISPs - effectively a duopoly. I always like to say, well, maybe not everything's perfect but they got that one aspect of it right.

Larry Peterson: Yeah, that's fantastic.

Bruce Davie: So, look, I think we've had a pretty good, long chat here. Any parting thoughts on the world of networking, before we wrap it up.

Larry Peterson: Well, as someone who - we both worked on this textbook for many, many years. And part of that was about training the next generation. So I guess the only other parting thought I had is that there are a lot of changes coming and I know that we've been thinking about how we teach networking and how that needs to change.

I think the old style served us very, very well. And it didn't really matter if you're going top down or bottom up. It was kind of the same story, a little bit different packaging. I think the story is on the verge of changing substantially. I don't know exactly how but I do think that change is coming. As people who are interested in teaching that, we have to figure out the right way to deliver that material.

Bruce Davie: Yeah, absolutely. And clearly, just right now, we're all living through a very strange time in terms of how people learn with online learning becoming almost a requirement and after a few false starts with online learning in the past. I think we've got to figure out how to get that to work.

So I guess this is the next project for you and me is we figure out how to train the next generation and some method other than just turning out a textbook every few years.

Larry Peterson: Yes. That's exactly right.

Bruce Davie: All right, well, Larry. I'll say thanks very much. And I'm bit envious of the beer but I'll hold off for a few more it hours. It was great to see you and we will talk again soon.

Larry Peterson: All right, bye.