Why Is Network Automation So Hard?

Ok, so automation can break your network. There’s no denying that fact. However, we’ve never needed automation to break our networks in the past. We’ve been quite handy at making silly errors that created spanning tree loops or routing black holes, disabled interfaces, or crashed important network devices. If you’ve not made any of these mistakes, then you must be new to networking.
To counter our silly errors, we’ve created heavy-weight processes that we hope will help us identify mistakes before they get to the network. We use peer reviews and change review boards to help prevent big mistakes. We use configuration/back-out procedures to help us undo an error when it does happen. Change windows hopefully give us time to make the change, test it, and back it out if it fails to perform as we intended.
Networks Are Complex
Networks are a complex combination of protocols and technologies, often with non-obvious interactions that must be understood. But the speed of business creates time pressures that don’t allow for understanding all the implications. Plus, training from network vendors has traditionally focused on protocols and how to configure a single box at a time. The training labs typically have a few devices, focusing our attention on how the features and configuration commands work. We then apply that view to a much bigger network, where things are a bit more complicated.
We also find that rumors about past problems persist. A good example is the rumor that you must configure the Ethernet interface duplex setting. The problem originated from some devices that didn’t implement speed and duplex negotiation correctly. Those days are behind us (well, unless you have some 20-year old devices in your network), so there’s no reason for this rumor to continue getting mindshare. Another rumor is that lots of bandwidth is a good replacement for prioritization of time-sensitive network traffic with QoS. To respond to this rumor, I’d advise that you go research microbursts and bufferbloat.
Why Are We Shy About Using Automation?
We’ve broken the network before with a single command, and there’s nothing preventing us from breaking everything with automation. We are carrying a lot of old baggage of past mistakes into the future, but new tools require learning new methods to optimize their use. A good tool, used incorrectly, is worse than a mediocre tool used well.
We’re not software developers, and we may not have programming skills. There’s a software development methodology called pair programming in which two people work together to create software. A modification of this process that involves pairing a network architect with a software developer will allow us network engineers to learn new skills while allowing us to teach software experts how networks operate.
Lastly, some of us are getting towards retirement and don’t want to learn something new (yes, I’ve heard that from network admins from time to time). I don’t have a good recommendation for these individuals. Maybe we can find something for you to do. Or maybe not…
Moving Forward
We must continue our education. We need to learn how to use our knowledge of devices and systems to design, build, and operate large networks using automation.
We’ve developed design rules of thumb that allow us to build stable networks without needing to understand all the details and implications. Vendors give us designs that have been vetted in real networks and we need to use them. Use simple building-block designs for different parts of the network. Bandwidth is inexpensive enough that we can afford to use one or two designs for all remote sites. The same applies to data center pods – make them all the same design. Learn how organizations are switching their networks and devices from being pets to being cattle (see "The History of Pets vs Cattle and How to Use the Analogy Properly").
I can hear the response now: But my network is unique!
If your network is truly unique, you should expect to have unique problems. But I’ll guarantee you that your network is a lot like many other networks across multiple industries. Some of the details are different, but the need to transfer files, carry on voice and video calls, and interact with applications are all the same. This is simply a larger instantiation of the pets vs. cattle analogy. Simplify network designs so that repeatable automation processes are easy to use.
Why Use Network Automation?
Automation brings a lot of benefits:
  • Fewer human errors
  • Consistency of configuration
  • Greater efficiency
  • Stop doing tedious functions (automate them)
  • Zero-touch provisioning
There is an interesting paradigm shift occurring with automation. It’s the switch from hardware infrastructure to infrastructure as code. We use software to create abstractions that decouple what we want to do from the implementation details. This makes us more efficient and we make fewer mistakes. That’s why we must transition to using network automation.
Lots to Learn
Yes, there is a lot to learn. That’s what makes it exciting. The time to start is now.
Join me at Enterprise Connect on Thursday, March 21 at 8 a.m. (yes, the start of the day) for my session, “Will Automation Break My Network Faster Than I Can Fix It?” It promises to be an interesting session, as we cover these topics and how to get started with automation.
If you haven’t yet gotten your pass for Enterprise Connect, coming to Orlando March 18 to 21, there’s still time! Register now using the code NJPOSTS to save $200 off your pass!