Securing the network itself is just as important as securing the servers and applications. Automation is an essential component for assuring that the network is safe and secure.
What’s on the network?
You can only manage devices that you know are on the network. Some organizations have processes that include steps to add or delete devices from the network inventory, but these processes don’t necessarily guarantee that all devices are found and registered. For example, the existence of a workplace process doesn’t eliminate the possibility that a device was connected to the network without following the process or without the network administrators’ knowledge.
An automated network discovery mechanism reduces these possibilities and guarantees a reliable network inventory. A variety of techniques and data sources must be employed to provide full coverage. These include:
- Routing tables to show Layer 3 adjacencies,
- Switch forwarding tables contain Layer 2 adjacencies,
- Address translation tables which provide relationships between Layer 3 (IP) addresses and Layer 2 (MAC) addresses,
- Device discovery protocols (Cisco Discovery Protocol -- CDP or Link-Layer Discovery Protocol -- LLDP) which identify Layer 2 adjacent devices.
- Ping sweeps should be a last resort and aren’t viable on an IPv6 network.
A good discovery mechanism will correlate data from each of the sources to arrive at a thorough network inventory.
Once a device is discovered, the network discovery system should determine what type of hardware it is and what software it is running. Some devices will have to be identified by fingerprinting the software’s network characteristics while other devices will have network management interfaces that simplify their identification. A complete network inventory then becomes the basis of network security .
Automate vulnerability checking processes
With the network inventory in hand, you can now determine the attack surface that must be protected. Automation simplifies this task: You can determine if any of the hardware and software combinations contain known vulnerabilities by checking those combinations against PSIRT (product security incident response team) and CVE (common vulnerabilities and exposures) announcements.
Vulnerability checks need to be done regularly, often on a daily basis, in order to detect vulnerabilities that need to be patched before a bad actor takes advantage of them. Don’t forget to check things like container systems (Kubernetes and Docker) as well as any applications running on network equipment or in SmartNICs on servers. The output of this process should be two-fold: a report for the IT staff and a table that can be used by an automated software/firmware upgrade system.
When vulnerable devices are found, you’ll want to use automation to perform software upgrades. To keep from breaking your network, start by testing the upgrade in the lab. This means that you must have a lab that mirrors the production network well enough that the tests are valid.
Once the software upgrade has been validated in the lab, then roll it out to a selected subset of production devices and verify its success. Only then should you consider applying the update to the rest of the network. It makes sense to do the update in increasingly larger chunks as you gain confidence that the update won’t break something. Using this crawl, walk, run process limits the extent of a failed update, which is often described as limiting the blast radius of the change .
Configuration drift and audit
Separate from the operating systems are the device configurations. Configuration drift monitoring and configuration audits are essential to ensuring that good security practices are being followed. Configuration drift identifies all changes from day to day so that you can track the progress of all network configuration modifications. Configuration audit checks configurations against a known configuration policies.
T-Mobile suffered a major network compromise late last year because a router was left in an unsecure state that allowed the bad actor access to the network and subsequently to servers through lateral spread techniques. Configuration drift and audit functions could have helped prevent the intrusion.
At a minimum, automated configuration audits should verify access controls to each network device’s management plane, and these audits should include things like the command line interface, web interfaces, and device APIs. These audits should be conducted on a regular basis and when device configurations are changed.
But be careful that the access control configurations don’t create a situation where the network team can’t access the network equipment when a network failure occurs. Just look back at the Facebook failure last year for an example.
Configuration audit failures can trigger automatic remediation, pushing known-good configurations to devices that are modified outside of the configuration control system. While you can use non-automated mechanisms, these tend to leave devices vulnerable for longer while the network staff finds time to manually implement the change.
Develop incident response plans
It is important to develop incident response plans for different types of attacks. A denial of service attack will need a different response than a targeted spear-phishing attack that loads malware into a laptop. Having a plan ahead of time can save time and limit the extent of an attack, ultimately reducing the cost and disruption to the business.
Should an incident response plan be automatically triggered by automation? Perhaps in some cases, perhaps not in other cases. The bad actors are using automation and the delay of waiting for a person to examine evidence of an intrusion and make a decision may result in a larger or more serious system compromise. As with most things in networking, it depends.
You’ll want to develop automated data gathering system to determine the extent of an intrusion , particularly for malware like ransomware. It will help you determine which systems should be isolated and which are safe for continued use.
Testing, testing, and more testing
How do you know that your network is actually secure? There’s no substitute for testing. You can start with simple validation tests that should trigger alerts so that you know the security system is working. In addition use the above processes to verify that the network is configured according to your security policies. There are a lot of security testing products and it is often worth investigating finding suitable products instead of building and maintaining your own test suite.
How do you know that the network security policies are sufficient? Did the incident response plans work as designed? The validation testing may pass but any flaws in the policy can allow bad actors to gain access. The ultimate test is to hire an external penetration testing team to break in. If they are successful, you’ll want to know the mechanisms that were successful, the level of difficulty they encountered, and whether your security systems were able to detect the intrusion.
Follow-up at Enterprise Connect 2022