How do you know if your UC system has problems? Do you rely on calls from your customers?
Anyone who has run a network has heard the typical story:
User: Oh, by the way, I had a call yesterday that was heavily garbled at times. I thought I'd let you know about it.
You: What time was it?
User: I don't recall exactly, but it was after lunch.
You: Who were you talking with and which phone were you using?
User: Well, it was more than one call, and I was on so many calls yesterday that I don't remember which ones. I was also in several meetings and some calls were from my desk, some from my cordless phone, and some from my cell phone.
You: Were they inside calls or calls outside?
User: I don't remember that either. I spend all day on the phone and I don't recall that level of detail.
So, there's a story about possible voice problems, but not enough information to conduct a reasonable investigation without impacting all the other projects on your plate. It could be something wrong with the phone system at the other end of the calls, it could be a poor cell phone connection, it could be the internal network, or it could be the PSTN gateway or session border controller. Or it could be the SIP carrier.
If you only had a systematic method for determining that voice and video calls are functioning at an acceptable level...
Causes of problems
Voice and video are surprisingly resilient to network problems that cause business applications to slow to a crawl. As long as packet loss is under one percent and jitter is small enough that the endpoints can adjust, it works well. Problems occur when there are bursts of packet loss or more jitter than the phones can handle.
One source of problems is interface errors, caused by bad cabling, cables that are run next to magnetic sources or RF noise (like fluorescent lights, high voltage power lines, or big uninterruptible power supplies). Fiber optic cables can also be a source of problems, either through dirty connectors or a pinched cable, both of which can cause enough power loss to create errors in received packets.
Another cause of packet loss is network congestion. Congestion occurs primarily at one of two places:
Identifying Problem Spots
One way to identify problem spots is to use the network management system reports for errors and drops/discards. Most management systems can create a "Top-20" report for discovering the interfaces that have the highest errors and drops.
Call quality reports from the UC controller can be another source of valuable information. For example, Cisco's Unified Communications Manager provides the ability to create reports showing call quality (referenced as call QoS in the documentation).
If no endpoints are reporting problems, then any voice quality problems are much more likely to be due to causes external to the organization. However, if there are calls with poor quality, then another level of investigation should occur. Was there a common geography or common set of links or endpoints involved? Or was a single carrier always part of the mix? I've seen a one-way call problem because the SIP carrier's IP address was not properly configured in the UC system. Interestingly, only one of two carriers had a problem with this configuration.
Solutions
There is no magic solution for interface errors. Simply put, the source of the problem needs to be identified and corrected.
Interface drops are quite a different matter. High volumes of drops (e.g., more than 100,000 drops per day or 6,000 drops per hour) are an indication of insufficient bandwidth. Adding more bandwidth is certainly the answer, but adding QoS will help prioritize the voice/video traffic. It helps to examine the traffic to determine if very low priority traffic exists that can be dropped by an appropriate QoS policy. We once found a T3 link (40Mbps) that was carrying about 20Mbps of entertainment traffic like music and popular movies. We used QoS to prioritize the business traffic over the entertainment traffic.
Low volumes of drops may simply indicate that traffic volume is closely matching the available bandwidth, and that micro-bursts are the cause of drops. A micro-burst is where a short burst of traffic over-runs the buffering available on an interface. QoS is the answer here, possibly with the addition of a moderate amount of buffering. Be careful of adding too many buffers as it can result in poor TCP performance for business applications.
What about Cloud UC?
Cloud UC doesn't really change much. The controller may be located at a cloud provider's facilities, but the endpoints are located within the enterprise. Voice and video transport is between the endpoints, so the cloud provider's systems are not involved in carrying the call data, unless it is a conference call in which the Media Control Unit is located at the cloud provider's facilities. Anyone looking at cloud UC should make sure that the provider can produce call quality reports that can be used to verify and troubleshoot any voice and video problems that may exist.
Follow-Up at Enterprise Connect!
I'll be taking a deeper dive into this topic in my Enterprise Connect session, "Current Challenges of Running Voice and Video on IP Nets," taking place on Monday, March 27 at 2:00 p.m. You'll walk away from this session better equipped to ensure that the voice and video traffic running over your enterprise IP network is receiving the proper treatment and providing end users with the experience they need.
I'll also be leading two panels on closely related topics:
Hope to see you there!
Learn more about systems management and network design trends and technologies at Enterprise Connect 2017, March 27 to 30, in Orlando, Fla. View the Systems Management & Network Design track, and register now using the code NOJITTER to receive $300 off an Entire Event pass or a free Expo Plus pass.
