"If I had an hour to solve a problem, I'd spend 55 minutes thinking about the problem and 5 minutes thinking about solutions."
-- Albert Einstein
I've been speaking at the annual IAUG (International Avaya Users Group) conference long before they called themselves the IAUG.
This year is no different; I am presenting two breakout sessions at the upcoming Engage 2016, one aimed at demystifying WebRTC and for the other, I'll be joined by my coworker and partner in unified communications crime, Dave Lover, to explore the art and science of debugging SIP on an Avaya Aura system (Session 705T).
Given the brevity of the debugging breakout session, I decided to write this article as both a teaser and a future study guide. Dave and I could talk for three or more hours on the subject and still have more to say -- and that's not because we are longwinded.... (which we are).
First, I need to emphasize that Avaya follows the SIP standards very closely. It didn't create new requests or responses. Nor did it change any of their meanings. A REGISTER is still used to associate a SIP AoR (Address of Record) with an IP address and BYE messages are sent to release sessions.
The same can be said for the standard SIP headers: A TO header still identifies the recipient, and a FROM header identifies the sender. Anyone who knows how to read a SIP message will have no problems looking at Avaya-formatted INVITE, PUBLISH, or OPTIONS requests.
However, looking through an Avaya SIP trace is not as straightforward as you might expect it to be. That's because nothing is truly point-to-point in an Aura system, and components such as Communication Manager and Session Manager insist on having a say in how SIP messages are processed. In particular, Avaya's Session Manager sits in the middle of all transactions and acts as something of a SIP traffic cop.
Without going into the gory details and pointing out the exceptions, Avaya implements half-call processing on incoming and outgoing SIP calls, which means that the calling party is handled independently from the called party.
For example, when Andrew calls Debbie, Session Manager sends the call to Communication Manager to be processed from the standpoint of the caller (Andrew). Upon completion of its work for the calling party, Communication Manager sends the call back to Session Manager, which then bounces the call back to Communication Manager to be processed from the standpoint of the called party (Debbie). As before, Communication Manager sends the call back to Session Manager, which will then determine the next hop for the call. This might be Debbie's telephone or it might be another SIP service such as a voicemail server.
If you were to trace this call flow, you would see INVITE messages going back and forth between Session Manager and Communication Manager a minimum of four times. If other SIP components such as Breeze or an Avaya Media Server are involved, four can become six, eight, and even more.
On top of IMS processing, Session Manager and Communication Manager implement resource reservation with SIP INVITE, a mechanism that introduces even more packets flying around the network. For the uninitiated, this can be very confusing. You expect to see one INVITE and wind up seeing a half dozen (or more).
Any guy or gal who is experienced in SIP knows all about Wireshark. This free application is the most common packet tracing tool and is used to debug every shade of VoIP problem. However, Avaya's best practices dictates that SIP messages should be encrypted and, therefore, unreadable. Technically, Wireshark is capable of deciphering and displaying encrypted packets, but it's a complicated process that requires information (e.g. security certificates) that is not readily available.
Thankfully, Avaya created a number of tools that can decrypt those TLS packets while formatting SIP messages in an extremely readable fashion. Different tools are used depending on where you are debugging and what level of information you are looking for.
Personally, I am a big fan of traceSM. In fact, one of the most popular articles I've ever written for my blog, SIP Adventures, is "A Necessary Guide to the Avaya TraceSM Utility." Not only will traceSM display formatted SIP messages, but it can trace Avaya's proprietary protocol, Personal Profile Manager. Additionally, hardcore users can choose to get down and dirty with Session Manager's internal debug messages.
If you are debugging messages that flow in and out of an Avaya Session Border Controller, you will want to use traceSBC. As I did with traceSM, I wrote the definitive guide in A Necessary Guide to the Avaya traceSBC Utility.
Lastly, Communication Manager has its own trace tool that can be used to view things from CM's point of view. This may be the most cryptic of all Avaya's tools, but it just might provide the most insight for a problem that is centered on how CM processes a SIP call. A typical invocation will look similar to the following where xxxx is the station number/telephone number.
list trace station xxxx/s
If this is the kind of geekiness that makes your heart beat a little faster, please join Dave and me as we explain as much as we can about debugging Avaya SIP in 60 minutes. We will also touch on Avaya subscriptions, authentication, System Manger configuration, dual registration, booting an Avaya SIP telephone, and a host of other important subjects. Pay attention, though, because those minutes will fly by quickly.
While I can't promise that you will leave our session fully equipped to handle every problem that might come your way, you will understand enough of the basics to know where to point your compass to start. I struggled the first time I had to delve into a complicated call flow with traceSM, but I lived to tell the tale, and so will you. All you need is a little knowledge and the right map.
Andrew Prokop writes about all things unified communications on his popular blog, SIP Adventures.
Follow Andrew Prokop on Twitter and LinkedIn!
Andrew Prokop on LinkedIn