We spend a lot of time at VoiceCon/Enterprise Connect looking at the big picture and trying to figure out how to conceptualize and architect the next-generation communications architecture. At the same time, though, we're constantly reminded that actually running the real-time traffic on IP networks is not a trivial or settled matter.One of the big surprises at VoiceCon Orlando last month was this: In its time slot, our session on Network Test Tools drew the biggest crowd. We've been puzzling over what that means--in a conference where a lot of time was spent on big-picture issues as summed up in the new Enterprise Connect tagline, "communications transforming business," there was still a strong interest in what it takes just to make the stuff work, keep it working, and being assured that it's working.
I went back and took another look at the slide deck that John Bartlett prepared for the session, and it offered some clues, I think. In particular, two slides resonated. This first one is a simple description of the basic categories of what to test (signaling and media) and how to test them (actively or passively): 
Later in his presentation, John offers a rundown on which vendors offer which types of tools in these categories. Meanwhile, he followed up with a slide that elaborates on the types of tools out there, categorizing them by what they test:
These slides help illustrate the challenge of testing real-time communications applications on IP networks. As the second slide here shows, you're testing multiple things: The first bullet is basic IP network infrastructure metrics; the second is characteristics of the voice application, specifically its signaling mechanisms and performance, an area where the metrics largely come out of legacy POTS; and the third bullet is an area--quality of experience (QoE) whose heritage is in legacy POTS but whose metrics increasingly come out of new standards developed specifically for VOIP.
Given the diversity of the testing and monitoring challenges, I'd be very surprised if most enterprises already have all the tools they need to test in all of these areas; have them deployed ubiquitously; and have personnel who are comfortable running these tools ubiquitously.
And I suspect that this relative inexperience with the tools and metrics had a lot to do with the crowd that the session drew. But the session wouldn't have been a big draw if people didn't feel that the lack of such tools and expertise in their use was a problem. I think this bears watching--it may be that enterprises are reaching a point where measuring and managing real-time application performance is becoming a necessity, not just a nice-to-have.
