On a previous post I discussed one of the reasons why I swapped my office Hybrid IP-PBX out for a new IP-PBX, and that was to be able to take advantage of SIP trunking. So is SIP trunking the new darling of IPT or is it just another smack upside the head to the TELCOs? Will you gain competitive advantages with SIP trunking, achieve marked cost benefits, gain strategic advantages or discover some other benefit?
In this post I’ll admit that I come away with more questions than I do answers. One thing is certain and that is the disruptive technology is disrupting market share once again. It doesn't happen overnight and this time it will be very interesting. AT&T and Verizon have both posted favorable results in their wireless divisions, and each suffer turnover in customers, which they attribute to the slowing economy.
AT&T’s hold on the iPhone crowd may soon be over as Verizon enters into talks with Apple about using an iPhone that will work with CDMA using CDMA 2000. Both companies have reported erosion in their landline business, and Verizon was hit with a drop of 34% in operating income and the WSJ also reports from an analyst, Craig Moffett, that, "Verizon's FIOS is doing a better job of replacing revenue than operating income."
Now along with erosion of landlines, other factors in the industry are stirring things up. SIP Trunks are reported to offer "feature-rich" services, but the reality is unless you buy into a hosted platform, chances are you won’t get anything near "feature-rich." Wireline carriers will have to do more than cut operating costs, and you can bet that their diminutive revenue means a reaction somewhere ahead to replace lost income. Please keep in my mind as you read on that my SIP trunking experience is limited and so are many others'.
I think the finance guys will love the initial cost benefits, but the IT/ITC or telephony folks may end up bearing the burden of making sure all’s well, even though management sees less red ink and perceives huge cost benefits.
My original mission was to find ways to exploit SIP trunks. I'll point out that calling on Verizon to inquire about SIP trunking got me nowhere fast. So in SIP trunking to begin with, there's an availability issue and even if they're available, it still depends upon your location and your available bandwidth and whether or not your gear is tested as interoperable with the provider.
Do you need an IP-PBX to get SIP trunks? No, but it sure does help. The gateways that SIP providers advertise may be purchased or leased, and this gear adds another layer of management.
A recent No Jitter post described a reader's experience using gateways and gaining significant cost benefits; I noted the enthusiasm. What would I do differently and why? For one, it's hard to say because each customer has different needs and at different times and stages in the business cycle. My needs or mostly wants were to leapfrog ahead of yanking the processor card from my hybrid IP-PBX and adding a new one to convert the hybrid to an IP-PBX. I just wanted the new IP box. My other needs are to discover first-hand so I can judge on some experience what my customers may think, say or do. Then, I wanted to find out about support. How do you support SIP trunks?
To save a couple of bucks is always good, but what would the call quality, call completion success rate and customer experience and service be like? The carrier sent me emails/letters that were more like auto responders, and when I did respond back with questions, concerns or suggestions I got no response at all. So their initial showing was good but lacking on substance, at least in the way of customer communications.
The first month I did my routine of nailing up calls between two systems for 30 minutes with music-on-hold. Next, I placed calls in and out, local and across the country to landlines, cell phones and other assorted places. After the second month I cancelled our Verizon freedom plan because I was convinced about acceptable call quality on the SIP Trunks.
What I really want from SIP trunking is the ability to have and use the same Telco features that I have and use today. The basic features I must have are 3-way calling and call transfer but was told, "Your PBX can do this." Sure, and tie up two CO lines or SIP trunks in the process, how inefficient! Don’t offer me advanced features without first meeting my basic telephony needs because my customers don’t really want to give up what already works as a sacrifice to get those "advanced features." Is SIP trunking feature rich? It depends upon the provider and your gear and the knowledge of the support on both sides, including that of the factory guys.
Initially, I've only encountered a couple of incomplete calls due to the SIP network. Audio has been very good and I’m not complaining about the cost. I gave some thought about network availability and all the stuff that folks like to talk about when addressing high availability, quality, five nines, etc. My IP-PBX has a cool feature that allows me to map a BLF (Busy lamp field) or illuminated phone button as an ALARM button. All major and minor alarms will illuminate the button and send the alarm codes too. I implemented this during the second month of service since there was no effective way to monitor whether or not the trunks were staying in service. We experienced 10 alarms from February 15 through March 24 before I called in a trouble report. ALL the alarms were showing loss of the SIP server connection. One alarm was that the Verizon FIOS and the attached Ethernet port rebooted and incurred a loss of network access for 10 minutes and 21 seconds. I traced this event from the IP-PBX back to the firewall to confirm it. The other 9 events never showed up in the firewall. Of those 9 events only one of them showed an authentication issue and immediately afterwards, the IP-PBX did authenticate and reconnected my two trunks.
I called the carrier and they didn't bother to note the times/dates and instead focused on one event being the most recent occurrence. ALL 9 events each had exactly 5 minutes of downtime. Since reporting the issue I had six consecutive days with no errors before the next Alarms (events). I called the provider and their NOC engineer offered little advice or direction. I was on a conference with ADTRAN's Chris Thompson discussing their new Alliance Program with the new partners of ADTRAN that included carriers offering SIP trunking. Chris said, he "thinks my issue is an authentication issue," and I remember seeing an interval setting for authenticating with the SIP Servers and changed that from 60 seconds (default) down to 30 and then 20 seconds. Since the last change was made, we hadn’t had any other alarms (events) for several weeks until recently. Our SIP trunks logged a series of failures on the same day, but this time they were different, as the length of outages varied. Before I could muster up courage to attack the problem again, I got a letter from the carrier's NOC stating they needed to perform emergency maintenance; and "due to the nature of the patch being applied we are advising all customers of a possible interruption in service during the maintenance window to install a patch to address recently discovered Denial of Service vulnerabilities."
We normally fax via our CustoPak (Centrex) lines connected to our IP-PBX. Our NCP-1000 does support T.38 fax over IP networks, and a year or so ago when I questioned "why aren't fax machines (standalones) dead or obsolete?" I got a response from a T.38 fax provider, and it would be nice to hear from him again. T.38 faxing may or may not be supported by SIP providers and then you may have issues of compatibility and support. We are successfully faxing (Group IV) through the SIP trunks with no issues to report.
SIP carriers that want to differentiate themselves can't just do so on cost--considering that all I really have are two cheap trunks that give me unlimited local calls and 100 minutes of long distance. Yes it's cheaper than Verizon, especially when you throw in the Caller-ID per line charge. I'd like to port my 800 numbers over but the carrier doesn’t provide two basic features--3-way calling and call transfer. The SIP service I'm using is provisioned with two trunks and Caller-ID, but I still need the basic features and I’m not willing to give up what basics do work for me just to take advantage of "advanced" features at the sacrifice of losing the basics. This is an all too familiar theme.
Now I need to add a comment about "advanced features." What are they? Strip away the significant cost benefits if you can support SIP trunking today and claim the cost savings, and then detail what these features are. Of these features--which ones don’t the Telcos offer today? I’m sure this will raise some hairs, but what I'd like to see as a buyer is some delineation of the SIP offers minus the hype, because in my mind today, the hype is the significant cost savings and if I’m experiencing what I call the "low-ball rate" which means low service, then I have to ask, are you willing to give up service to gain the great rates? I'm not saying that all providers are like this but I'm not convinced that most aren’t in the same realm.
My SIP carrier supports G.729a and G.711u. This feature is important but I found a hole. How my calls are negotiated remains unknown in our call accounting software; again, the audio is fine and no complaints about it. For the sake of accountability--how do you know which calls connect using which compression scheme? Does it matter? Maybe not for my two trunks but for a large enterprise it better matter if you want to manage your bandwidth and traffic. Then I considered changing the preference from the first, second and third choices for compression but then decided against it. G.711u means using more bandwidth but it also carries a higher MOS score. I don't want our customers' perception of our voice quality to diminish, after all if our call quality sounds off then what confidence would they have in our gear? Similar thinking leads to different paths--had I multiple offices and if I wanted to use the SIP provider as the glue for a private network, then I would encourage the use of less bandwidth for interoffice calling but I would refrain from lower bandwidth calls that touch the customers. Again, this is only my view and I know it's not aligned with the millennial generation. The important thing is to discuss and agree on a policy of what is acceptable for your company and users, and not leave it up to fate.
My take on this experience is that interoperability remains outside the grasp of any one carrier. The manufacturers of IPT gear must obtain certification with each SIP carrier and take great pains to document and profile what needs to be done to achieve consistency in the delivery of services. Otherwise, give it up--call Pizza Hut, Dominoes, Little Caesar's or McDonald's and they will gladly remind you what consistency is.
Some manufacturers errantly believe that certifying with one or two SIP providers is sufficient and I disagree. Again, the promise of IPT, including SIP and SIP services, is tied to open standards and interoperability. The lack of interoperability is significant reason for the existing disparity of services available to customers. Then, the dealers/VARs must have that same consistency driven as standard practices to be able to install and deliver SIP trunks. This really means a better track for the factory guys to chart and it must start with interoperability and not end with certification of one provider. Then, training and educational resources are in order because until the dealers/VARs and enterprise users can carry forth the sword, SIP could end up being a call to the wild.
Repeating what I've stated before, telecom needs to evolve to Telecom as A Service, meaning best of class in delivery to the end users. I'll also repeat what I've heard from carriers about not competing with Managed Service Providers--they don't want to offer features such as what I want, because it would be competitive against the MSPs who are also customers of SIP providers. Some of the SIP providers are in fact MSPs themselves and offer the features on the condition that I buy into their hosted IP-PBX. Then, whether or not you have a gateway with a TDM box or an IP-PBX with just licensing and an Ethernet connection, you must realize that there is no demarcation. How will you isolate and identify the root cause of problems and support SIP trunking going forward?
SIP trunking is another alternative and it's customary for Interconnects to spend money on finding out what works and doesn’t before suggesting it to customers. There's a local car dealership in my area, and the owner says she wants to WOW customers. SIP providers listen up--if you want to compete on cost alone you will die a painful death. For those of you that want to create competitive and strategic advantages, cost benefits and differentiated services then you are delivering what "WOWs" customers, because price alone doesn’t tend to last.
Depending upon the degree of how critical your communications service is, this is how I would view SIP trunking today. I'm not yet willing to put all my eggs on SIP trunks, and yet I want to. Instead, I'm using SIP trunks as a realistic alternative to save money on our outbound long distance and local dialing.
It's very unlikely that I'll be moving my own office again before retirement. For those that do move offices such as those in the SMB market for reasons of contraction, expansion or better leases; SIP trunking isn't as portable as it should be. Will you get to keep what you have today at your current location if you move across town? Then, if the Telcos don't see this as a threat, what becomes of the PSTN as companies displace services?
For many, SIP trunking makes sense because you can significantly reduce your costs but you must remember, your SIP trunks are as only good as your WAN connection. Again, with the carriers' loss of wireline revenue, what will they do in reaction to replace those lost earnings? The last mile for most means the transport, and that’s what could dilute savings enjoyed today by exchanging bandwidth for dial tone in the future. In an attempt to overcome the fear of going ALL SIP, I've asked my buddy Chris Thompson at ADTRAN to give me some advice on alternative routing schemes. I'll be pursuing it later and reporting back.
I've already ordered and just completed installation of a new IAD (Integrated Access Device) from ADTRAN. I'll be retiring my trusty 1224 STR PoE box to our lab and utilizing the 1335 with WiFi, VPN and VQM (Voice Quality Monitoring) for the office. VQM is an ADTRAN strength that I think will enable folks to readily adopt SIP. For now, packet traces are what most vendors will steer you towards when troubleshooting, unless you have something better onboard.
The monthly savings we enjoy got me thinking, so I called Verizon back. How much does it cost to up my FIOS service? First they gave me all the good stuff and I balked. Then the rep said, "We do have the advanced BASIC package that will give you 10 MEG down and 2 MEG up for $9 a month more." SOLD! I could hear the Verizon agent typing away and he told me "this will be done by tomorrow." I heard no more typing so I went to a website to clock my FIOS speed--it concurred, nearly 10 MBPS down and 2 MBPS up. This rep went on to tell me that my company was eligible for a new discount on CustoPak (Centrex) lines with a 3 year commitment, further reducing our line cost by $6 per month/per line. I hoped his typing was as effective to turn on the discount as it was to throttle up the bandwidth. Later, I did confirm getting my discount.
So another observation: Are the Telcos dropping prices of POTS and Centrex? I think they are. Does it mean that I won’t port my numbers over to SIP? No, my Centrex lines run over FIOS already so if my FIOS is down, our voice lines are virtually never down, since we have embedded call forwarding to overflow calls to our cell phones.
I faced a long journey in just getting my new router to connect with Verizon's FIOS, and the experience brought back a memory from the last time I went through this process. We shut down the router and removed the ground plug from the battery connected to the FIOS ONT (Optical Network Terminal). Then, we removed power to the power supply and kept the fiber down for several hours, and only then would Verizon recognize our router. So in part, this is just one reason why I need to have Call Forwarding on ALL our lines--at least we can forward to cell phones.
The other kick is changing routers. I got what I wanted but at a price of disruption. Getting everything back to the way it was isn’t just shoulder-surfing the old router configuration and typing in the new. I am appreciating more than ever why folks resist patch and change over what’s in and working and keeping everyone quiet and happy. So in contemplating SIP trunking or any hosted service, be keenly aware of future changes and plan for the time to do the work and how to resolve any issues. More services and applications on one pipe means more consideration, planning and time to do and verify that the work is done, to ensure that everything is working as it should be. I took three days to get everything back but I will say the learning curve for the new features in the router was worth it. I have to hand it to ADTRAN--they were right, even I could setup the router and I didn't call ADTRAN for help with that (this time)--although I did make a simple mistake in configuring the WiFi, and even I know better.
While SIP is tempting, don't try and dive into it without first having in place alternative routing and something to support SIP other than a packet trace and an 800 number to call for help.
Security is even more critical and with SIP trunking you must have security, security, security along with location, location, location--otherwise you may not want SIP. As my buddy Eric queried me about whether or not my SIP trunk was attacked via the transport or whether the provider suffered at the source, it's pretty clear that it doesn't matter because what matters most is that either attack must be endured. VLANs I think still play a key role in helping to keep voice safe from the data and data safe from voice. As SIP trunking gains ground I expect we will hear about more security issues and vulnerabilities. I do have planned yet another run and that is to implement voice encryption. Security is often focused on what is going into a company instead of what is going out of it, and that is where the intelligence thrives.
I added routes in our IP-PBX for overflow of outbound calls in case the SIP trunks do fail, and our box automatically puts the SIP trunks out of service when they do fail and will automatically overflow to other routes (trunk groups).
I can cancel our SIP trunks after 90 days since we are using a dealer demo package. For users, consider a "trial" or "evaluation" period to ensure you are getting what you want and asked for. Then, get the provider to extend the period so you can judge their responsiveness to service. We've been using SIP trunks since November 2008, and are still not resolved to keep the service but instead to seek out other providers in hopes of finding that more perfect fit. I mentioned the 90 days because for those SIP providers offering "feature-rich," consider this an invitation to offer me what you have. I’ve already called upon several SIP providers and the answers were generally the same--"We don’t provide features with SIP trunks."
In the SMB world there are scores of hundred-dollar firewalls and switches, and most of this gear will not sustain a viable network running SIP trunks let alone VoIP/IP-PBXs and hosted solutions. Many firewalls still in service lack SIP traversal capabilities and then are they even stateful inspecting? Other concerns include the fact that some SIP providers now call for Deep Packet Inspection (DPI) firewall patrolling. There are seemingly more barriers to SIP trunk adoption, but again the allure is the cost benefit. Whether or not the benefits will outweigh the risk or the costs to upgrade I think is premature. For now, we’re proceeding slowly and another reality has sunk in--I am now dependent upon my router, very dependent.
Even if or when my router fails, it doesn't mean I'm derailed unless I put all my services on a single provider. The ONT separates my Verizon Centrex service at the interface. I mentioned ADTRAN's VQM and this was the primary reason I bought the new router. I'll be reporting back on VQM and I can say this early on, WOW! ADTRAN has wowed one customer and the value of VQM makes the new router worth the purchase. Up to this point, my perception has been that there have been no audio (quality) problems with our SIP trunks. Only now I can prove it or fight it.
RECAP: Potential Barriers to SIP Trunking
* What are the competitive advantages of using SIP trunks?
* Clearly identify the cost benefits against the true TCO
* What are the strategic advantages of using SIP trunks?
* Do you have an SLA or just a basic contract for service?
* How will you support your SIP trunks? What kind of tools do you have?
* Are you sacrificing basic features to obtain advanced features?
* Are your SIP calls accountable, showing how much bandwidth they used?
* Is your network SIP ready and compatible?
* Do you have adequate security in place?
* Will you maintain portability with your SIP trunking and services if you move your business location(s)?
* What is the interoperability of your gear with the provider?
* Are your choices limited by manufacturer certification?
* How will future network changes impact your SIP trunks and services?
* How dependent are you willing to be upon SIP trunking?
* Do you have alternative and diverse routing of services?
* What is your exit strategy?
Deploying SIP Trunks: What Went Wrong & Other Snags
* Communications--I had to hassle the provider’s VP of Marketing to get a response for the costs, offerings, procedures, etc to sign up for SIP trunks since sales response was zero.
* Communications--the provider’s turn-up engineer didn’t know what button or line appearances mean and I didn’t know if I was getting a group button--you can do either or both but you better do single-line appearances first so you can do appropriate testing in and out of the IP-PBX or other platform.
* Communications--between the manufacturer of the IP-PBX and the SIP trunk provider, things weren’t exactly glued down. One said it won’t work or it can’t be done while another said something else. It can be done if you have enough willingness to see it through.
* I dumped our Sonicwall firewall previously and forgot why. Had I left it in place we would still be trying to figure out why the SIP trunks aren’t working.
* H.323--don’t forget to enable it in the router.
* ALG (SIP) turned on may not be a good idea so turn it OFF first in the router, unless you are using a managed service provider (MSP) or hosted solution.
* Undocumented settings--if it’s there, I will play with it. Ooops!
* How do you know your SIP trunks are always available?