SHARE



ABOUT THE AUTHOR


Matt Brunk
Matt Brunk has worked in past roles as director of IT for a multisite health care firm; president of Telecomworx,...
Read Full Bio >>
SHARE



Matt Brunk | July 17, 2015 |

 
   

Hat Off to AT&T, Unexpected Residential Partner

Hat Off to AT&T, Unexpected Residential Partner Here's a case of support techs going above and beyond in getting to the root of and fixing a connectivity problem, and making sure the customer feels valued.

Here's a case of support techs going above and beyond in getting to the root of and fixing a connectivity problem, and making sure the customer feels valued.

A recent move, even though just a short distance, meant I had to disconnect my AT&T U-verse modem and then reconnect it. Wouldn't you know, a few days later the Internet connection started failing, with three- to four-minute spans of complete service loss. Figures, right?

One important thing to know about AT&T U-verse service is that it is a fiber-to-the-home (FTTH) offering based on a Gigabit Passive Optical Network, or GPON, architecture. OK, not that I have that out of the way, let me describe the troubleshooting process.

Troubleshooting began when I called the AT&T support center and talked to an agent, who eventually decided to dispatch an inside technician to my house. The technician's first move was to replace the Cisco modem I was using. When I told him about the recent blackouts, he then added an external Belkin uninterruptible power supply (UPS), designed specifically for the new modem. All was well... for a short while.

Within a few hours of the tech's departure, I again lost my Internet connection. This happened as the modem would either reboot or simply reconnect and then sync back up to the carrier.

AT&T like Verizon, uses a UPS with each optical network terminal (ONT) it deploys to a residence in order to "keep the light on" and, of course, the span up. In my situation, the ONT was new, just like the fiber, and a second modem still didn't garner success. What was causing the fiber to drop?

The logs simply showed the modem timing out and then reinitializing. The continuous ping from my MacBook Air revealed packets dropping intermittently:

64 bytes from 4.2.2.2: icmp_seq=6846 ttl=50 time=50.027 ms
64 bytes from 4.2.2.2: icmp_seq=6847 ttl=50 time=58.469 ms
64 bytes from 4.2.2.2: icmp_seq=6848 ttl=50 time=626.177 ms
64 bytes from 4.2.2.2: icmp_seq=6849 ttl=50 time=46.380 ms
64 bytes from 4.2.2.2: icmp_seq=6850 ttl=50 time=39.296 ms
Request timeout for icmp_seq 6851
Request timeout for icmp_seq 6852
Request timeout for icmp_seq 6853
Request timeout for icmp_seq 6854
Request timeout for icmp_seq 6855
Request timeout for icmp_seq 6856
Request timeout for icmp_seq 6857

Eventually the pings would resume, and the erratic latency would continue:

64 bytes from 4.2.2.2: icmp_seq=7038 ttl=50 time=35.405 ms
64 bytes from 4.2.2.2: icmp_seq=7039 ttl=50 time=35.624 ms
64 bytes from 4.2.2.2: icmp_seq=7040 ttl=50 time=35.605 ms
64 bytes from 4.2.2.2: icmp_seq=7041 ttl=50 time=36.135 ms
64 bytes from 4.2.2.2: icmp_seq=7042 ttl=50 time=35.749 ms
64 bytes from 4.2.2.2: icmp_seq=7043 ttl=50 time=423.246 ms
64 bytes from 4.2.2.2: icmp_seq=7044 ttl=50 time=36.107 ms
64 bytes from 4.2.2.2: icmp_seq=7045 ttl=50 time=36.318 ms
64 bytes from 4.2.2.2: icmp_seq=7046 ttl=50 time=115.176 ms
64 bytes from 4.2.2.2: icmp_seq=7047 ttl=50 time=36.825 ms
64 bytes from 4.2.2.2: icmp_seq=7048 ttl=50 time=36.770 ms
64 bytes from 4.2.2.2: icmp_seq=7049 ttl=50 time=36.395 ms
64 bytes from 4.2.2.2: icmp_seq=7050 ttl=50 time=36.440 ms
64 bytes from 4.2.2.2: icmp_seq=7051 ttl=50 time=36.226 ms
64 bytes from 4.2.2.2: icmp_seq=7052 ttl=50 time=35.571 ms
64 bytes from 4.2.2.2: icmp_seq=7053 ttl=50 time=39.528 ms
64 bytes from 4.2.2.2: icmp_seq=7054 ttl=50 time=36.201 ms
64 bytes from 4.2.2.2: icmp_seq=7055 ttl=50 time=42.419 ms
64 bytes from 4.2.2.2: icmp_seq=7056 ttl=50 time=36.756 ms
64 bytes from 4.2.2.2: icmp_seq=7057 ttl=50 time=568.390 ms
64 bytes from 4.2.2.2: icmp_seq=7058 ttl=50 time=36.421 ms
64 bytes from 4.2.2.2: icmp_seq=7059 ttl=50 time=36.365 ms
64 bytes from 4.2.2.2: icmp_seq=7060 ttl=50 time=86.979 ms
64 bytes from 4.2.2.2: icmp_seq=7061 ttl=50 time=35.505 ms
64 bytes from 4.2.2.2: icmp_seq=7062 ttl=50 time=599.214 ms
64 bytes from 4.2.2.2: icmp_seq=7063 ttl=50 time=35.592 ms
64 bytes from 4.2.2.2: icmp_seq=7064 ttl=50 time=36.211 ms
64 bytes from 4.2.2.2: icmp_seq=7065 ttl=50 time=35.823 ms
64 bytes from 4.2.2.2: icmp_seq=7066 ttl=50 time=436.547 ms
64 bytes from 4.2.2.2: icmp_seq=7067 ttl=50 time=36.287 ms
64 bytes from 4.2.2.2: icmp_seq=7068 ttl=50 time=36.313 ms

Walking through the neighborhood, I spotted an AT&T truck and struck up a conversation with the technician. He happened to be an outside fiber tech, whereas the guy who did my service call was an indoor tech. After explaining my problem, he said, "Let's go look at your ONT and test the fiber."

Using an optical time-domain reflectometer, he tested the optical fiber and didn't find anything wrong with it. He then called in, and spoke with his boss. They asked if I could help isolate the issue by disconnecting the Ethernet connections, which I did. Everything else was served by the onboard Wi-Fi. After the Internet continued to fail, I noted the modem state and saw that only the power light stayed on.

The AT&T outside tech called to inquire about the status. Hearing I was still having problems, he came back to my residence -- and brought his boss with him. We talked about the possibility of a defective ONT, but because the retail appliance doesn't have the same statistics as a managed switch or appliance, we had no way of knowing for sure if that was the case. Even so, AT&T replaced the ONT and reworked and retested the fiber.

Within a few hours the Internet connection once again failed. When I called the AT&T fiber tech to inform him, he decided to have a central office tech check the other end. But, again, the Internet connection continued to fail after a short period of time, with the modem rebooting or just re-synching up in a few minutes.

Next the fiber tech dropped off a surge protector, which I installed in front of the UPS. Again, the Internet failed several times and never with any pattern.

portable

This time, the AT&T fiber tech unlatched the ONT so I could open the cover and view the status lights whenever the Internet failed. In doing so, I discovered that the "DATA" light was off -- meaning, the link failure wasn't the fiber but the Ethernet connection to the modem (as shown at right).

The next AT&T inside technician dispatched was assigned the task of replacing the modem again, this time with a new Motorola model. This unit contains a small lithium battery that snaps into the bottom of the unit and eliminates the need for the UPS brick I previously needed.

The Internet connection has been working since then, with no erratic ping times, either.

The AT&T techs commented that they really appreciated me working with them on this, and that they really wanted to resolve the issue. Not once did any of these techs tell me that if he found an inside customer cause to the problem that AT&T would bill me. And, in fact, each tech went out of his way to help me, even spending time to examine the logs and ping results I gathered.

I believe the original modem was disrupted by the blackout since the power went down a couple of times in succession before being restored. The techs told me the second modem was inferior. The third modem is the replacement AT&T is using for new installations going forward.

The service and level of customer service that all the AT&T members exhibited were exemplary. The fact that they brought out a new surge protector and a new and improved modem with an onboard lithium battery reveals that they understand potential issues caused by bad power and, more importantly, that they value customers. FTTH or fiber to the curb is not a cure-all solution, but if AT&T advances its footprint and provides the level of service I experienced, I feel confident that the new era of telecommunications will be better, faster and, with improvements and patience, one day maybe even easier to troubleshoot.

Follow Matt Brunk on Twitter and Google+!
@telecomworx
Matt Brunk on Google+





COMMENTS



Enterprise Connect Orlando 2018
March 12-15 | Orlando, FL

Connect with the Entire Enterprise Communications & Collaboration Ecosystem


Stay Up-to-Date: Hear industry visionaries in Keynotes and General Sessions delivering the latest insight on UC, mobility, collaboration and cloud

Grow Your Network: Connect with the largest gathering of enterprise IT and business leaders and influencers

Learn From Industry Leaders: Attend a full range of Conference Sessions, Free Programs and Special Events

Evaluate All Your Options: Engage with 190+ of the leading equipment, software and service providers

Have Fun! Mingle with sponsors, exhibitors, attendees, guest speakers and industry players during evening receptions

Register now with code NOJITTEREB to save $200 Off Advance Rates or get a FREE Expo Pass!

November 29, 2017

As video conferencing use rises in the enterprise, businesses are looking for ways to bring this technology out of traditional conference room and make it more broadly accessible. That's made the h

November 1, 2017

Your customers (internal and external) demand that you offer them the ability to connect by any means. With the adoption of cloud communications tools you now have access to an expanded portfolio o

October 18, 2017

Microsofts recent Ignite event had some critically important announcements for enterprise communications. Namely, Microsofts new Team Collaboration offering, Teams, will be its primary communicatio

October 23, 2017
Wondering which Office 365 collaboration tool to use when? Get quick pointers from CBT Nuggets instructor Simona Millham.
September 22, 2017
In this podcast, we explore the future of work with Robert Brown, AVP of the Cognizant Center for the Future of Work, who helps us answer the question, "What do we do when machines do everything?"
September 8, 2017
Greg Collins, a technology analyst and strategist with Exact Ventures, delivers a status report on 5G implementation plans and tells enterprises why they shouldn't wait to move ahead on potential use ....
August 25, 2017
Find out what business considerations are driving the SIP trunking market today, and learn a bit about how satisfied enterprises are with their providers. We talk with John Malone, president of The Ea....
August 16, 2017
World Vision U.S. is finding lots of goodness in RingCentral's cloud communications service, but as Randy Boyd, infrastructure architect at the global humanitarian nonprofit, tells us, he and his team....
August 11, 2017
Alicia Gee, director of unified communications at Sutter Physician Services, oversees the technical team supporting a 1,000-agent contact center running on Genesys PureConnect. She catches us up on th....
August 4, 2017
Andrew Prokop, communications evangelist with Arrow Systems Integration, has lately been working on integrating enterprise communications into Internet of Things ecosystems. He shares examples and off....
July 27, 2017
Industry watcher Elka Popova, a Frost & Sullivan program director, shares her perspective on this acquisition, discussing Mitel's market positioning, why the move makes sense, and more.
July 14, 2017
Lantre Barr, founder and CEO of Blacc Spot Media, urges any enterprise that's been on the fence about integrating real-time communications into business workflows to jump off and get started. Tune and....
June 28, 2017
Communications expert Tsahi Levent-Levi, author of the popular BlogGeek.me blog, keeps a running tally and comprehensive overview of communications platform-as-a-service offerings in his "Choosing a W....
June 9, 2017
If you think telecom expense management applies to nothing more than business phone lines, think again. Hyoun Park, founder and principal investigator with technology advisory Amalgam Insights, tells ....
June 2, 2017
Enterprises strategizing on mobility today, including for internal collaboration, don't have the luxury of learning as they go. Tony Rizzo, enterprise mobility specialist with Blue Hill Research, expl....
May 24, 2017
Mark Winther, head of IDC's global telecom consulting practice, gives us his take on how CPaaS providers evolve beyond the basic building blocks and address maturing enterprise needs.
May 18, 2017
Diane Myers, senior research director at IHS Markit, walks us through her 2017 UC-as-a-service report... and shares what might be to come in 2018.
April 28, 2017
Change isn't easy, but it is necessary. Tune in for advice and perspective from Zeus Kerravala, co-author of a "Digital Transformation for Dummies" special edition.
April 20, 2017
Robin Gareiss, president of Nemertes Research, shares insight gleaned from the firm's 12th annual UCC Total Cost of Operations study.
March 23, 2017
Tim Banting, of Current Analysis, gives us a peek into what the next three years will bring in advance of his Enterprise Connect session exploring the question: Will there be a new model for enterpris....
March 15, 2017
Andrew Prokop, communications evangelist with Arrow Systems Integration, discusses the evolving role of the all-important session border controller.
March 9, 2017
Organizer Alan Quayle gives us the lowdown on programmable communications and all you need to know about participating in this pre-Enterprise Connect hackathon.
March 3, 2017
From protecting against new vulnerabilities to keeping security assessments up to date, security consultant Mark Collier shares tips on how best to protect your UC systems.
February 24, 2017
UC analyst Blair Pleasant sorts through the myriad cloud architectural models underlying UCaaS and CCaaS offerings, and explains why knowing the differences matter.
February 17, 2017
From the most basics of basics to the hidden gotchas, UC consultant Melissa Swartz helps demystify the complex world of SIP trunking.
February 7, 2017
UC&C consultant Kevin Kieller, a partner at enableUC, shares pointers for making the right architectural choices for your Skype for Business deployment.
February 1, 2017
Elka Popova, a Frost & Sullivan program director, shares a status report on the UCaaS market today and offers her perspective on what large enterprises need before committing to UC in the cloud.
January 26, 2017
Andrew Davis, co-founder of Wainhouse Research and chair of the Video track at Enterprise Connect 2017, sorts through the myriad cloud video service options and shares how to tell if your choice is en....
January 23, 2017
Sheila McGee-Smith, Contact Center/Customer Experience track chair for Enterprise Connect 2017, tells us what we need to know about the role cloud software is playing in contact centers today.