When Cutovers Cause ChaosWhen Cutovers Cause Chaos

With over 25 years' experience in implementing and managing carrier and client telecom projects at The BAZ Group, we have certainly seen it all. And while each project carries its own unique challenges, if there is one thing we have learned, when it comes to carrier cutovers, if anything can go wrong, it most definitely will. They say that experience is the best teacher, and nowhere is that more true than in Wi-Fi installs and upgrades.

Prepare and Plan
Wi-Fi upgrades present unique challenges in implementations. The need for the proper installation of access points (AP) requires that implementation documents include a detailed, complete, and current floor-plan, properly marked as to AP location placement. Simplified floor plans or even "eyeballing it" is the quickest way to invite Murphy's Law into a project. Ceiling heights, shielding, and even unknown structural metal can cause blockages in signal, leading to "scope creep" for additional AP requirements.

Lesson learned? A fully vetted, detailed site plan can save significant dollars and frustration.

Coordination is Key
With large enterprises, it's common for several different departments to be working on separate upgrades at the same time. It's also common that these departments will fail to notify each other of changes being made that could affect the other.

This happened on a Wi-Fi upgrade implementation we were executing for a client that was deploying an enhanced mode conversion upgrade of software on cash registers. Unfortunately, unbeknownst to the client's project team, this conversion upgrade was scheduled at the same time and on the same night as a Wi-Fi network cutover was scheduled. Thankfully, this conflict was caught early, and the Wi-Fi upgrade was able to be rescheduled. Had they both occurred concurrently, the outcome and time spent could have been devastating to not only the project schedule, but the physical operations of the retail outlet as well.

Additionally, it goes without saying that departmental coordination and communication must be free-flowing when it comes to establishing cutover dates with carriers. Often, IT departments will schedule an implementation with the carrier, but fail to work with the hardware vendors or equipment teams to ensure that the routers, controllers, etc., are properly configured and delivered well in advance of the turn-up. This is truly an expensive mistake, as many enterprises not only have outside, on-site installers coordinating the cutover ($$), but carriers will begin billing from the point the cutover is considered "complete" on their end -- not when the client is ready to deploy.

The lesson here? Failure to coordinate and communicate large changes to the telecom environment can wreak havoc. To cut the chaos, coordinate and communicate.

It Depends...on Dependencies
Even with updated routers, switches, controllers, and APs in place, many organizations are stymied by wireless devices that simply will not connect or work properly. The root cause? Incompatibility of dependent devices. Inventory scanners, check readers, printers, RFID equipment, and even door alarms that worked yesterday no longer function. Such was the case with a recent cutover at a manufacturing site. Called in to troubleshoot these devices, we found that their inventory scanners were running a legacy, non-supported software version that was incompatible with the speeds and architecture of the new Wi-Fi network. The solution? New configurations for each of the non-connecting devices, resulting in additional time and expense.

What is the key lesson here? Understand exactly what is being placed on the network from a legacy hardware and device perspective, and determine the compatibility of these dependent devices.

Location, Location, Location
As they say in real estate, it's all about location, location, location. In this instance though, it isn't the location of APs or other hardware; issues can arise with the physical site itself. Obvious issues include delayed carrier build-outs to deliver services that push timelines. The not so obvious? Asbestos concerns, excessively long cable runs that can degrade performance as a result of remote demarc locations, and let's not forget those previously mentioned structural steel issues and concerns.

The lesson? Physical location matters. Performing a thorough site evaluation with a qualified consultant, enterprise real estate teams, and carrier representatives can uncover many of these concerns before they become catastrophic and costly cutover events. Asbestos abatement contractors can be engaged to mitigate cable runs, APs can be moved to less shielded areas, and budgets can be built around carrier build outs and appropriate cabling.

After the Transition
Finally, a key way to learn from the cutover process and ensure additional project success involves surveying the end-user locations immediately after transition. In this survey, the user is asked specific questions to uncover immediate concerns, and the enterprise learns what additional issues and pitfalls to watch out for in the future. Often, even the simplest questions can yield very telling answers. Here are some sample questions that are often used by organizations:

It is pretty clear that experience is the best lesson. Unfortunately, many enterprises learn too late that even the simplest of projects can be derailed by the unknown and unexpected. Yet, with proper preparation, communication, collaboration and a fully vetted process, cutovers can go much more smoothly. And in the end, that is the best lesson to be learned.

"SCTC Perspectives" is written by members of the Society of Communications Technology Consultants, an international organization of independent information and communications technology professionals serving clients in all business sectors and government worldwide.