"Trust, but verify" was a signature phrase adopted and made famous in the 1980s by actor, California Governor, and 40th U.S. President, Ronald Reagan. He said it frequently (and in every meeting with Soviet Union President Mikhail Gorbachev) when discussing U.S.-Soviet Union relations.
The phrase still hangs around in Hollywood. According to Wikipedia, it is also used regularly by TV actor David Caruso's character Horatio Caine in CBS's television series CSI: Miami. Anyway, the point is don't believe it until you see it and still don't believe it until you've proved it with empirical data.
The Lure of Managed Services
After struggling through two years of recession, many IT staffs are smaller, trying to keep the lights on and deal with the growing demands of their business units for faster, cheaper, better IT services. Forget strategic planning. IT is having difficulty keeping up with the "Run" part of their job let alone their "Design and Build" stuff.
As a result, the idea of "right sourcing" with hosted and managed services for critical IT infrastructure is both valid and tempting. By putting specific service and support in the hands of others so that IT can focus on strategy and business partnerships, there is the potential to drive cost out of the business and accomplish more.
A word of caution at this point, or as ex-Intel CEO Andy Grove tried to advise with his book title--"Only the Paranoid Survive".
What Should I Care?
I bring up this whole point of "trust but verify" after meeting recently with an IT VP from a left coast company. In our meeting, she talked candidly about having a critical business application crash and what her life has been like over the past two months trying to rebuild the application database--and her credibility.
Her experience over the past many weeks has been a nightmare. A grueling emotional roller coaster ride each day, walking in to the office not knowing if she'd be fired.
What Happened?
Call the company "Mary" and the managed service provider "Steve."
Steve did good job on a complex IT project and so Mary decided to contract them to provide managed services to support their IT/Telecom systems and applications. Because Steve handled the IT project so well, Mary opted to give him the contract without having to go through a competitive bid process and nominal vetting of their capabilities.
And even though Mary had never done an IT managed services provider deal before, she didn't have anyone outside of her company review the contract to ensure it protected them and had the roles and responsibilities of both Mary and Steve clearly defined.
Long story short, one of Mary's critical application fails and she learns that the backup data isn't available to recover the system. Experts are hired at significant cost to recover the data (some but not all) and conduct a root cause analysis.
At this time, no one can definitively say why things went so terribly wrong. All she knows is that the backup files weren't available when she needed them most. The root cause analysis continues by expensive 3rd parties. She can't fire the vendor without paying hefty early termination penalties. Lawsuits are being considered and her roller coaster ride goes on and on.
Bottom Line
I know some readers will say "what happened to Mary will never happen to me" and I hope they're right. Still, Mary's story is an excellent case study that what can go wrong probably will unless you take the steps to protect yourself.
Besides having a solid contract in place, one very important step is to make sure the service provider is doing their job. Trust but verify. For anyone that has or ever had children, you know what I mean.
It may be good to remind yourself that for the vendors, the margins (profits) are in professional services, software licenses, and maintenance; not hardware. With all of the cloud talk, it seems like everybody wants in on the hosted/managed services market. Even companies with limited or no managed services experience.
In the end, it's the customer's responsibility to do their homework when selecting and managing the Hosting/Service Provider. Before, during, and after executing the contract, the customer needs to know:
* Does the service provider have sufficient experience supporting a comparable infrastructure and what do their customer references have to say? * Is the vendor assigning staff that have the right skills sets and adequate depth?
* Do the contract terms and conditions clearly define the service provider and customer responsibilities? Have you had the contract reviewed by an expert in this area?
* Are Service Level Requirements crystal clear and with appropriate penalties for non-performance?
* Do I have an escape clause if the vendor fails to perform?
* Have I performed appropriate and regularly scheduled meetings, reviews, and audits of the vendor work to confirm they are fulfilling their contract terms? Am I meeting regularly with the vendor plus getting weekly status reports to document their work performance?
We all know that systems and people fail. What matters most is the "safety net" you've built internally and with the vendors who maintain your systems.
I'll say it one more time; when looking at hosted and managed services providers--trust but verify. Don't be like Mary.
