This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.
How to Manage A Cloud Collaboration Solution Effectively
Managing a cloud-based collaboration solution is a different proposition than managing an in-house infrastructure. It’s more like managing a managed service with the service provider having a very different view of how to manage you (e.g., more automation, less interaction), so some adjustments are in order.
The ultimate goal of IT is making sure your enterprise is getting the service they need, which means ensuring that the service isn't only available but providing a quality experience. By analyzing when a service is on/off-line, measuring the quality of experience, and working with providers customer success teams, you can effectively manage your vendor and make sure your enterprise is getting the services that they need.
Measuring how often a solution isn’t available can easily be done through the existing enterprise ticketing system. Users call your help desk when the solution isn’t available, a ticket is opened, and then the ticket is closed when the system comes back on-line. Measuring the duration of time between when a ticket was opened and closed gives you the rough availability of the solution. Tickets can also be used to determine if availability is site- or geography-related, as the tickets will specify what is and isn’t working and will be raised in the geography where users complain.
The vendor will also supply information about their uptime on a customer-facing portal or blog site. This can be useful in reviewing how often and what types of outages occur and will also show outages that weren’t detected by users (e.g., a failure at midnight isn’t detected until users start work at 7:00 AM).
I recommend tracking this information monthly, and reviewing it monthly or quarterly with the vendor and with the network team as appropriate. Just letting the vendor know you are tracking this metric will help push them in the right direction.
Quality of Experience
Having a solution available and having a solution that users want to use are two different things. The quality of the user experience is critical to the uptake of the technology, and thus to the promised increase in productivity and resulting ROI. Some of the issues of quality of experience (QoE) are under the service provider’s control, but some of them belong to the enterprise or the network providers.
The four phases of a collaboration call are:
- Call Initiation (joining)
- In-call experience
Tracking the experience of users in each phase provides insight into how hard or easy it is to use the solution, and thus the likelihood that they will continue to do so and will recommend the service to colleagues.
Scheduling done right is as easy as using a plug-in within your standard calendaring tool (e.g., Outlook) to add collaboration information into a meeting. Making scheduling of a collaboration call as easy as scheduling a one-on-one or a meeting in a local conference room is important to ensure rapid technology uptake. Some enterprises are using higher-level scheduling tools for building facilities that include guest welcoming services, catering, room scheduling, and management of information panels outside of the conference room. It’s critical at the design stage to make these systems as transparent or as integrated as possible. If users have to bounce back and forth between two different systems to schedule a collaboration session (meeting rooms on one system and personnel on another), they often give up and just go to a phone bridge or something else.
Analytics don’t help very much here. Most managers are using the solution and thus have direct experience with how hard or easy it is to use. For the most direct feedback, ask your executive administrators as they are likely doing the most complex scheduling and will have pointed feedback on what does and doesn’t work well.
Call initiation should be as simple as clicking on a link (desktop or mobile) or one touch on a touch screen (for room systems). If this part of the solution is not working well, feedback will come from direct experience or user feedback.
In-call experience is a component that can be directly monitored through tools, and I recommend some investment here to make sure you have ongoing insight into this aspect. Vendors will often provide an analytics portal that gives information about the use of their solution. The easy metrics are numbers of meetings and minutes in meetings, which is helpful to determine adoption rates but otherwise not very helpful.
A call history report with information on time-of-day, the location of the users, type of call, and call modalities used (audio, video, content sharing) can provide insightful metrics into the quality of calls. Additionally, as much information as possible about the network connectivity of those users, including how they are connected (direct, WiFi, or cellular) and their location, is useful. The vendor may also be able to provide network analytics for the call including packet loss, jitter, and latency. Knowing to which service provider data center or PoP they connect is also critical to good analysis. This information gives you the ability to discover which components of the solution (endpoints, network, vendor) and which components of the network (local connect, enterprise LAN, enterprise WAN, Internet) are causing quality issues.
It’s also very useful to be able to build reports showing the instantaneous demand on various portions of your network from the collaboration solution. This means knowing where the user is located (e.g., local IP address), when they joined and left each call, the call bandwidth, and what modality they were using (audio, video, content). While the vendor can provide this raw data, it’s unlikely that the vendor will compile the needed reports on instantaneous demand because this requires knowledge of your enterprise network structure. Spending time to create these reports is very helpful in sizing network links or in determining where Internet access points should be deployed across the enterprise.
Getting direct user feedback is invaluable in managing your user’s quality of experience. Tickets are one measure of user feedback. The vendor may provide an integrated user feedback mechanism, like a simple five-star rating input at the end of each call. Or perhaps they provide some simple problem-reporting templates that are automatically presented to users (e.g., audio was poor, could not see the other side, content sharing was slow to present, etc.).
Users have a much higher probability of providing provide feedback at the end of a call rather than a provider asking about their experience at a later time. If this information is tied to details of the call, then doing correlations on user complaints with location or network type or time-of-day can be very helpful in isolating and solving quality issues. This feedback can also let you know where more user training or support is needed.
Vendor Customer Success Support
Most cloud-based solution providers either have or are planning to have an internal customer success team, which ensures a contracted enterprise’s goals are being met and that they renew at the end of the service's current term. These teams can solve problems quickly and act as a conduit between the provider’s development team and the contracted enterprise to relay issues and feature requests. Large customers typically have a dedicated customer success team member assigned, and so a good relationship can be built at the working level between the enterprise and the vendor to ensure the solution is providing the needed services. I recommend an explicit assignment or process in your organization to leverage this resource and relationship.
Assigning someone as an executive sponsor to focus on the progress of a transition plan and to promote new technology adoption might help transition users to a new service. The executive sponsor models the new approach an enterprise wants its users to adopt and pushes other parts of the organization to follow their example. Your sponsor plays a key role (along with the CIO) in engaging with the service provider management to encourage them to up their game, whether it be in terms of feature set, reliability, or analytics. The best sponsor is a business manager within the organization where the collaboration solution is integral to their business delivery, not just a part of the expected infrastructure of the organization.
Include your executive sponsor in periodic update meetings during the transition to ensure their buy-in on the solution and the process.
Also, connect your executive sponsor and your CIO with the appropriate levels of management within your chosen service provider. This executive connection is valuable for supporting problem escalations and for encouraging your provider to add capabilities that will benefit your organization. This feedback path is especially powerful if your executive sponsor is a business manager. Showing your provider a direct correlation between a requested feature and a business result will give them an incentive to include that feature as it helps the vendor increase their market opportunity.
Office-based collaboration rooms, whether they are audio and content only or video-enabled, still need to be managed. Using your current approach is an option, but consider using a managed service provider (MSP) if you aren’t already doing so. An MSP should have more in-depth knowledge of the room equipment and be better able to actively monitor those systems, stay ahead of software upgrades, understand security vulnerabilities, and know-how to help users through on-the-spot issues better than an in-house team, just because that is their full-time job. If they are a large MSP, they will have automation solutions in place that can be much more cost-effective than an in-house manual approach.
Managing a cloud-based collaboration solution is about listening to your users, understanding their experience, diagnosing the technical challenges occurring either in-house, with the network service providers or with the cloud service, and then driving solutions to solve those issues. Use all your standard techniques of managing KPIs to know if you are on track, and leverage the analytics to see if long-term improvement is occurring.