Where Data Processing Lives

Information processing can reside anywhere in a network. Traditionally, information has been processed centrally in the data center; but more recently, trends like the Internet of Things have forced a reconsideration of centralized processing, as information is increasingly processed in the cloud.

Get ready for a new class of micro data centers that are operated with little or no human involvement. When processing is performed at the network edge near the location of information production, it is called fog computing (see related post, "IoT at the Network Edge").

Is Fog Computing Right for Your IoT?

The Internet of Things (IoT) will involve millions to billions of endpoints in fixed and mobile conditions, and as I mentioned previously, the data output from those endpoints can be processed in either the cloud or a data center.

The response time (latency) of data centers and clouds will not be fast enough for some of the applications, for example, those in vehicles. For these situations, information processing will likely be performed locally, and the cloud will come into play to deliver information to the in-vehicle app such as maps, traffic conditions, and even advertising.

There will be situations where there are hundreds of sensors that need to connect to the cloud. In such a scenario, a controller will be implemented to connect the endpoints to the cloud. It is more cost effective to have the controller communicate to the cloud rather than many endpoints. Since there has to be a controller and processing is so cheap, it is likely that the controller will act as an analytic processor for the IoT endpoints. Once the analytic work is done, it can be communicated to the cloud for further processing and actions.

How Close Is the Data?

Edge processing, also called fog computing and sometimes referred to as micro data centers, is the candidate for many IoT installations. The edge processor collects the raw data, analyzes it, and provides the necessary information to the cloud. Transmitting all the raw data to the cloud is entirely unnecessary.

Determine whether any measurements require real-time decision making and response and configure the edge processor accordingly. While the edge processor is performing rapid analysis, the cloud will be analyzing the edge processor, providing historical information to complement any real-time analysis being performed.

Real-Time, or Near Real-Time

IoT endpoints communicating to the cloud will produce a lot of data. Some of the data will require actions faster than the cloud can provide. Some of the IoT endpoints will be working in very rapid real-time, milliseconds. When the response time is in seconds, the cloud can be the source of actions and responses. You need to classify the responses into real-time (milliseconds), then it has to be done at the edge. If seconds of response times are acceptable, it can to be done by the cloud in near real time.

Processing in the Data Center or Cloud

IoT endpoints will depend on network access and continuity. It can be dangerous and inefficient, or even unacceptable, for IoT endpoints to fail because of lost access to the cloud. IoT endpoints will be dependent upon the edge processor to provide business continuity. I would not want to be driving a car that requires cloud access to deal with my safety. What if the wireless network is just not accessible? Then my safety is in jeopardy.

Edge processors will be very important in providing business continuity when access to the cloud fails. This means a number of decisions have to be made locally for the endpoints. The data may also have to be collected and sent when the cloud access is restored.

Processing at the Edge

The cost of processing has decreased considerably. A laptop at a remote location could act as the edge processor. The real difference will be in the software. There's already software for specific industries and endpoints. Not every business organization is the same. There are APIs available so that customers who own edge processors can develop and install software specific to their business needs. This means that the cloud will also become the central depository for all software and updates.

Security is a concern for IoT edge processors. They will need to be highly secure to ensure that the IoT data is accurate and timely, and that they operate correctly. Security of IoT endpoints may not be good, but security of the edge processor can make up for security limitations.

Micro Data Center Characteristics

The edge processor (micro data center) will probably operate unattended, with little or no human interaction. It will:

  • Have few, if any, mechanical parts to produce long term reliability
  • Operate in a wide range of conditions (temperature, humidity, vibration, etc.)
  • Have a long operational life consistent with the IoT endpoint life
  • Contain significant battery backup
  • Be able to safely store data with a loss of power
  • Require low power to operate

Since this is a new device to manage, expanded network, system and application management tools have to be implemented.

Should it be Both Ways, i.e. Hybrid?

This is another situation where a hybrid approach will be very likely. The processing of data and actions that need to be taken in sub-second time periods will be assigned to the edge processor. For those functions that can be performed in seconds, the cloud or data center can perform the work.

In the end all the data collected from IoT endpoints will probably be stored elsewhere. The best place for that is the cloud or data center, not in the edge processor. Historical information, time-consuming analysis, and modified processing analytics of the IoT data should reside in the cloud or data center.

Explore IoT further at Enterprise Connect, coming March 27 to 30 in Orlando, Fla. In the session, "IoT & UC: Connecting Things to People in Your Enterprise," co-moderators Dave Michels and Michelle Burbick will lead a panel discussion on the intersection of these topics, featuring executive panelists from Oracle, Mitel, Sprint, Cisco, and Verizon. View the Unified Communications & Collaboration track, and register now using the code NOJITTER to receive $300 off an Entire Event pass or a free Expo Plus pass.