Cloud computing lacks a measure of success, survey finds

Organizations struggle to determine the impact of their investments

Organizations are struggling with how to measure the payoff from cloud computing, according to a survey conducted by The Open Group.

More than 60 percent of respondents surveyed say their organizations do not currently have a return-on-investment mechanism in place. The Open Group surveyed 307 cloud specialists at global organizations that ranged in size from fewer than 200 employees or more than 5,000 employees in February and March of 2011.

The survey included government IT professionals, but the results have not been broken out by sector, said a spokesperson for The Open Group, a vendor- and technology-neutral consortium that focuses on the development of IT certifications and standards.

Cloud computing provides on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or interaction from the service provider.


Related coverage:

How can you trust a cloud? Verify.


Cloud will continue to be at the forefront of IT discussions due to the potential system optimization and cost savings that are associated with it. However, industrywide motivation and the business impact of cloud implementation remains unclear, the survey states.

“Return on investment is probably the most commonly used measure of success of a technical change," Chris Harding, forum director for The Open Group Cloud Computing Work Group, wrote in a blog on May 9.

Some survey respondents think that cloud ROI should be easy to evaluate and justify. Cost, quality of delivered result, utilization, speed of operation, and scale of operation are the most useful metrics. But only 35 percent have mechanisms in place to measure cloud ROI, as opposed to 45 percent that did not, with the other 20 percent being unsure, Harding said.

“The question on the impact of cloud produced the most striking of the survey’s results,” he said. “While 82 percent said that they expected their cloud initiatives to have significant impact on one or more business processes, only 28 percent said that they were prepared for these changes,” Harding said.

Last year, The Open Group released a white paper, “Building Return on Investment from Cloud Computing,” to give organizations guidance on measuring cloud ROI.

The Open Group plans to do more to develop understanding of the business impact of cloud computing, Harding said. The organization will publish The Open Group Guide to Cloud Computing for Business later this year and will continue to develop the theme at conferences and in the Cloud Computing Work Group discussions, he added.

Federal and state agencies are increasingly moving operations to the cloud, looking to free up data center space, cut maintenance costs and power use, increase the availability of systems for mobile users, and, above all, save money. Also, the Office of Management and Budget requires federal agencies to move three applications to the cloud in the next 12 to 18 months.

Major conclusions of the survey include:

  • A majority of organizations needed buy-in from the CIO or another C-level executive to fund cloud computing initiatives.
  • The main drivers behind cloud computing implementations were cost, resource optimization and timeliness/agility of new services.
  • Most organizations using cloud computing did not have an ROI mechanism in place.
  • Eighty-two percent of survey participants said cloud computing would significantly impact one or more business processes.
  • The top concerns surrounding cloud computing were security, governance, integration issues and ability to cope with business process change.

About the Author

Rutrell Yasin is is a freelance technology writer for GCN.

Reader Comments

Mon, Jun 6, 2011 Tom Price Texas

Companies working in a legacy environment have chargeback tools to measure costs and ROI, but when migrating a portion of their workloads to a cloud computing infrastructure, they need a cost optimization tool designed for heterogeneous environments; Cloud Cruiser is one example. Organizations moving from a commoditized to an elasticized infrastructure, whether private/hybrid/or public cloud, need a granular, proactive view of their costs down to the individual user level. This is called organizational mapping. As a result of these business analytics, they can determine workloads in real time through alerts and ensure that all work from a project, to a business unit is aligned with the company’s strategic direction.

Wed, May 11, 2011 Felix Rausch Alexandria Va

People just don't get it. Look at the last paragraph of the article: The question on the impact of cloud produced the most striking of the survey’s results, he said. “While 82 percent said that they expected their cloud initiatives to have significant impact on one or more business processes, only 28 percent said that they were prepared for these changes,” Harding said. DUH-- when will we ever figure out once and for all that planning is needed and such complex and risky "outsourcing" to the "cloud" must be architected. Not learning from mistakes in the past is the definition for .... FEAC Institute graduates completed many projects in their EA certification program Practica where teams successfully architected for such unknown environments. Its about holistically considering and modeling all of the stakeholders' needs and issues directly or indirectly related that can effect one's decision how far to go into the cloud.

Tue, May 10, 2011 Rob Livingstone

I have had solid realworld commercial and technical experience in implementing private cloud and integrating it with public cloud in the enterprise. The reality has only some bearing on the ‘hype’ of quick, easy and cheap in many cases. It was only after I transformed an organisation’s entire IT cost base to a per user per month per application (PUMPA) equivalent, did I prove that a ‘low cost’ enterprise SaaS system was one of the highest per user per month costs for all enterprise apps, companywide. It was more expensive on a PUPM basis than the BI system, and well above the ERP system. Not to mention the complexity and effort in managing the SaaS system’s governance to meet minimum SDLC change control criteria. What percentage of SaaS providers can support an Escrow arrangement in case they go out of business (and some will)? Some can provide some sort of Escrow, others just cannot. Moreover, the ‘commodity’ computing – electricity grid analogy is only partly true (at this stage in the maturity of Cloud) as, unlike the real utility companies, the interchange barriers for Cloud are rich with cost, risk and governance challenges for the unwary. The true analogy is like having to re-wire all or part of your house if you wished to switch electicity utility providers! Proceed in haste, count the cost at your liesure, or you may be lucky and hit your goal first time. Most importantly, know which questions to ask. Undoubtedly, in the right hands, and with the right due diligence, SaaS has the potential to be extremely powerful, high value and even transformational, however it’s not the answer to the world’s IT problems (in the enterprise, at least, at this stage) The benefits are often true, however the cost, risk and other issues are sometimes not clearly exposed for scritiny and assessment by the consumer of Cloud computing. If you don't know the TRUE cost (and risk), how can you do an ROI, irrespective of the intangibles? A balanced approach is needed, especially in corporations where risk, cost and governance are of prime concerns.

Please post your comments here. Comments are moderated, so they may not appear immediately after submitting. We will not post comments that we consider abusive or off-topic.

Please type the letters/numbers you see above