Challenges in Heterogeneous Resources and the Cloud

This is the first post in a series of blog posts that describe the fundamental concepts behind CloudLightning. In this post we will establish some basic terminology, discuss the challenges of heterogeneous resources and the cloud and outline the goals and objectives of the project.


Cloud computing is defined by the National Institute of Standards and Technology (NIST) as a model for enabling ubiquitous, convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction.

It represents a convergence of two major trends in information technology:

  • IT efficiency, whereby modern computers are utilised more efficiently via highly scalable hardware and software resources
  • Business agility, whereby IT can be used as a competitive tool through rapid deployment, parallel batch processing, use of computer-intensive business analytics and mobile interactive applications that respond in real time to consumer requirements (Kim, 2009).

Current cloud infrastructures are mostly homogeneous composed of a large number of machines of the same type – centrally managed and made available to the end user using the three standard service models:

  • Infrastructure as a Service (IaaS)
  • Platform as a Service (PaaS)
  • Software as a Service (SaaS)

As clouds increase in size and as machines of different types are added to the infrastructure in order to maximize performance and power efficiency, heterogeneous clouds are being created. The heterogeneous cloud is still at a nascent stage, however, larger cloud infrastructure providers are offering commercial services e.g., Amazon Web Services offers a variety of GPU services. As demand for better processor, price and power performance increases, it is anticipated that larger infrastructure providers will need to cater for several of these processor types and specifically for the emerging HPC public cloud market.

However, integrating and managing these different architectures independently and with an existing general purpose cloud architecture is not without challenges; not least of which is access to a pool of qualified engineers with deep IT knowledge. The adoption of heterogeneous resources will dramatically increase the complexity of an already complex cloud ecosystem. We present self-organisation and self-management as powerful techniques for addressing this complexity.

The adoption of heterogeneous computing resources by cloud consumers will also allow for improved resource efficiency and hence reduced energy use. Market demand for greater resource management at the PaaS and IaaS layers combined with both demand and adoption of heterogenous resources is rapidly increasing complexity of the cloud ecosystem which over time will render traditional cloud management techniques ineffectual.

An important objective of CloudLightning is to remove the burden of low-level service provisioning, optimisation and orchestration from the cloud consumer and to vest them in the collective response of the individual resource elements comprising the cloud infrastructure.Our goal is to address this inefficient use of resources and consequently to deliver savings to the cloud provider and the cloud consumer in terms of reduced power consumption and improved service delivery, with hyperscale systems particularly in mind.

Leave a Reply