cloud computing – A perspective

cc1Cloud computing is the delivery of computing as a service rather than a product, whereby shared resources, software, and information are provided to computers and other devices as a utility (like the electricity grid) over a network (typically the Internet )

Cloud computing entrusts services (typically centralized) with a user’s data, software and computation on a published application programming interface (API) over a network. It has considerable overlap with software as a service (SaaS).

End users access cloud based applications through a web browser or a light weight desktop or mobile app while the business software and data are stored on servers at a remote location. Cloud application providers strive to give the same or better service and performance than if the software programs were installed locally on end-user computers.

At the foundation of cloud computing is the broader concept of infrastructure convergence (or Converged Infrastructure) and shared services. This type of data centre environment allows enterprises to get their applications up and running faster, with easier manageability and less maintenance, and enables IT to more rapidly adjust IT resources (such as servers, storage, and networking) to meet fluctuating and unpredictable business demand.

Cloud computing providers offer their services according to three fundamental models: Infrastructure as a service (IaaS), platform as a service (PaaS), and software as a service (SaaS) where IaaS is the most basic and each higher model abstracts from the details of the lower models.

Platform as a Service (PaaS) is a way to rent hardware, operating systems, storage and network capacity over the Internet. The service delivery model allows the customer to rent virtualized servers and associated services for running existing applications or developing and testing new ones.

Platform as a Service (PaaS) is an outgrowth of Software as a Service (SaaS), a software distribution model in which hosted software applications are made available to customers over the Internet. PaaS has several advantages for developers. With PaaS, operating system features can be changed and upgraded frequently. Geographically distributed development teams can work together on software development projects. Services can be obtained from diverse sources that cross international boundaries. Initial and ongoing costs can be reduced by the use of infrastructure services from a single vendor rather than maintaining multiple hardware facilities that often perform duplicate functions or suffer from incompatibility problems. Overall expenses can also be minimized by unification of programming development efforts.

On the downside, PaaS involves some risk of “lock-in” if offerings require proprietary service interfaces or development languages. Another potential pitfall is that the flexibility of offerings may not meet the needs of some users whose requirements rapidly evolve.

 Infrastructure as a Service

In this most basic cloud service model, cloud providers offer computers – as physical or more often as virtual machines –, raw (block) storage, firewalls, load balancers, and networks. IaaS providers supply these resources on demand from their large pools installed in data centers. Local area networks including IP addresses are part of the offer. For the wide area connectivity, the Internet can be used or – in carrier clouds – dedicated virtual private networks can be configured.

To deploy their applications, cloud users then install operating system images on the machines as well as their application software. In this model, it is the cloud user who is responsible for patching and maintaining the operating systems and application software. Cloud providers typically bill IaaS services on a utility computing basis, that is, cost will reflect the amount of resources allocated and consumed.

Platform as a Service
(PaaS)

In the PaaS model, cloud providers deliver a computing platform and/or solution stack typically including operating system, programming language execution environment, database, and web server. Application developers can develop and run their software solutions on a cloud platform without the cost and complexity of buying and managing the underlying hardware and software layers. With some PaaS offers, the underlying compute and storage resources scale automatically to match application demand such that the cloud user does not have to allocate resources manually.

 Software as a Service (SaaS)

In this model, cloud providers install and operate application software in the cloud and cloud users access the software from cloud clients. The cloud users do not manage the cloud infrastructure and platform on which the application is running. This eliminates the need to install and run the application on the cloud user’s own computers simplifying maintenance and support. What makes a cloud application different from other applications is its elasticity. This can be achieved by cloning tasks onto multiple virtual machines at run-time to meet the changing work demand. Load balancers distribute the work over the set of virtual machines. This process is
transparent to the cloud user who sees only a single access point. To accomodate a large number of cloud users, cloud applications can be multitenant, that is, any machine serves more than one cloud user organization.
It is common to refer to special types of cloud based application software with a similar naming convention: desktop as a service, business process as a service, Test Environment as a Service, communication as a service.

The pricing model for SaaS applications is typically a monthly or yearly flat fee per user.

Essential Characteristics:

On-demand self-service. A consumer can unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with each service provider.
Broad network access. Capabilities are available over the network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, tablets, laptops, and workstations).
Resource pooling. The provider’s computing resources are pooled to serve multiple consumers using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to consumer demand. There is a sense of location independence in that the customer generally has no control or knowledge over the exact location of the provided resources but may be able to specify location at a higher level of abstraction (e.g., country, state, or datacenter). Examples of resources include storage, processing, memory, and network bandwidth.
Rapid elasticity. Capabilities can be elastically provisioned and released, in some cases automatically, to scale rapidly outward and inward commensurate with demand. To the consumer, the capabilities available for provisioning often appear to be unlimited and can be appropriated in any quantity at any time.
Measured service. Cloud systems automatically control and optimize resource use by leveraging a metering capability1 at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported, providing transparency for both the provider and consumer of the utilized service.

Service Models:

Cloud clients

Users access cloud computing using networked client devices, such as desktop computers, laptops, tablets and smartphones. Some of these devices – cloud clients – rely on cloud computing for all or a majority of their applications so as to be essentially useless without it. Examples are thin clients and the browser-based Chrome-book. Many cloud applications do not require specific software on the client and instead use a web browser to interact with the cloud application. With Ajax and HTML5 these Web user interfaces can achieve a similar or even better look and feel as native applications. Some cloud applications, however, support specific client
software dedicated to these applications (e.g., virtual desktop clients and most email clients). Some legacy applications (line of business applications that until now have been prevalent in thin client Windows computing) are delivered via a screen-sharing technology.

 Deployment models

Cloud computing types Public cloud Applications, storage, and other resources are made available to the general public by a service provider. Public cloud services may be free or offered on a pay-per-usage model. There are limited service providers like Microsoft, Google etc owns all Infrastructure at their Data Center and the access will be through Internet mode only. No direct connectivity proposed in Public Cloud Architecture.

Community cloud
Community cloud shares infrastructure between several organizations from a specific community with common concerns (security, compliance, jurisdiction, etc.), whether managed internally or by a third-party and hosted internally or externally. The costs are spread over fewer users than a public cloud (but more than a private cloud), so only some of the cost savings potential of cloud computing are realized.

 Hybrid cloud
Hybrid cloud is a composition of two or more clouds (private, community or public)
that remain unique entities but are bound together, offering the benefits of multiple deployment models.

 Private cloud
Private cloud is cloud infrastructure operated solely for a single organization, whether managed internally or by a third-party and hosted internally or externally.

They have attracted criticism because users “still have to buy, build, and manage them” and thus do not benefit from less hands-on management, essentially “[lacking] the economic model that makes cloud computing such an intriguing concept”.

Private cloud is a computing model that uses resources which are dedicated to your organization. A private cloud shares many of the characteristics of public cloud computing including resource pooling, self-service, elasticity and pay-by-use delivered in a standardized manner with the additional control and customization available from dedicated resources.
While virtualization is an important technological component of private cloud, the key differentiators is the continued abstraction of computing resources from infrastructure and the machines (virtual or otherwise) used to deliver those resources.
Only by delivering this abstraction can customers achieve the benefits of private cloud – including improved agility and responsiveness, reduced TCO, and increased business alignment and focus. Most importantly, a private cloud promises to exceed the cost effectiveness of a virtualized infrastructure through higher workload density and greater resource utilization.

With a private cloud, you get many of the benefits of public cloud computing—including self-service, scalability, and elasticity—with the additional control and customization available from dedicated resources.
Two models for cloud services can be delivered in a private cloud:
Infrastructure as a Service (IaaS) and Platform as a Service (PaaS). With IaaS, you can use infrastructure resources (compute, network, and storage) as a service, while PaaS provides a complete application platform as a service.

 Architecture

Cloud architecture, the systems architecture of the software systems involved in the delivery of cloud computing, typically involves multiple cloud components communicating with each other over a loose coupling mechanism such as a messaging queue.
Elastic provision implies intelligence in the use of tight or loose coupling as applied to mechanisms such as these and others.

 The Inter-cloud

The Inter-cloud is an interconnected global “cloud of clouds” and an extension of the Internet “network of networks” on which it is based.

 Cloud engineering
Cloud engineering is the application of engineering disciplines to cloud computing. It brings a systematic approach to the high level concerns of commercialization, standardization, and governance in conceiving, developing, operating and maintaining cloud computing systems. It is a multidisciplinary method encompassing contributions from diverse areas such as systems, software, web, performance, information, security, platform, risk, and quality engineering.

Private cloud solutions are from Microsoft and VMware

.
Microsoft private cloud solutions are licensed on a per processor basis, so customers get the cloud computing benefits of scale with unlimited virtualization and lower costs – consistently and predictably over time.

VMware private cloud solutions are licensed by either the number of virtual machines or the virtual memory allocated to those virtual machines – charging you more as you grow. This difference in approach means that with Microsoft your private cloud ROI increases as your private cloud workload density increases. With VMware,  cost grows, as  workload density does. The reason for this significant cost difference is VMware’s per-VM licensing for private cloud products like VMware vCenter Operations Management Suite   and drives this cost differential.

Economics has always been a powerful force in driving industry transformations and as more and more customers evaluate cloud computing investments that will significantly affect ROI, now is the time to provide the information they need to make informed decisions, for today and tomorrow.

Microsoft Private Cloud – Unlimited Virtualization Rights Microsoft private cloud solutions are built using Windows Server with Hyper-V and System Center – the combination of which provides enterprise class virtualization, end-to-end service management and deep insight into applications so users can focus more attention on delivering business value.
Microsoft private cloud solutions are delivered through  wide ecosystem of partners and are offered as custom, pre-configured, or hosted offerings – so, no matter  unique business need; there is a Microsoft private cloud solution for it.
Microsoft private cloud solution is licensed through the Microsoft Enrollment for Core Infrastructure1 (ECI) licensing program. ECI is a Microsoft Enterprise Agreement (EA) enrollment, available in two editions (Datacenter and Standard), that allows a
simple and flexible per processor licensing option. Its approach is focused on delivering the benefits of scale – through unlimited virtualization rights and significantly simplified licensing for Windows Server and System Center.

VMware
Private Cloud – Per-VM Licensing In 2011, VMware announced the latest version of its virtualization platform, vSphere 5.0, along with updated versions of surrounding technologies;
vCenter Site Recovery Manager,
vShield Security and vCloud Director.

These products collectively are referred to as Cloud Infrastructure Suite. VMware has also released several management products like vCenter Operations Management Suite and vFabric Application Performance Manager (APM) to provide capabilities like monitoring, application performance management, and configuration
management. To build a comparable private cloud solution using VMware technologies,  require components from VMware Cloud Infrastructure suite, vCenter Operations Management Suite and vFabric APM as a private cloud solution requires capabilities like monitoring, configuration, automation, orchestration and security in addition to the virtualization platform.
Unlike Microsoft ECI Datacenter, VMware Cloud Infrastructure Suite, vCenter Operations Management Suite, and vFabric APM cannot be licensed as a single SKU, but have to be licensed separately for individual products. Moreover, VMware private cloud products follow a combination of three different licensing schemes-   vSphere 5.0 is licensed on a per processor basis with virtual RAM entitlements vCenter is licensed on a per-instance basis Cloud Infrastructure products – vCloud Director, vCenter Site Recovery Manager, and vShield are licensed on a per-VM basis3   vCenter
Operations Management Suite and vFabric APM are licensed on a per-VM
basis5
Microsoft® offers solutions that deliver IaaS and PaaS for both private and public cloud deployments. This  focuses on Microsoft solutions for IaaS and provides an overview of Microsoft Hyper-V™ Cloud, a set of programs and initiatives to help customers and partners accelerate deployment of IaaS

Organizations can build a private cloud today with Windows Server® 2008 R2, Microsoft Hyper-V, and Microsoft System Center.
The foundation is built on the Windows Server platform with the Windows Server Active Directory® identity framework, Hyper-V virtualization capability, and System Center end-to-end service management capabilities.
The new System Center Virtual Machine Manager Self-Service Portal 2.0 simplifies the pooling, allocation, provisioning, and usage tracking of datacenter resources, so that your business units can consume Infrastructure as a Service.

Advertisements

Emergence of Open Source clouds

OpenStack: The Open Source Cloud Operating System

About OpenStack

OpenStack is a cloud operating system that controls large pools of compute, storage, and networking resources throughout a datacenter, all managed through a dashboard that gives administrators control while empowering their users to provision resources through a web interface.

When choosing an open source cloud operating system, a vibrant commercial ecosystem is key to the long term viability of your platform choice. With OpenStack, there are over 180 participating companies and we encourage you to dig in and find the right path for you.

Distributions

To start, there are many ways to install and deploy OpenStack through software distributions, each of which add their own value to the cloud operating system. Software distributions powered by OpenStack include Cloudscaling, Debian, Fedora, Piston Cloud Computing, Red Hat, SwiftStackSUSE, Ubuntu and Stackops. To verify an OpenStack distribution, look for the logo on the right.

What is CloudStack?

Apache CloudStack is open source software designed to deploy and manage large networks of virtual machines, as a highly available, highly scalable Infrastructure as a Service (IaaS) cloud computing platform. CloudStack is used by a number of service providers to offer public cloud services, and by many companies to provide an on-premises (private) cloud offering, or as part of a hybrid cloud solution.

CloudStack is a turnkey solution that includes the entire “stack” of features most organizations want with an IaaS cloud: compute orchestration, Network-as-a-Service, user and account management, a full and open native API, resource accounting, and a first-class User Interface (UI).

Apache CloudStack is an open source Infrastructure-as-a-Service platform that manages and orchestrates pools of storage, network, and computer resources to build a public or private IaaS compute cloud.

With CloudStack you can:

  • Set up an on-demand elastic cloud computing service.
  • Allow end-users to provision resources

Multiple Hypervisor Support

CloudStack works with a variety of hypervisors and hypervisor-like technologies. A single cloud can contain multiple hypervisor implementations. As of the current release CloudStack supports:

  • vSphere (via vCenter)
  • KVM
  • Xenserver
  • LXC
  • BareMetal (via IPMI)

Massively Scalable Infrastructure Management

CloudStack can manage tens of thousands of physical servers installed in geographically distributed datacenters. The management server scales near-linearly eliminating the need for cluster-level management servers. Maintenance or other outages of the management server can occur without affecting the virtual machines running in the cloud.

Automatic Cloud Configuration Management

CloudStack automatically configures the network and storage settings for each virtual machine deployment. Internally, a pool of virtual appliances support the operation of configuration of the cloud itself. These appliances offer services such as firewalling, routing, DHCP, VPN, console proxy, storage acces, and storage replication. The extensive use of horizontally scalable virtual machines simplifies the installation and ongoing operation of a cloud.

Graphical User Interface

CloudStack offers an administrators web interface used for provisioning and managing the cloud, as well as an end-user’s Web interface, used for running VMs and managing VM templates. The UI can be customized to reflect the desired service provider or enterprise look and feel.

API

CloudStack provides a REST-like API for the operation, management and use of the cloud.

AWS EC2 API Support

CloudStack provides an EC2 API translation layer to permit the common EC2 tools to be used in the use of a CloudStack cloud.

High Availability

CloudStack has a number of features to increase the availability of the system. The Management Server itself may be deployed in a multi-node installation where the servers are load balanced. MySQL may be configured to use replication to provide for failover in the event of database loss. For the hosts, CloudStack supports NIC bonding and the use of separate networks for storage as well as iSCSI Multipath.

Deployment Architecture Overview

Generally speaking, most CloudStack deployments consist of the management server and the resources to be managed. During deployment you inform the management server of the resources to be managed, such as IP address blocks, storage devices, hypervisors, and VLANs.

The minimum installation consists of one machine running the CloudStack Management Server and another machine to act as the cloud infrastructure (in this case, a very simple infrastructure consisting of one host running hypervisor software). In its smallest deployment, a single machine can act as both the Management Server and the hypervisor host (using the KVM hypervisor).

_images/basic-deployment.png

A more full-featured installation consists of a highly-available multi-node Management Server installation and up to tens of thousands of hosts using any of several networking technologies.

Management Server Overview

The management server orchestrates and allocates the resources in your cloud deployment.

The management server typically runs on a dedicated machine or as a virtual machine. It controls allocation of virtual machines to hosts and assigns storage and IP addresses to the virtual machine instances. The Management Server runs in an Apache Tomcat container and requires a MySQL database for persistence.

The management server:

  • Provides the web interface for both the adminstrator and end user.
  • Provides the API interfaces for both the CloudStack API as well as the EC2 interface.
  • Manages the assignment of guest VMs to a specific compute resource
  • Manages the assignment of public and private IP addresses.
  • Allocates storage during the VM instantiation process.
  • Manages snapshots, disk images (templates), and ISO images.
  • Provides a single point of configuration for your cloud.

Cloud Infrastructure Overview

Resources within the cloud are managed as follows:

  • Regions: A collection of one or more geographically proximate zones managed by one or more management servers.
  • Zones: Typically, a zone is equivalent to a single datacenter. A zone consists of one or more pods and secondary storage.
  • Pods: A pod is usually a rack, or row of racks that includes a layer-2 switch and one or more clusters.
  • Clusters: A cluster consists of one or more homogenous hosts and primary storage.
  • Host: A single compute node within a cluster; often a hypervisor.
  • Primary Storage: A storage resource typically provided to a single cluster for the actual running of instance disk images. (Zone-wide primary storage is an option, though not typically used.)
  • Secondary Storage: A zone-wide resource which stores disk templates, ISO images, and snapshots.

Networking Overview

CloudStack offers many types of networking, but they typically fall into one of two scenarios:

  • Basic: Most analogous to AWS-classic style networking. Provides a single flat layer-2 network where guest isolation is provided at layer-3 by the hypervisors bridge device.
  • Advanced: This typically uses layer-2 isolation such as VLANs, though this category also includes SDN technologies such as Nicira NVP.

Cloud Infrastructure Concepts

Regions

To increase reliability of the cloud, you can optionally group resources into multiple geographic regions. A region is the largest available organizational unit within a CloudStack deployment. A region is made up of several availability zones, where each zone is roughly equivalent to a datacenter. Each region is controlled by its own cluster of Management Servers, running in one of the zones. The zones in a region are typically located in close geographical proximity. Regions are a useful technique for providing fault tolerance and disaster recovery.

By grouping zones into regions, the cloud can achieve higher availability and scalability. User accounts can span regions, so that users can deploy VMs in multiple, widely-dispersed regions. Even if one of the regions becomes unavailable, the services are still available to the end-user through VMs deployed in another region. And by grouping communities of zones under their own nearby Management Servers, the latency of communications within the cloud is reduced compared to managing widely-dispersed zones from a single central Management Server.

Usage records can also be consolidated and tracked at the region level, creating reports or invoices for each geographic region.

_images/region-overview.pngRegions are visible to the end user. When a user starts a guest VM on a particular CloudStack Management Server, the user is implicitly selecting that region for their guest. Users might also be required to copy their private templates to additional regions to enable creation of guest VMs using their templates in those regions.

Zones

A zone is the second largest organizational unit within a CloudStack deployment. A zone typically corresponds to a single datacenter, although it is permissible to have multiple zones in a datacenter. The benefit of organizing infrastructure into zones is to provide physical isolation and redundancy. For example, each zone can have its own power supply and network uplink, and the zones can be widely separated geographically (though this is not required).

A zone consists of:

  • One or more pods. Each pod contains one or more clusters of hosts and one or more primary storage servers.
  • A zone may contain one or more primary storage servers, which are shared by all the pods in the zone.
  • Secondary storage, which is shared by all the pods in the zone.

_images/zone-overview.pngZones are visible to the end user. When a user starts a guest VM, the user must select a zone for their guest. Users might also be required to copy their private templates to additional zones to enable creation of guest VMs using their templates in those zones.

Zones can be public or private. Public zones are visible to all users. This means that any user may create a guest in that zone. Private zones are reserved for a specific domain. Only users in that domain or its subdomains may create guests in that zone.

Hosts in the same zone are directly accessible to each other without having to go through a firewall. Hosts in different zones can access each other through statically configured VPN tunnels.

For each zone, the administrator must decide the following.

  • How many pods to place in each zone.
  • How many clusters to place in each pod.
  • How many hosts to place in each cluster.
  • (Optional) How many primary storage servers to place in each zone and total capacity for these storage servers.
  • How many primary storage servers to place in each cluster and total capacity for these storage servers.
  • How much secondary storage to deploy in a zone.

In order to support zone-wide functions for VMware, CloudStack is aware of VMware Datacenters and can map each Datacenter to a CloudStack zone. To enable features like storage live migration and zone-wide primary storage for VMware hosts, CloudStack has to make sure that a zone contains only a single VMware Datacenter. Therefore, when you are creating a new CloudStack zone, you can select a VMware Datacenter for the zone. If you are provisioning multiple VMware Datacenters, each one will be set up as a single zone in CloudStack.

Pods

A pod often represents a single rack or row of racks. Hosts in the same pod are in the same subnet. A pod is the second-largest organizational unit within a CloudStack deployment. Pods are contained within zones. Each zone can contain one or more pods. A pod consists of one or more clusters of hosts and one or more primary storage servers. Pods are not visible to the end user.

_images/pod-overview.png

Clusters

A cluster consists of one or more hosts and one or more primary storage resources.

A cluster provides a way to group hosts. To be precise, a cluster is a XenServer server pool, a set of KVM servers, or a VMware cluster preconfigured in vCenter. The hosts in a cluster should all have identical hardware, run the same hypervisor, are on the same subnet, and access the same shared primary storage. Virtual machine instances (VMs) can be live-migrated from one host to another within the same cluster, without interrupting service to the user.

The size of the cluster is limited by the underlying hypervisor, although the CloudStack recommends less in most cases; see Best Practices.

Even when local storage is used exclusively, clusters are still required organizationally, even if there is just one host per cluster.

When VMware is used, every VMware cluster is managed by a vCenter server. An Administrator must register the vCenter server with CloudStack. There may be multiple vCenter servers per zone. Each vCenter server may manage multiple VMware clusters.

Hosts

A host is a single physical computer. Hosts provide the computing resources that run the guest machines.

The host is the smallest organizational unit within a CloudStack deployment and are not visible to an end user.

Primary Storage

Primary storage is associated with a cluster and/or a zone. It stores the disk volumes for all of the VMs running on hosts in that cluster. You can add multiple primary storage servers to a cluster or a zone (at least one is required at the cluster level). Primary storage is typically located close to the hosts for increased performance. CloudStack manages the allocation of guest virtual disks to particular primary storage devices.

Primary storage can be either static or dynamic. Static primary storage is what CloudStack has traditionally supported. In this model, the administrator must present CloudStack with a certain amount of preallocated storage (ex. a volume from a SAN) and CloudStack can place many of its volumes on this storage. In the newer, dynamic model, the administrator can present CloudStack with a storage system itself (i.e. a SAN). CloudStack, working in concert with a plug-in developed for that storage system, can dynamically create volumes on the storage system. A valuable use for this ability is Quality of Service (QoS). If a volume created in CloudStack can be backed by a dedicated volume on a SAN (i.e. a one-to-one mapping between a SAN volume and a CloudStack volume) and the SAN provides QoS functionality, then CloudStack can also orchestrate storage QoS.

CloudStack is designed to work with all standards-compliant iSCSI and NFS servers that are supported by the underlying hypervisor

You may also use local disks as secondary storage, though naturally they don’t support live migration.

Secondary Storage

Secondary storage stores the following:

  • Templates — OS images that can be used to boot VMs and can include additional configuration information, such as installed applications
  • ISO images — disc images containing data or bootable media for operating systems
  • Disk volume snapshots — saved copies of VM data which can be used for data recovery or to create new templates

The items in secondary storage are available to all hosts in the scope of the secondary storage, which may be defined as per zone or per region. CloudStack supports both NFS and Object Storage supporting either the AWS S3 API or the Swift API as a backing store for Secondary Storage.

Physical Networks

One or more physical networks can be associated with each zone. The physical network typically corresponds to a physical NIC on the host. Each physical network can carry one or more types of network traffic. The choices of traffic type for each network vary depending on your network choices.

A physical network is the actual network hardware and wiring in a zone. A zone can have multiple physical networks.

  • An administrator can:
  • Add/Remove/Update physical networks in a zone
  • Configure VLANs on the physical network
  • Configure a name so the network can be recognized by hypervisors
  • Configure the service providers (firewalls, load balancers, etc.) available on a physical network
  • Configure the IP addresses available to a physical network
  • Specify what type of traffic is carried on the physical network, as well as other properties like network speed

Basic Zone Network Types

When basic networking is used, there can be only one physical network in the zone. That physical network carries the following traffic types:

  • Guest: When end users run VMs, they generate guest traffic. The guest VMs communicate with each other over a network that can be referred to as the guest network. Each pod in a basic zone is a broadcast domain, and therefore each pod has a different IP range for the guest network. The administrator must configure the IP range for each pod.
  • Management: When CloudStack’s internal resources communicate with each other, they generate management traffic. This includes communication between hosts, system VMs (VMs used by CloudStack to perform various tasks in the cloud), and any other component that communicates directly with the CloudStack Management Server. You must configure the IP range for the system VMs to use.
  • Public: Public traffic is generated when VMs in the cloud access the Internet. Publicly accessible IPs must be allocated for this purpose. End users can use the CloudStack UI to acquire these IPs to implement NAT between their guest network and the public network, as described in Acquiring a New IP Address.
  • Storage: While labeled “storage” this is specifically about secondary storage, and doesn’t affect traffic for primary storage. This includes traffic such as VM templates and snapshots, which is sent between the secondary storage VM and secondary storage servers. CloudStack uses a separate Network Interface Controller (NIC) named storage NIC for storage network traffic. Use of a storage NIC that always operates on a high bandwidth network allows fast template and snapshot copying. You must configure the IP range to use for the storage network.

In a basic network, configuring the physical network is fairly straightforward. In most cases, you only need to configure one guest network to carry traffic that is generated by guest VMs. If you use a NetScaler load balancer and enable its elastic IP and elastic load balancing (EIP and ELB) features, you must also configure a network to carry public traffic. CloudStack takes care of presenting the necessary network configuration steps to you in the UI when you add a new zone.

Basic Zone Guest IP Addresses

When basic networking is used, CloudStack will assign IP addresses in the CIDR of the pod to the guests in that pod. The administrator must add a Direct IP range on the pod for this purpose. These IPs are in the same VLAN as the hosts.

Advanced Zone Network Types

When advanced networking is used, there can be multiple physical networks in the zone. Each physical network can carry one or more traffic types, and you need to let CloudStack know which type of network traffic you want each network to carry.

The traffic types in an advanced zone are:

  • Guest: When end users run VMs, they generate guest traffic. The guest VMs communicate with each other over a network that can be referred to as the guest network. This network can be isolated or shared. In an isolated guest network, the administrator needs to reserve VLAN ranges to provide isolation for each CloudStack account’s network (potentially a large number of VLANs). In a shared guest network, all guest VMs share a single network.
  • Management: When CloudStack’s internal resources communicate with each other, they generate management traffic. This includes communication between hosts, system VMs (VMs used by CloudStack to perform various tasks in the cloud), and any other component that communicates directly with the CloudStack Management Server. You must configure the IP range for the system VMs to use.
  • Public: Public traffic is generated when VMs in the cloud access the Internet. Publicly accessible IPs must be allocated for this purpose. End users can use the CloudStack UI to acquire these IPs to implement NAT between their guest network and the public network, as described in “Acquiring a New IP Address” in the Administration Guide.
  • Storage: While labeled “storage” this is specifically about secondary storage, and doesn’t affect traffic for primary storage. This includes traffic such as VM templates and snapshots, which is sent between the secondary storage VM and secondary storage servers. CloudStack uses a separate Network Interface Controller (NIC) named storage NIC for storage network traffic. Use of a storage NIC that always operates on a high bandwidth network allows fast template and snapshot copying. You must configure the IP range to use for the storage network.

These traffic types can each be on a separate physical network, or they can be combined with certain restrictions.

Advanced Zone Guest IP Addresses

When advanced networking is used, the administrator can create additional networks for use by the guests. These networks can span the zone and be available to all accounts, or they can be scoped to a single account, in which case only the named account may create guests that attach to these networks. The networks are defined by a VLAN ID, IP range, and gateway. The administrator may provision thousands of these networks if desired. Additionally, the administrator can reserve a part of the IP address space for non-CloudStack VMs and servers.

Advanced Zone Public IP Addresses

In an advanced network, Public IP addresses are typically on one or more dedicated VLANs and are routed or NATed to guest VMs.

System Reserved IP Addresses

In each zone, you need to configure a range of reserved IP addresses for the management network. This network carries communication between the CloudStack Management Server and various system VMs, such as Secondary Storage VMs, Console Proxy VMs, and DHCP.

The reserved IP addresses must be unique across the cloud. You cannot, for example, have a host in one zone which has the same private IP address as a host in another zone.

The hosts in a pod are assigned private IP addresses. These are typically RFC1918 addresses. The Console Proxy and Secondary Storage system VMs are also allocated private IP addresses in the CIDR of the pod that they are created in.

Make sure computing servers and Management Servers use IP addresses outside of the System Reserved IP range. In example, suppose the System Reserved IP range starts at 192.168.154.2 and ends at 192.168.154.7. CloudStack can use .2 to .7 for System VMs. This leaves the rest of the pod CIDR, from .8 to .254, for the Management Server and hypervisor hosts.

In all zones

Provide private IPs for the system in each pod and provision them in CloudStack.

For KVM and XenServer, the recommended number of private IPs per pod is one per host. If you expect a pod to grow, add enough private IPs now to accommodate the growth.

In a zone that uses advanced networking

For zones with advanced networking, we recommend provisioning enough private IPs for your total number of customers, plus enough for the required CloudStack System VMs. Typically, about 10 additional IPs are required for the System VMs. For more information about System VMs, see the section on working with SystemVMs in the Administrator’s Guide.

When advanced networking is being used, the number of private IP addresses available in each pod varies depending on which hypervisor is running on the nodes in that pod. Citrix XenServer and KVM use link-local addresses, which in theory provide more than 65,000 private IP addresses within the address block. As the pod grows over time, this should be more than enough for any reasonable number of hosts as well as IP addresses for guest virtual routers. VMWare ESXi, by contrast uses any administrator-specified subnetting scheme, and the typical administrator provides only 255 IPs per pod. Since these are shared by physical machines, the guest virtual router, and other entities, it is possible to run out of private IPs when scaling up a pod whose nodes are running ESXi.

To ensure adequate headroom to scale private IP space in an ESXi pod that uses advanced networking, use one or both of the following techniques:

  • Specify a larger CIDR block for the subnet. A subnet mask with a /20 suffix will provide more than 4,000 IP addresses.
  • Create multiple pods, each with its own subnet. In example, if you create 10 pods and each pod has 255 IPs, this will provide 2,550 IP addresses.

CloudStack Terminology

About Regions

To increase reliability of the cloud, you can optionally group resources into multiple geographic regions. A region is the largest available organizational unit within a CloudStack deployment. A region is made up of several availability zones, where each zone is roughly equivalent to a datacenter. Each region is controlled by its own cluster of Management Servers, running in one of the zones. The zones in a region are typically located in close geographical proximity. Regions are a useful technique for providing fault tolerance and disaster recovery.

By grouping zones into regions, the cloud can achieve higher availability and scalability. User accounts can span regions, so that users can deploy VMs in multiple, widely-dispersed regions. Even if one of the regions becomes unavailable, the services are still available to the end-user through VMs deployed in another region. And by grouping communities of zones under their own nearby Management Servers, the latency of communications within the cloud is reduced compared to managing widely-dispersed zones from a single central Management Server.

Usage records can also be consolidated and tracked at the region level, creating reports or invoices for each geographic region.

region-overview.png: Nested structure of a region.

Regions are visible to the end user. When a user starts a guest VM on a particular CloudStack Management Server, the user is implicitly selecting that region for their guest. Users might also be required to copy their private templates to additional regions to enable creation of guest VMs using their templates in those regions.

About Zones

A zone is the second largest organizational unit within a CloudStack deployment. A zone typically corresponds to a single datacenter, although it is permissible to have multiple zones in a datacenter. The benefit of organizing infrastructure into zones is to provide physical isolation and redundancy. For example, each zone can have its own power supply and network uplink, and the zones can be widely separated geographically (though this is not required).

A zone consists of:

  • One or more pods. Each pod contains one or more clusters of hosts and one or more primary storage servers.
  • A zone may contain one or more primary storage servers, which are shared by all the pods in the zone.
  • Secondary storage, which is shared by all the pods in the zone.

zone-overview.png: Nested structure of a simple zone.

Zones are visible to the end user. When a user starts a guest VM, the user must select a zone for their guest. Users might also be required to copy their private templates to additional zones to enable creation of guest VMs using their templates in those zones.

Zones can be public or private. Public zones are visible to all users. This means that any user may create a guest in that zone. Private zones are reserved for a specific domain. Only users in that domain or its subdomains may create guests in that zone.

Hosts in the same zone are directly accessible to each other without having to go through a firewall. Hosts in different zones can access each other through statically configured VPN tunnels.

For each zone, the administrator must decide the following.

  • How many pods to place in each zone.
  • How many clusters to place in each pod.
  • How many hosts to place in each cluster.
  • (Optional) How many primary storage servers to place in each zone and total capacity for these storage servers.
  • How many primary storage servers to place in each cluster and total capacity for these storage servers.
  • How much secondary storage to deploy in a zone.

When you add a new zone using the CloudStack UI, you will be prompted to configure the zone’s physical network and add the first pod, cluster, host, primary storage, and secondary storage.

In order to support zone-wide functions for VMware, CloudStack is aware of VMware Datacenters and can map each Datacenter to a CloudStack zone. To enable features like storage live migration and zone-wide primary storage for VMware hosts, CloudStack has to make sure that a zone contains only a single VMware Datacenter. Therefore, when you are creating a new CloudStack zone, you can select a VMware Datacenter for the zone. If you are provisioning multiple VMware Datacenters, each one will be set up as a single zone in CloudStack.

Note

If you are upgrading from a previous CloudStack version, and your existing deployment contains a zone with clusters from multiple VMware Datacenters, that zone will not be forcibly migrated to the new model. It will continue to function as before. However, any new zone-wide operations, such as zone-wide primary storage and live storage migration, will not be available in that zone.

About Pods

A pod often represents a single rack. Hosts in the same pod are in the same subnet. A pod is the third-largest organizational unit within a CloudStack deployment. Pods are contained within zones. Each zone can contain one or more pods. A pod consists of one or more clusters of hosts and one or more primary storage servers. Pods are not visible to the end user.

pod-overview.png: Nested structure of a simple pod

About Clusters

A cluster provides a way to group hosts. To be precise, a cluster is a XenServer server pool, a set of KVM servers, , or a VMware cluster preconfigured in vCenter. The hosts in a cluster all have identical hardware, run the same hypervisor, are on the same subnet, and access the same shared primary storage. Virtual machine instances (VMs) can be live-migrated from one host to another within the same cluster, without interrupting service to the user.

A cluster is the fourth-largest organizational unit within a CloudStack deployment. Clusters are contained within pods, and pods are contained within zones. Size of the cluster is limited by the underlying hypervisor, although the CloudStack recommends less in most cases; see Best Practices.

A cluster consists of one or more hosts and one or more primary storage servers.

cluster-overview.png: Structure of a simple cluster

CloudStack allows multiple clusters in a cloud deployment.

Even when local storage is used exclusively, clusters are still required organizationally, even if there is just one host per cluster.

When VMware is used, every VMware cluster is managed by a vCenter server. An Administrator must register the vCenter server with CloudStack. There may be multiple vCenter servers per zone. Each vCenter server may manage multiple VMware clusters.

About Hosts

A host is a single computer. Hosts provide the computing resources that run guest virtual machines. Each host has hypervisor software installed on it to manage the guest VMs. For example, a host can be a Citrix XenServer server, a Linux KVM-enabled server, an ESXi server, or a Windows Hyper-V server.

The host is the smallest organizational unit within a CloudStack deployment. Hosts are contained within clusters, clusters are contained within pods, pods are contained within zones, and zones can be contained within regions.

Hosts in a CloudStack deployment:

  • Provide the CPU, memory, storage, and networking resources needed to host the virtual machines
  • Interconnect using a high bandwidth TCP/IP network and connect to the Internet
  • May reside in multiple data centers across different geographic locations
  • May have different capacities (different CPU speeds, different amounts of RAM, etc.), although the hosts within a cluster must all be homogeneous

Additional hosts can be added at any time to provide more capacity for guest VMs.

CloudStack automatically detects the amount of CPU and memory resources provided by the hosts.

Hosts are not visible to the end user. An end user cannot determine which host their guest has been assigned to.

For a host to function in CloudStack, you must do the following:

  • Install hypervisor software on the host
  • Assign an IP address to the host
  • Ensure the host is connected to the CloudStack Management Server.

About Primary Storage

Primary storage is associated with a cluster or (in KVM and VMware) a zone, and it stores the disk volumes for all the VMs running on hosts.

You can add multiple primary storage servers to a cluster or zone. At least one is required. It is typically located close to the hosts for increased performance. CloudStack manages the allocation of guest virtual disks to particular primary storage devices.

It is useful to set up zone-wide primary storage when you want to avoid extra data copy operations. With cluster-based primary storage, data in the primary storage is directly available only to VMs within that cluster. If a VM in a different cluster needs some of the data, it must be copied from one cluster to another, using the zone’s secondary storage as an intermediate step. This operation can be unnecessarily time-consuming.

For Hyper-V, SMB/CIFS storage is supported. Note that Zone-wide Primary Storage is not supported in Hyper-V.

CloudStack is designed to work with all standards-compliant iSCSI and NFS servers that are supported by the underlying hypervisor, including, for example:

  • SolidFire for iSCSI
  • Dell EqualLogic™ for iSCSI
  • Network Appliances filers for NFS and iSCSI
  • Scale Computing for NFS

If you intend to use only local disk for your installation, you can skip adding separate primary storage.

About Secondary Storage

Secondary storage stores the following:

  • Templates — OS images that can be used to boot VMs and can include additional configuration information, such as installed applications
  • ISO images — disc images containing data or bootable media for operating systems
  • Disk volume snapshots — saved copies of VM data which can be used for data recovery or to create new templates

The items in secondary storage are available to all hosts in the scope of the secondary storage, which may be defined as per zone or per region.

To make items in secondary storage available to all hosts throughout the cloud, you can add object storage in addition to the zone-based NFS Secondary Staging Store. It is not necessary to copy templates and snapshots from one zone to another, as would be required when using zone NFS alone. Everything is available everywhere.

For Hyper-V hosts, SMB/CIFS storage is supported.

CloudStack provides plugins that enable both OpenStack Object Storage (Swift, swift.openstack.org) and Amazon Simple Storage Service (S3) object storage. When using one of these storage plugins, you configure Swift or S3 storage for the entire CloudStack, then set up the NFS Secondary Staging Store for each zone. The NFS storage in each zone acts as a staging area through which all templates and other secondary storage data pass before being forwarded to Swift or S3. The backing object storage acts as a cloud-wide resource, making templates and other data available to any zone in the cloud.

Warning

Heterogeneous Secondary Storage is not supported in Regions. For example, you cannot set up multiple zones, one using NFS secondary and the other using S3 or Swift secondary.

About Physical Networks

Part of adding a zone is setting up the physical network. One or (in an advanced zone) more physical networks can be associated with each zone. The network corresponds to a NIC on the hypervisor host. Each physical network can carry one or more types of network traffic. The choices of traffic type for each network vary depending on whether you are creating a zone with basic networking or advanced networking.

A physical network is the actual network hardware and wiring in a zone. A zone can have multiple physical networks. An administrator can:

  • Add/Remove/Update physical networks in a zone
  • Configure VLANs on the physical network
  • Configure a name so the network can be recognized by hypervisors
  • Configure the service providers (firewalls, load balancers, etc.) available on a physical network
  • Configure the IP addresses trunked to a physical network
  • Specify what type of traffic is carried on the physical network, as well as other properties like network speed

Basic Zone Network Traffic Types

When basic networking is used, there can be only one physical network in the zone. That physical network carries the following traffic types:

  • Guest. When end users run VMs, they generate guest traffic. The guest VMs communicate with each other over a network that can be referred to as the guest network. Each pod in a basic zone is a broadcast domain, and therefore each pod has a different IP range for the guest network. The administrator must configure the IP range for each pod.
  • Management. When CloudStack’s internal resources communicate with each other, they generate management traffic. This includes communication between hosts, system VMs (VMs used by CloudStack to perform various tasks in the cloud), and any other component that communicates directly with the CloudStack Management Server. You must configure the IP range for the system VMs to use.

Note

We strongly recommend the use of separate NICs for management traffic and guest traffic.

  • Public. Public traffic is generated when VMs in the cloud access the Internet. Publicly accessible IPs must be allocated for this purpose. End users can use the CloudStack UI to acquire these IPs to implement NAT between their guest network and the public network, as described in Acquiring a New IP Address.
  • Storage. While labeled “storage” this is specifically about secondary storage, and doesn’t affect traffic for primary storage. This includes traffic such as VM templates and snapshots, which is sent between the secondary storage VM and secondary storage servers. CloudStack uses a separate Network Interface Controller (NIC) named storage NIC for storage network traffic. Use of a storage NIC that always operates on a high bandwidth network allows fast template and snapshot copying. You must configure the IP range to use for the storage network.

In a basic network, configuring the physical network is fairly straightforward. In most cases, you only need to configure one guest network to carry traffic that is generated by guest VMs. If you use a NetScaler load balancer and enable its elastic IP and elastic load balancing (EIP and ELB) features, you must also configure a network to carry public traffic. CloudStack takes care of presenting the necessary network configuration steps to you in the UI when you add a new zone.

Basic Zone Guest IP Addresses

When basic networking is used, CloudStack will assign IP addresses in the CIDR of the pod to the guests in that pod. The administrator must add a Direct IP range on the pod for this purpose. These IPs are in the same VLAN as the hosts.

Advanced Zone Network Traffic Types

When advanced networking is used, there can be multiple physical networks in the zone. Each physical network can carry one or more traffic types, and you need to let CloudStack know which type of network traffic you want each network to carry. The traffic types in an advanced zone are:

  • Guest. When end users run VMs, they generate guest traffic. The guest VMs communicate with each other over a network that can be referred to as the guest network. This network can be isolated or shared. In an isolated guest network, the administrator needs to reserve VLAN ranges to provide isolation for each CloudStack account’s network (potentially a large number of VLANs). In a shared guest network, all guest VMs share a single network.
  • Management. When CloudStack’s internal resources communicate with each other, they generate management traffic. This includes communication between hosts, system VMs (VMs used by CloudStack to perform various tasks in the cloud), and any other component that communicates directly with the CloudStack Management Server. You must configure the IP range for the system VMs to use.
  • Public. Public traffic is generated when VMs in the cloud access the Internet. Publicly accessible IPs must be allocated for this purpose. End users can use the CloudStack UI to acquire these IPs to implement NAT between their guest network and the public network, as described in “Acquiring a New IP Address” in the Administration Guide.
  • Storage. While labeled “storage” this is specifically about secondary storage, and doesn’t affect traffic for primary storage. This includes traffic such as VM templates and snapshots, which is sent between the secondary storage VM and secondary storage servers. CloudStack uses a separate Network Interface Controller (NIC) named storage NIC for storage network traffic. Use of a storage NIC that always operates on a high bandwidth network allows fast template and snapshot copying. You must configure the IP range to use for the storage network.

These traffic types can each be on a separate physical network, or they can be combined with certain restrictions. When you use the Add Zone wizard in the UI to create a new zone, you are guided into making only valid choices.

Advanced Zone Guest IP Addresses

When advanced networking is used, the administrator can create additional networks for use by the guests. These networks can span the zone and be available to all accounts, or they can be scoped to a single account, in which case only the named account may create guests that attach to these networks. The networks are defined by a VLAN ID, IP range, and gateway. The administrator may provision thousands of these networks if desired. Additionally, the administrator can reserve a part of the IP address space for non-CloudStack VMs and servers.

Advanced Zone Public IP Addresses

When advanced networking is used, the administrator can create additional networks for use by the guests. These networks can span the zone and be available to all accounts, or they can be scoped to a single account, in which case only the named account may create guests that attach to these networks. The networks are defined by a VLAN ID, IP range, and gateway. The administrator may provision thousands of these networks if desired.

System Reserved IP Addresses

In each zone, you need to configure a range of reserved IP addresses for the management network. This network carries communication between the CloudStack Management Server and various system VMs, such as Secondary Storage VMs, Console Proxy VMs, and DHCP.

The reserved IP addresses must be unique across the cloud. You cannot, for example, have a host in one zone which has the same private IP address as a host in another zone.

The hosts in a pod are assigned private IP addresses. These are typically RFC1918 addresses. The Console Proxy and Secondary Storage system VMs are also allocated private IP addresses in the CIDR of the pod that they are created in.

Make sure computing servers and Management Servers use IP addresses outside of the System Reserved IP range. For example, suppose the System Reserved IP range starts at 192.168.154.2 and ends at 192.168.154.7. CloudStack can use .2 to .7 for System VMs. This leaves the rest of the pod CIDR, from .8 to .254, for the Management Server and hypervisor hosts.

In all zones:

Provide private IPs for the system in each pod and provision them in CloudStack.

For KVM and XenServer, the recommended number of private IPs per pod is one per host. If you expect a pod to grow, add enough private IPs now to accommodate the growth.

In a zone that uses advanced networking:

For zones with advanced networking, we recommend provisioning enough private IPs for your total number of customers, plus enough for the required CloudStack System VMs. Typically, about 10 additional IPs are required for the System VMs. For more information about System VMs, see the section on working with SystemVMs in the Administrator’s Guide.

When advanced networking is being used, the number of private IP addresses available in each pod varies depending on which hypervisor is running on the nodes in that pod. Citrix XenServer and KVM use link-local addresses, which in theory provide more than 65,000 private IP addresses within the address block. As the pod grows over time, this should be more than enough for any reasonable number of hosts as well as IP addresses for guest virtual routers. VMWare ESXi, by contrast uses any administrator-specified subnetting scheme, and the typical administrator provides only 255 IPs per pod. Since these are shared by physical machines, the guest virtual router, and other entities, it is possible to run out of private IPs when scaling up a pod whose nodes are running ESXi.

To ensure adequate headroom to scale private IP space in an ESXi pod that uses advanced networking, use one or both of the following techniques:

  • Specify a larger CIDR block for the subnet. A subnet mask with a /20 suffix will provide more than 4,000 IP addresses.
  • Create multiple pods, each with its own subnet. For example, if you create 10 pods and each pod has 255 IPs, this will provide 2,550 IP addresses.

CloudStack currently supports the most popular hypervisors: VMware, KVM, XenServer and Xen Cloud Platform (XCP).

Users can manage their cloud with an easy to use Web interface, command line tools, and/or a full-featured RESTful API. In addition, CloudStack provides an API that’s compatible with AWS EC2 and S3 for organizations that wish to deploy hybrid clouds.

Open-source software has been on the rise at many businesses during the extended economic downturn, and one of the areas where it is starting to offer companies a lot of flexibility and cost savings is in cloud computing. Cloud deployments can save money, free businesses from vendor lock-ins that could really sting over time, and offer flexible ways to combine public and private applications. The following are 11 top open-source cloud applications, services, educational resources, support options, general items of interest, and more.

Eucalyptus. Ostatic broke the news about UC Santa Barbara’s open-source cloud project last year. Released as an open-source (under a FreeBSD-style license) infrastructure for cloud computing on clusters that duplicates the functionality of Amazon’s EC2, Eucalyptus directly uses the Amazon command-line tools. Startup Eucalyptus Systems was launched this year with venture funding, and the staff includes original architects from the Eucalyptus project. The company recently released its first major update to the software framework, which is also powering the cloud computing features in the new version of Ubuntu Linux.

Red Hat’s Cloud. Linux-focused open-source player Red Hat has been rapidly expanding its focus on cloud computing. Red Hat held its Open Source Cloud Computing Forum, which included a large number of presentations from movers and shakers focused on open-source cloud initiatives.  The speakers include Rich Wolski (CTO of Eucalyptus Systems), Brian Stevens (CTO of Red Hat), and Mike Olson (CEO of Cloudera). Stevens’ webcast can bring you up to speed on Red Hat’s cloud strategy. Novell is also an open source-focused company that is increasingly focused on cloud computing.

Traffic Server. Yahoo this week moved its open-source cloud computing initiatives up a notch with the donation of its Traffic Server product to the Apache Software Foundation. Traffic Server is used in-house at Yahoo to manage its own traffic, and it enables session management, authentication, configuration management, load balancing, and routing for entire cloud computing software stacks. Acting as an overlay to raw cloud computing services, Traffic Server allows IT administrators to allocate resources, including handling thousands of virtualized services concurrently.

Cloudera. The open-source Hadoop software framework is increasingly used in cloud computing deployments due to its flexibility with cluster-based, data-intensive queries and other tasks. It’s overseen by the Apache Software Foundation, and Yahoo has its own time-tested Hadoop distribution. Cloudera is a promising startup focused on providing commercial support for Hadoop. You can read much more about Cloudera here.

Puppet. Virtual servers are on the rise in cloud computing deployments, and Reductive Labs’ open-source software, built upon the legacy of the Cfengine system, is hugely respected by many system administrators for managing them. You can use it to manage large numbers of systems or virtual machines through automated routines, without having to do a lot of complex scripting.

Enomaly. The company’s Elastic Computing Platform (ECP) has its roots in widely used Enomalism open-source provisioning and management software, designed to take much of the complexity out of starting a cloud infrastructure. ECP is a programmable virtual cloud computing infrastructure for small, medium and large businesses, and you can read much more about it here.

Joyent. In January of this year, Joyent purchased Reasonably Smart, a fledgling open-source cloud startup based on JavaScript and Git. Joyent’s cloud hosting infrastructure and cloud management software incorporate many open-source tools for public and private clouds.  The company can also help you optimize a speedy implementation of the open-source MySQL database for cloud use.

Zoho. Many people use Zoho’s huge suite of free, online applications, which is competitive with Google Docs. What lots of folks don’t realize, though, is that Zoho’s core is completely open source — a shining example of how SaaS solutions can work in harmony with open source. You can find many details on how Zoho deploys open-source tools in this interview.

Globus Nimbus. This open-source toolkit allows businesses to turn clusters into Infrastructure-as-a-Service (IaaS) clouds. The Amazon EC2 interface is carried over, but is not the only interface you can choose.

Reservoir. This is the main European research initiative on virtualized infrastructures and cloud computing. It’s a far-reaching project targeted to develop open-source technology for cloud computing, and help businesses avoid vendor lock-in.

OpenNebula. The OpenNebula VM Manager is a core component of Reservoir. It’s an open-source answer to the many virtual machine management offerings from proprietary players, and interfaces easily with cloud infrastructure tools and services. “OpenNebula is an open-source virtual infrastructure engine that enables the dynamic deployment and re-placement of virtual machines on a pool of physical resources,” according to project leads.

It’s good to see open-source tools and resources competing in the cloud computing space. The end result should be more flexibility for organizations that want to customize their approaches. Open-source cloud offerings also have the potential to keep pricing for all competitive services on a level playing field

One of the leading cloud providers has today revealed a host of upgrades to its OpenSource cloud offerings.
Article | 03.06.13 | By

Rackspace

The open cloud company, Rackspace, has today announced new features in its open source Rackspace Private Cloud Software.

The developments build upon the company’s existing integration and deployment expertise for firms looking to move from the public cloud arena into private enterprise data centres. Central to these new features is OpenCentre, a single interface for deploying, configuring and operating clouds at scale in any enterprise data centre of choice.

As well as OpenCentre, Rackspace has included additional features for customers including a choice of hosts for their operating systems (Ubuntu, Red Hat Enterprise Linux or CentOS). There will also be a point-and-click deployment and management of high availability environments for the OpenStack controller nodes, aimed to improve up-time and reduce firms’ business risks.

Rackspace has announced that it is developing its cloud capabilities with the aim of enabling customers to have a ‘cloud anywhere’ experience. The company promises that the new features to OpenStack will allow customers “to move and scale workloads between Rackspace public and private clouds, while network interoperability enables customers to create a true hybrid cloud on a single network”.

It is a combination of stable OpenStack community ‘trunk’ code and proven Rackspace tools, packaging and configuration.

Juan Caceres, services director at JC Global Resources said of Rackspace Private Cloud Software: “It gives us more choices on how to best manage our cloud environment and bring DevOps capabilities to our customers.  We can either leverage Rackspace to support our environment when needed or have them completely operate it for us – on or off premise.”

The new features come as IDC predict that worldwide spending on hosted private cloud services will surpass $24bn (£16bn) in 2016.

The company was set up in 1998 to deliver enterprise-level hosting services to businesses of all sizes and kinds and now serves more than 205,538 customers. Since its launch in August 2012, the company provides training, support or operations expertise on its Private Cloud powered by OpenStack to over 100 organisations, including 22 per cent of the Fortune 100.

Cloud Conversations: AWS EBS, Glacier and S3 Overview

Cloud Conversations: AWS EBS, Glacier and S3 Overview |

Simple Storage Services (S3), Glacier and Elastic Block Storage (EBS) are part of the AWS cloud storage portfolio of services

http://awsmedia.s3.amazonaws.com/pdf/AWS_Security_Whitepaper.pdf

Amazon Web Services (AWS) recently added EBS Optimized support for enhanced bandwidth EC2 instances (read more here). This industry trends and perspective cloud conversation is the first (looking at EBS) in a three part series companion to the AWS EBS optimized post found here. Part II is here (closer look at S3) and part III is here (tying it all together).For those not familiar, Simple Storage Services (S3), Glacier and Elastic Block Storage (EBS) are part of the AWS cloud storage portfolio of services. There are several other storage and data related service for little data database (SQL and NoSql based) other offerings include compute, data management, application and networking for different needs shown in the following image.AWS services console image
AWS Services Console via www.amazon.com Simple Storage Service (S3) is commonly used in the context of cloud storage and object storage accessed via its S3 API. S3 can be used externally from outside AWS as well as within or via other AWS services. For example with Elastic Cloud Compute (EC2) including via the Amazon Storage Gateway (read more here and about EC2 here). Glacier is the AWS cold or deep storage service for inactive data and is a companion to S3 that you can read more about here.S3 is well suited for both big and little data repositories of objects ranging from backup to archive to active video images and much more. In fact if you are using some of the different AaaS or SaaS services including backup or file and video sharing, those may be using S3 as its back-end storage repository. For example NetFlix leverages various AWS capabilities as part of its data and applications infrastructure (read more here).AWS Basics
AWS consists of multiple regions that contain multiple availability zones where data and applications are supported from.yyyyNote that objects stored in a region never leave that region, for example data stored in the EU west never leave Ireland, or data in the US East never leaves Virginia.AWS does support the ability for user controlled movement of data between regions for business continuance (BC), high availability (HA) and disaster recovery (DR). Read more here at the AWS Security and Compliance site and also in this AWS white paper.

What About EBS?
That brings us to Elastic Block Storage (EBS) that is used by EC2 (read more about EC2 and instances here) as storage for cloud and virtual machines or compute instances. In addition to using S3 as a persistent backing store or target for holding snapshots EBS can be thought of as primary storage. You can provision and allocate EBS volumes in the different data centers of the various AWS availability zones. As part of allocating your EBS volume you indicate the type (standard) or provisioned IOP’s or the new EBS Optimized volumes. EBS Optimized volumes enables instances that support the feature to have better IO performance to storage.The following image shows an EC2 instance with EBS volumes (standard and provisioned IOPS’s) along with S3 volumes and snapshots. In the following example the instance and volumes are being served via the AWS US East region (Northern Virginia) using availability zone US East 1a. In addition, EBS optimized volumes are shown being used in the example to increase bandwidth or throughput performance between storage and the compute instance.xxxxxxxUsing the above as a basis, you can build off of that to leverage multiple availability zones or regions for HA, BC and DR combined with application, network load balancing and other capabilities. Note that EBS volumes are protected for durability by being spread across different servers and storage in an availability zone. Additional protection is provided by using snapshots combined with S3. Additional BC and DR or HA protection can be accomplished by replicating data across availability zones.SQL applications using cloud and object storage servicesThe above is an example of tying various components and services together. For example using different AWS availability zones, instances, EBS, S3 and other tools including those from third parties. Here is a link to a free chapter download from Cloud and Virtual Data Storage Networking (CRC Press) pertaining to data protection, BC and DR (available at Amazon here and Kindle here). In addition here is an AWS white paper pertaining to using their services for BC, HA and DR.EBS volumes are created ranging in size from 1GByte to 1Tbyte in space capacity with multiple volumes being mapped or attached to an EC2 instances. EBS volumes appear as a virtual disk drive for block storage. From the EC2 instance and guest operating system you can mount, format and use the EBS volumes as any other block disk drive with your favorite tools and file systems. In addition to space capacity, EBS volumes are also provisioned with standard IO (e.g. disk based) performance or high performance Provisioned IOPS (e.g. SSD) for thousands of IOPS per instance. AWS states that a standard EBS volume should support about 100 IOP’s on average, with about 2,000 IOPS for a provisioned IOP volume. Need more than 2,000 IOPS, then the AWS recommendation is to use multiple IOP provisioned volumes with data spread across those. Following is an example of AWS EBS volumes seen via the EC2 management interface.Image of mapping AWS EBS to ECS instance
AWS EC2 and EBS configuration statusNote that there is a 10 to 1 ratio of space capacity to IOP’s being provisioned. If you try to play a game of 1,000 IOPS provisioned on a 10GByte EBS volume to keep your costs down you are out of luck. Thus to get 1,000 IOPS’s you would need to allocate at least a 100GByte EBS volume of which you will be billed for the actual space used on a monthly pro-rated basis. The following is an example of provisioning an AWS EBS volume using provisioned IOPS in the US East region in the 1a availability zone.

Cloud conversations: AWS EBS, Glacier and S3 overview (Part II S3)

Storage I/O industry trends image

Amazon Web Services (AWS) recently added EBS Optimized support for enhanced bandwidth EC2 instances (read more here). This industry trends and perspective cloud conversation is the second (looking at S3) in a three-part series companion to the AWS EBS optimized post found here. Part I is here (closer look at EBS) and part III is here (tying it all together).

AWS image via Amazon.com

For those not familiar, Simple Storage Services (S3), Glacier and Elastic Block Storage (EBS) are part of the AWS cloud storage portfolio of services. With S3, you specify a region where a bucket is created that will contain objects that can be written, read, listed and deleted. You can create multiple buckets in a region with unlimited number of objects ranging from 1 byte to 5 Tbytes in size per bucket. Each object has a unique, user or developer assigned access key. In addition to indicating which AWS region, S3 buckets and objects are provisioned using different levels of availability, durability, SLA’s and costs (view S3 SLA’s here).

AWS S3 example image

Cost will vary depending on the AWS region being used, along if Standard or Reduced Redundancy Storage (RSS) selected. Standard S3 storage is designed with 99.999999999% durability (how many copies exists) and 99.99% availability (how often can it be accessed) on an annual basis capable of two data centers becoming un-available.

As its name implies, for a lower fee and level of durability, S3 RRS has an annual durability of 99.999% and availability of 99.99% capable of a single data center loss. In the following figure durability is how many copies of data exist spread across different servers and storage systems in various data centers and availability zones.

cloud storage and object storage across availability zone image

What would you put in RRS vs. Standard S3 storage?

Items that need some level of persistence that can be refreshed, recreated or restored from some other place or pool of storage such as thumbnails or static content or read caches. Other items would be those that you could tolerant some downtime while waiting for data to be restored, recovered or rebuilt from elsewhere in exchange for a lower cost.

Different AWS regions can be chosen for regulatory compliance requirements, performance, SLA’s, cost and redundancy with authentication mechanisms including encryption (SSL and HTTPS) to make sure data is kept secure. Various rights and access can be assigned to objects including making them public or private. In addition to logical data protection (security, identity and access management (IAM), encryption, access control) policies also apply to determine level of durability and availability or accessibility of buckets and objects. Other attributes of buckets and objects include life-cycle management polices and logging of activity to the items. Also part of the objects are meta data containing information about the data being stored shown in a generic example below.

Cloud storage and object storage spread across availability zones figure

Access to objects is via standard REST and SOAP interfaces with an Application Programming Interface (API). For example default access is via HTTP along with a Bit Torrent interface with optional support via various gateways, appliances and software tools.

Cloud storage and object storage IO figure
Example cloud and object storage access

The above figure via Cloud and Virtual Data Storage Networking (CRC Press) shows a generic example applicable to AWS services including S3 being accessed in different ways.

Image of AWS S3 usage
AWS S3 buckets and objects in different regions

The following figure is a generic example of how cloud and object storage are accessed using different tools, hardware, software and API’s along with gateways. AWS is an example of what is shown in the following figure as a Cloud Service and S3, EBS or Glacier as cloud storage. Common example API commands are also shown which will vary by different vendors, products or solution definitions or implementations. While Amazon S3 API which is REST HTTP based has become an industry de facto standard, there are other API’s including CDMI (Cloud Data Management Interface) developed by SNIA which has gained ISO accreditation.

Cloud storage and object storage I/O figure
Cloud and object storage access example via Cloud and Virtual Data Storage Networking

Cyberduck image for cyber duck tool

Cyberduck tool.

Additional reading and related items:

Apple’s iCloud

Apple’s iCloud is most-used cloud service in the US, beating Dropbox & Amazon

With support built into every Mac and iOS device, Apple’s iCloud is the most-used cloud media service by U.S. consumers, a new survey has found.

Cloud

Strategy Analytics graphic via Engadget.

iCloud accounts for 27 percent of cloud customers in America, according to new data published Thursday by Strategy Analytics. That places Apple’s service ahead of second-place Dropbox, with 17 percent, and third-place Amazon, with 15 percent.

Apple’s rival Google comes in fourth with its Google Drive service, used by 10 percent of U.S. consumers. And in fifth is the cloud movie service Ultraviolet, used by just 4 percent of respondents.

The survey of 2,300 people found that cloud storage is particularly popular among people ages 20 to 24, and the most common use for cloud storage is music. Of those surveyed, 90 percent of iCLoud, Amazon and Google Drive users store music files in the cloud.

The story is different with Dropbox users, as 45 percent of them use the service to store music files.

“Music is currently the key battleground in the war for cloud domination,” said Ed Barton, director of Digital Media at Strategy Analytics. “Google is tempting users by giving away free storage for 20,000 songs which can be streamed to any Android device, a feature both Amazon and Apple charge annual subscriptions for.

“However, the growth of video streaming and the desire to access content via a growing range of devices will see services such as the Hollywood-backed digital movie initiative Ultraviolet – currently used by 4% of Americans – increase market share.”

In its quarterly earnings report in January, Apple revealed that it has more than 250 million active iCloud users, growing significantly from 190 million in October. Users are automatically prompted to open a free iCloud account with 5 gigabytes of storage when setting up a new iOS device.

IBM Smart Cloud computing

IBM SmartCloud Foundation is a set of technologies for building and managing virtualized infrastructures and private and hybrid clouds. Together these technologies can help build a fully functional cloud management system that aids business transformation and new service delivery. Individually, these technologies can help nearly any cloud project make quick and incremental progress towards a longer term cloud strategy.

open-cloud-hero

Featured Capabilities

IT service management

Service and IT asset management and process automation across the organization.

IBM® Service Delivery and Process Automation software gives you the visibility, control and automation needed to provide quality service delivery, addressing all stages of the service lifecycle.

Tivoli® Service Delivery and Process Automation software offerings provide a complete solution, automating the full lifecycle of service requests, incidents and trouble tickets from their creation through to the environmental changes they produce. Tivoli software is integrated to capture incoming requests, incidents and problems; route them to the correct decision-makers; and expedite resolution with enterprise-strength server and desktop provisioning tools. They do this while keeping an accurate record of all the configuration items in a federated management database and a real-time picture of the deployed infrastructure – matching hardware and software services with the business needs they fulfill.

By automating change, configuration, provisioning, release and asset management tasks, IBM Service Delivery and Process Automation software and services help reduce cost and eliminate error.

Common process automation platform combines asset and service management in one environment

IBM Service Delivery and Process Automation products leverage a common process automation engine. This engine is unique in its ability to:

  • Provide a self-service portal interface for reservation of computer, storage, and networking resources.
  • Automate provisioning and de-provisioning resources.
  • Increase availability of resources with real-time monitoring and energy management.
  • Combine asset and service management into one environment.
  • Deliver a federated configuration management system.
  • Provide advanced business process management and integration with other Web-based tools.
  • Offer full end-to-end management views of business applications.

With the implementation of IBM service delivery and process automation software solutions, clients can expect to improve the efficiency and effectiveness of IT, enable greater convergence of technology and business processes, and see results in areas like mean time to repair, service quality, and customer satisfaction.

With Tivoli You Can…

  • Optimize efficiency and accuracy of Service Delivery by automating best practices for common tasks, service requests, incident reports, and change and release management
  • Lower cost of management and compliance by discovering and tracking all deployed resources and their configurations, and matching them against established policies.
  • Improve productivity by giving users direct access to a catalog of automated service requests
  • Improve customer satisfaction through higher availability of critical business applications
  • Help control energy consumption in the data center by managing workloads and the provisioning/deprovisioning servers to meet SLAs.
  • Dynamically deploy, manage, secure and retire physical and virtual servers, clients and applications according to users’ needs & organizational guidelines

Anticipated Results

  • Improved resource utilization, resulting in 50% decrease in need for new additional equipment.
  • Labor savings of 10-20% (reduced man-hours due to task automation and software distribution)
  • Increased productivity of supported services by 10-25%
  • Improved success rate for change and release deployments by 10-30%
  • 10-20% reduction in deployed application rollbacks
  • 84% reduction in time taken to inventory physical and software assets
  • IT staff cost savings of $120 per PC/device/year through use of Packaging Tools and Automated Software Distribution
  • Reduced labor cost of 10-40% to maintain multiple configuration databases

Featured products

  • IBM Service Delivery Manager
    Enables businesses to rapidly implement a complete service management solution within a private cloud computing environment. Delivered as a pre-integrated software stack and deployed as a set of virtual images, it allows automated IT service deployment and provides rapid self-service provisioning, resource monitoring, cost management, and high availability of services in a cloud.
  • IBM Tivoli System Automation Application Manager
    Designed for high availability and disaster recovery solutions, providing the ability to automatically initiate, execute, and coordinate the starting, stopping, restarting and failing over of applications running in heterogeneous and virtual IT environments.
  • Tivoli Provisioning Manager
    Provides automated provisioning, improved resource utilization and enhanced IT service delivery.
  • Tivoli Change and Configuration Management Database
    Provides an enterprise-ready platform for storing deep, standardized data on configurations and change histories to help integrate people, processes, information and technology.
  • IBM Tivoli Service Request Manager
    Enables service efficiencies, reduces disruptions, streamlines service desk operations, improves customer satisfaction, and reduces costs by unifying key service support and asset management processes.
  • IBM Tivoli Workload Scheduler
    Enables automated workload management and monitoring across the enterprise, featuring a single console, self-healing capabilities, real-time alerts and reports.
  • Tivoli System Automation
    Protects business and IT services with end-to-end high availability, advanced policy-based automation, and single point control for heterogeneous environments.

Monitoring and performance management

Management and monitoring of application, middleware, server, and network environments for dynamic IT infrastructures.

Efficiently manage the cloud

In a 2012 IBM global study, CEOs ranked technology as the #1 factor impacting their organizations. And 90 percent of those CEOs viewed the cloud as critical to their plans. But as organizational demand for cloud services increases, so do the operational costs and the business risks. If managed incorrectly, this can result in revenue losses, performance degradation and more.

Monitoring and performance management solutions from IBM help you manage the cloud effectively. IBM monitoring and performance management solutions are designed to lower hardware and software costs and minimize performance risks by:

Provisioning and orchestration

Deployment and orchestration of virtual and cloud environments across the service delivery lifecycle.

Accelerate cloud service delivery

Organizations are increasingly turning to the cloud to accelerate the delivery of services and simplify the management of virtualized environments. But the cloud introduces new challenges. Provisioning workloads, controlling image sprawl and managing application deployment become much more complex in virtual and cloud environments.

Cloud provisioning and orchestration solutions from IBM are designed to reduce the IT management complexities introduced by virtual and cloud environments. This accelerates cloud service delivery allowing the enterprise to quickly respond to changing business needs―all while reducing operational costs in a heterogeneous hypervisor and hardware environment. IBM cloud provisioning and orchestration solutions do this by helping you:

Scheduling and systems automation

Cloud management with the added value of choice and automation above and beyond provisioning of virtual machines

IBM SmartCloud Enterprise+ is a fully managed, security-rich and production-ready cloud environment designed to ensure enterprise-class performance and availability. SCE+ offers complete governance, administration and management control along with service-level agreements (SLAs) to align your specific business and usage requirements. Multiple security and isolation options built into the virtual infrastructure and network keep this cloud separate from other cloud environments.

Transform and automate the provisioning of dynamic workloads using cloud services.

Highlights

  • Virtualization built into the Power platform, not bolted on, ensures optimal utilization of resources, efficiency, security and Enterprise Quality of Service for mission critical and compute intensive workloads
  • Scalability and resource elasticity improves workload availability and performance
  • Automated management, provisioning and service delivery decreases deployment times and increases flexibility and agility for faster responsiveness to changing business demands
  • Self-service portal and standardized service catalog provide consistent, reliable and responsive service delivery for improved customer satisfaction
  • Insight into resource utilization provides cost transparency and empowers IT organizations to direct costs back to the business.

As the world changes and IT plays an increasingly critical role, all types of organizations, businesses, and governments are seeking to transform the way they deliver IT services and improve operational efficiency so they can quickly respond to changing business demands. Cloud computing can improve asset utilization, workload optimization and service delivery while reducing complexity and delivering superior IT economics.

Traditional IT Infrastructure presents challenges on many levels. It is typically:

  • Composed of silos that lead to infrastructure disconnected from the priorities of the business
  • Static islands of computing resources which result in inefficiencies and underutilized assets
  • Struggling with rapid data growth, regulatory compliance, information integrity and security concerns—all while trying to control continuously rising IT costs
  • Inflexible in the face of rapid, unprecedented changes in markets, service demands and stakeholder expectations

As a result of the challenges traditional IT infrastructures are facing, organizations are looking towards a Smarter Computing infrastructure to meet the demand for a service delivery model that enables growth and innovation while lowering overall IT costs. A cloud computing environment built with IBM® Power Systems™ helps organizations transform their data centers to meet these challenges. Power Systems cloud solutions:

  • Deliver an integrated virtualization foundation for uncompromised efficiency, maximum utilization, and the ability to scale up or down in line with business needs
  • Address the information challenge by delivering flexible and secure access to critical information where it is needed and meets the highest standards for risk management and compliance mandates
  • Redistribute the IT budget through advanced virtualization, automation and datacenter analytics
  • Utilize flexible delivery models to greatly simplify IT service delivery while providing enterprise QOS capabilities including continuous application availability, optimized performance, more scalability and enterprise-class security

IBM Power Systems cloud solutions can help customers quickly build a powerful, dynamic, and efficient cloud computing environment enabling them to reduce IT costs, improve service delivery, and foster business innovation.

Storage management backup and recovery

Automate backup and restore, centralize storage management and ensure efficient movement and retention of data.

Benefits of VDI ( Virtual Desktop Infrastructure )

desktop

If your organization is interested in optimizing its workstation requisition and maintenance routine, it should consider adopting a virtual desktop infrastructure (VDI). VDI is the practice of hosting workstation operating systems and applications on a server. Users can access the “virtualized” operating systems and applications from thin clients, terminals, workstations, tablets, smart phones, etc., as long as the devices can connect to the host server. Because the operating systems and applications are “virtualized” they can be accessed by devices running on different operating systems such as Android, Linux, Microsoft, etc.

VDI benefits

Depending on the nature of your organization’s IT infrastructure and on the VDI solution that your organization chooses to implement, it can take advantage of a great number of benefits. VDI solutions typically have unique features that will appeal to specific organizational needs, but most VDI solutions will, at the least, provide the following benefits:

  • Quick and easy “workstation” provisioning: Once your organization’s IT team has created a virtualized workstation with an operating system, applications, and security configurations, the virtualized workstation can be used as a template that can be reproduced any time a user needs a new workstation. Copying a template to create workstations as needed can save time and allow users to be productive instead of waiting around for your IT staff to build a computer, install software, and patches.
  • Centralized patch management: Patch management is always an IT nightmare. While some programs like Microsoft Windows and antivirus can be configured on individual workstations to auto-update, other applications like Java have to be manually downloaded and installed. With a VDI solution, because all machines are hosted on one server, it is easy for your organization’s IT staff to ensure that all patches are updated in a timely manner.
  • Standardized security configurations: Because new security threats are discovered every day, your organization’s IT staff likely spends a lot of time updating security patches and maintaining standardized security settings on individual workstations. With a VDI solution, the IT staff can quickly update security patches to all virtualized workstations and ensure that security settings are standardized on all virtualized workstations.
  • Secured data: With a VDI solution, sensitive data is secure because all workstations are virtualized and hosted on servers or hosted data centers. Sensitive data can be created and worked with from numerous computing devices, but it will never reside on the device since the host server houses the virtualized workstation and provides the operating system, applications, data, and processing power. In the event that one of your organization’s laptops, tablets, or smartphones is lost or stolen, your organization won’t have to worry about data exposure because the data is not stored on the devices.
  • Anywhere access of virtualized workstations: If your organization adopts a VDI solution, it will realize increased productivity because users can access their virtualized workstations from home, work, or vacation from many computing devices such as smart phones, tablets, and laptops. Users can safely use personal devices like home computers to access sensitive organizational data because virtual workstations are isolated from the personal devices hard drives.

Learn more

Your organization can realize many benefits from adopting a VDI solution from a reputable vendor such as VMware or Microsoft. To learn more about VDI in general, or to discover how a VDI solution can benefit your organization, please contact the VDI experts at All Covered.

Plan

Cloud readiness assessment, ROI and migration strategies should be clear as you embark on the cloud journey. IBM offers a host of services such as IBM Strategy and Change Services for Cloud Adoption (US) and IBM Strategy and Design Services for Cloud Infrastructure (US) to help clients develop a cloud roadmap.

Assess and build a security roadmap with IBM Professional Security Services – cloud security assessment (US) and cloud security strategy roadmap (US).

Build
Accelerate your application development and test efforts with the IBM SmartCloud Enterprise. Realize cost savings and faster time to value in your private cloud environment.Provide anytime, anywhere access to applications, information and resources with the IBM Smart Desktop Cloud.IBM Cloud Service Provider Platform (CSP2) (US) accelerates and simplifies deployment of a complete cloud services environment.
Deliver
Unleash employee potential with world-class social networking services and on-line collaboration tools including file sharing, web conferencing and instant messaging with IBM LotusLive™ Collaboration Suite.IBM Information Protection Services – managed backup cloud (US) is a cloud-based service that enables security-rich, managed protection of your critical business data.Get fast and flexible SaaS and cloud-based application integration with your existing IT environment with Cast Iron Systems (US).

Amazon EC2 (Elastic Compute Cloud)

Amazon Elastic Compute Cloud (Amazon EC2) is a web service that provides resizable compute capacity in the cloud. It is designed to make web-scale computing easier for developers.

Amazon EC2’s simple web service interface allows you to obtain and configure capacity with minimal friction. It provides you with complete control of your computing resources and lets you run on Amazon’s proven computing environment. Amazon EC2 reduces the time required to obtain and boot new server instances to minutes, allowing you to quickly scale capacity, both up and down, as your computing requirements change. Amazon EC2 changes the economics of computing by allowing you to pay only for capacity that you actually use. Amazon EC2 provides developers the tools to build failure resilient applications and isolate themselves from common failure scenarios.
Get Started with
AWS for Free Sign Up Now »

AWS Free Tier includes 750 hours of Linux or Windows Micro Instances each month for one year. To stay within the Free Tier, use only EC2 Micro instances.
View AWS Free Tier Details »

Instagram can drive data to its computing systems on Amazon EC2 20 times as fast with solid-state drives.

This page contains the following categories of information. Click to jump down:

 

Amazon EC2 Functionality

Amazon EC2 presents a true virtual computing environment, allowing you to use web service interfaces to launch instances with a variety of operating systems, load them with your custom application environment, manage your network’s access permissions, and run your image using as many or few systems as you desire.

To use Amazon EC2, you simply:

Select a pre-configured, templated Amazon Machine Image (AMI) to get up and running immediately. Or create an AMI containing your applications, libraries, data, and associated configuration settings.
Configure security and network access on your Amazon EC2 instance.
Choose which instance type(s) you want, then start, terminate, and monitor as many instances of your AMI as needed, using the web service APIs or the variety of management tools provided.
Determine whether you want to run in multiple locations, utilize static IP endpoints, or attach persistent block storage to your instances.
Pay only for the resources that you actually consume, like instance-hours or data transfer.

Service Highlights

Elastic – Amazon EC2 enables you to increase or decrease capacity within minutes, not hours or days. You can commission one, hundreds or even thousands of server instances simultaneously. Of course, because this is all controlled with web service APIs, your application can automatically scale itself up and down depending on its needs.

Completely Controlled – You have complete control of your instances. You have root access to each one, and you can interact with them as you would any machine. You can stop your instance while retaining the data on your boot partition and then subsequently restart the same instance using web service APIs. Instances can be rebooted remotely using web service APIs. You also have access to console output of your instances.

Flexible – You have the choice of multiple instance types, operating systems, and software packages. Amazon EC2 allows you to select a configuration of memory, CPU, instance storage, and the boot partition size that is optimal for your choice of operating system and application. For example, your choice of operating systems includes numerous Linux distributions, and Microsoft Windows Server.

Designed for use with other Amazon Web Services – Amazon EC2 works in conjunction with Amazon Simple Storage Service (Amazon S3), Amazon Relational Database Service (Amazon RDS), Amazon SimpleDB and Amazon Simple Queue Service (Amazon SQS) to provide a complete solution for computing, query processing and storage across a wide range of applications.

Reliable – Amazon EC2 offers a highly reliable environment where replacement instances can be rapidly and predictably commissioned. The service runs within Amazon’s proven network infrastructure and datacenters. The Amazon EC2 Service Level Agreement commitment is 99.95% availability for each Amazon EC2 Region.
Secure – Amazon EC2 provides numerous mechanisms for securing your compute resources.

Amazon EC2 includes web service interfaces to configure firewall settings that control network access to and between groups of instances.
When launching Amazon EC2 resources within Amazon Virtual Private Cloud (Amazon VPC), you can isolate your compute instances by specifying the IP range you wish to use, and connect to your existing IT infrastructure using industry-standard encrypted IPsec VPN. You can also choose to launch Dedicated Instances into your VPC. Dedicated Instances are Amazon EC2 Instances that run on hardware dedicated to a single customer for additional isolation.
For more information on Amazon EC2 security refer to our Amazon Web Services: Overview of Security Process document.

Inexpensive – Amazon EC2 passes on to you the financial benefits of Amazon’s scale. You pay a very low rate for the compute capacity you actually consume. See Amazon EC2 Instance Purchasing Options for a more detailed description.

On-Demand Instances – On-Demand Instances let you pay for compute capacity by the hour with no long-term commitments. This frees you from the costs and complexities of planning, purchasing, and maintaining hardware and transforms what are commonly large fixed costs into much smaller variable costs. On-Demand Instances also remove the need to buy “safety net” capacity to handle periodic traffic spikes.
Reserved Instances – Reserved Instances give you the option to make a low, one-time payment for each instance you want to reserve and in turn receive a significant discount on the hourly charge for that instance. There are three Reserved Instance types (Light, Medium, and Heavy Utilization Reserved Instances) that enable you to balance the amount you pay upfront with your effective hourly price. The Reserved Instance Marketplace is also available, which provides you with the opportunity to sell Reserved Instances if your needs change (i.e. want to move instances to a new AWS Region, change to a new instance type, or sell capacity for projects that end before your Reserved Instance term expires).
Spot Instances – Spot Instances allow customers to bid on unused Amazon EC2 capacity and run those instances for as long as their bid exceeds the current Spot Price. The Spot Price changes periodically based on supply and demand, and customers whose bids meet or exceed it gain access to the available Spot Instances. If you have flexibility in when your applications can run, Spot Instances can significantly lower your Amazon EC2 costs.

Easy to Start – Quickly get started with Amazon EC2 by visiting AWS Marketplace to choose preconfigured software on Amazon Machine Images (AMIs). You can quickly deploy this software to EC2 via 1-Click launch or with the EC2 console.

Features

Amazon EC2 provides a number of powerful features for building scalable, failure resilient, enterprise class applications, including:

Amazon Elastic Block Store – Amazon Elastic Block Store (EBS) offers persistent storage for Amazon EC2 instances. Amazon EBS volumes are network-attached, and persist independently from the life of an instance. Amazon EBS volumes are highly available, highly reliable volumes that can be leveraged as an Amazon EC2 instance’s boot partition or attached to a running Amazon EC2 instance as a standard block device. When used as a boot partition, Amazon EC2 instances can be stopped and subsequently restarted, enabling you to only pay for the storage resources used while maintaining your instance’s state. Amazon EBS volumes offer greatly improved durability over local Amazon EC2 instance stores, as Amazon EBS volumes are automatically replicated on the backend (in a single Availability Zone). For those wanting even more durability, Amazon EBS provides the ability to create point-in-time consistent snapshots of your volumes that are then stored in Amazon S3, and automatically replicated across multiple Availability Zones. These snapshots can be used as the starting point for new Amazon EBS volumes, and can protect your data for long term durability. You can also easily share these snapshots with co-workers and other AWS developers. Amazon EBS provides two volume types: Standard volumes and Provisioned IOPS volumes. Standard volumes offer cost effective storage that is ideal for applications with moderate or bursty I/O requirements. Provisioned IOPS volumes are designed to deliver predictable, high performance for I/O intensive applications such as databases. See Amazon Elastic Block Store for more details.

EBS-Optimized Instances – For a low, additional, hourly fee, customers can launch selected Amazon EC2 instances types as “EBS-Optimized” instances. EBS-Optimized instances enable Amazon EC2 instances to fully utilize the IOPS provisioned on an EBS volume. EBS-Optimized instances deliver dedicated throughput between Amazon EC2 and Amazon EBS, with options between 500 Mbps and 1000 Mbps depending on the instance type used. When attached to EBS-Optimized instances, Provisioned IOPS volumes are designed to deliver within 10% of their provisioned performance 99.9% of the time. See Amazon EC2 Instance Types to find out more about instance types that can be launched as EBS-Optimized instances.

Multiple Locations – Amazon EC2 provides the ability to place instances in multiple locations. Amazon EC2 locations are composed of Regions and Availability Zones. Availability Zones are distinct locations that are engineered to be insulated from failures in other Availability Zones and provide inexpensive, low latency network connectivity to other Availability Zones in the same Region. By launching instances in separate Availability Zones, you can protect your applications from failure of a single location. Regions consist of one or more Availability Zones, are geographically dispersed, and will be in separate geographic areas or countries. The Amazon EC2 Service Level Agreement commitment is 99.95% availability for each Amazon EC2 Region. Amazon EC2 is currently available in nine regions: US East (Northern Virginia), US West (Oregon), US West (Northern California), EU (Ireland), Asia Pacific (Singapore), Asia Pacific (Tokyo), Asia Pacific (Sydney), South America (Sao Paulo), and AWS GovCloud.

Elastic IP Addresses – Elastic IP addresses are static IP addresses designed for dynamic cloud computing. An Elastic IP address is associated with your account not a particular instance, and you control that address until you choose to explicitly release it. Unlike traditional static IP addresses, however, Elastic IP addresses allow you to mask instance or Availability Zone failures by programmatically remapping your public IP addresses to any instance in your account. Rather than waiting on a data technician to reconfigure or replace your host, or waiting for DNS to propagate to all of your customers, Amazon EC2 enables you to engineer around problems with your instance or software by quickly remapping your Elastic IP address to a replacement instance. In addition, you can optionally configure the reverse DNS record of any of your Elastic IP addresses by filling out this form.

Amazon Virtual Private Cloud – Amazon VPC is a secure and seamless bridge between a company’s existing IT infrastructure and the AWS cloud. Amazon VPC enables enterprises to connect their existing infrastructure to a set of isolated AWS compute resources via a Virtual Private Network (VPN) connection, and to extend their existing management capabilities such as security services, firewalls, and intrusion detection systems to include their AWS resources. See Amazon Virtual Private Cloud for more details.

Amazon CloudWatch – Amazon CloudWatch is a web service that provides monitoring for AWS cloud resources and applications, starting with Amazon EC2. It provides you with visibility into resource utilization, operational performance, and overall demand patterns—including metrics such as CPU utilization, disk reads and writes, and network traffic. You can get statistics, view graphs, and set alarms for your metric data. To use Amazon CloudWatch, simply select the Amazon EC2 instances that you’d like to monitor. You can also supply your own business or application metric data. Amazon CloudWatch will begin aggregating and storing monitoring data that can be accessed using web service APIs or Command Line Tools. See Amazon CloudWatch for more details.

Auto Scaling – Auto Scaling allows you to automatically scale your Amazon EC2 capacity up or down according to conditions you define. With Auto Scaling, you can ensure that the number of Amazon EC2 instances you’re using scales up seamlessly during demand spikes to maintain performance, and scales down automatically during demand lulls to minimize costs. Auto Scaling is particularly well suited for applications that experience hourly, daily, or weekly variability in usage. Auto Scaling is enabled by Amazon CloudWatch and available at no additional charge beyond Amazon CloudWatch fees. See Auto Scaling for more details.

Elastic Load Balancing – Elastic Load Balancing automatically distributes incoming application traffic across multiple Amazon EC2 instances. It enables you to achieve even greater fault tolerance in your applications, seamlessly providing the amount of load balancing capacity needed in response to incoming application traffic. Elastic Load Balancing detects unhealthy instances within a pool and automatically reroutes traffic to healthy instances until the unhealthy instances have been restored. You can enable Elastic Load Balancing within a single Availability Zone or across multiple zones for even more consistent application performance. Amazon CloudWatch can be used to capture a specific Elastic Load Balancer’s operational metrics, such as request count and request latency, at no additional cost beyond Elastic Load Balancing fees. See Elastic Load Balancing for more details.

High Performance Computing (HPC) Clusters – Customers with complex computational workloads such as tightly coupled parallel processes, or with applications sensitive to network performance, can achieve the same high compute and network performance provided by custom-built infrastructure while benefiting from the elasticity, flexibility and cost advantages of Amazon EC2. Cluster Compute, Cluster GPU, and High Memory Cluster instances have been specifically engineered to provide high-performance network capability and can be programmatically launched into clusters – allowing applications to get the low-latency network performance required for tightly coupled, node-to-node communication. Cluster instances also provide significantly increased throughput making them well suited for customer applications that need to perform network-intensive operations. Learn more about how Amazon EC2 and other AWS services can be used for HPC Applications.

High I/O Instances – Customers requiring very high, low latency, random I/O access to their data can benefit from High I/O instances. High I/O instances are an Amazon EC2 instance type that can provide customers with random I/O rates over 100,000 IOPS. High I/O instances are backed by Solid State Disk (SSD) technology and are ideally suited for customers running very high performance NoSQL and relational databases. See Amazon EC2 Instance Types to find out more about High I/O instances.

High Storage Instances – Customers requiring very high storage density per instance, and high sequential I/O for data-intensive applications like data warehousing and Hadoop can benefit from High Storage instances. High Storage instances are an Amazon EC2 instance type that can provide customers with sequential I/O throughout of 2.4 GB/s and provide customers with 48 TB of instance storage across 24 hard disk drives. See Amazon EC2 Instance Types to find out more about High Storage instances.

VM Import/Export – VM Import/Export enables you to easily import virtual machine images from your existing environment to Amazon EC2 instances and export them back at any time. By importing virtual machines as ready to use EC2 instances, you can leverage your existing investments in virtual machines that meet your IT security, configuration management, and compliance requirements. You can export your previously imported EC2 instances back to your on-premise environment at any time. This offering is available at no additional charge beyond standard usage charges for Amazon EC2 and Amazon S3. Learn more about VM Import/Export.

AWS Marketplace – AWS Marketplace is an online store that helps you find, buy and quickly deploy software that runs on AWS. You can use AWS Marketplace’s 1-Click deployment to quickly launch pre-configured software and be charged for what you use, by the hour or month. AWS handles billing and payments, and software charges appear on your AWS bill. Learn more about AWS Marketplace.

Instance Types
Standard Instances
First Generation

First generation (M1) Standard instances provide customers with a balanced set of resources and a low cost platform that is well suited for a wide variety of applications.

M1 Small Instance (Default) 1.7 GiB of memory, 1 EC2 Compute Unit (1 virtual core with 1 EC2 Compute Unit), 160 GB of local instance storage, 32-bit or 64-bit platform
M1 Medium Instance 3.75 GiB of memory, 2 EC2 Compute Units (1 virtual core with 2 EC2 Compute Units each), 410 GB of local instance storage, 32-bit or 64-bit platform
M1 Large Instance 7.5 GiB of memory, 4 EC2 Compute Units (2 virtual cores with 2 EC2 Compute Units each), 850 GB of local instance storage, 64-bit platform
M1 Extra Large Instance 15 GiB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of local instance storage, 64-bit platform

Second Generation

Second generation (M3) Standard instances provide customers with a balanced set of resources and a higher level of processing performance compared to First Generation Standard instances. Instances in this family are ideal for applications that require higher absolute CPU and memory performance. Examples of applications that will benefit from the performance of Second Generation Standard instances include encoding, high traffic content management systems, and memcached.

M3 Extra Large Instance 15 GiB of memory, 13 EC2 Compute Units (4 virtual cores with 3.25 EC2 Compute Units each), EBS storage only, 64-bit platform
M3 Double Extra Large Instance 30 GiB of memory, 26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each), EBS storage only, 64-bit platform

Micro Instances

Micro instances (t1.micro) provide a small amount of consistent CPU resources and allow you to increase CPU capacity in short bursts when additional cycles are available. They are well suited for lower throughput applications and web sites that require additional compute cycles periodically. You can learn more about how you can use Micro instances and appropriate applications in the Amazon EC2 documentation.

Micro Instance 613 MiB of memory, up to 2 ECUs (for short periodic bursts), EBS storage only, 32-bit or 64-bit platform

High-Memory Instances

Instances of this family offer large memory sizes for high throughput applications, including database and memory caching applications.

High-Memory Extra Large Instance 17.1 GiB memory, 6.5 ECU (2 virtual cores with 3.25 EC2 Compute Units each), 420 GB of local instance storage, 64-bit platform
High-Memory Double Extra Large Instance 34.2 GiB of memory, 13 EC2 Compute Units (4 virtual cores with 3.25 EC2 Compute Units each), 850 GB of local instance storage, 64-bit platform
High-Memory Quadruple Extra Large Instance 68.4 GiB of memory, 26 EC2 Compute Units (8 virtual cores with 3.25 EC2 Compute Units each), 1690 GB of local instance storage, 64-bit platform

High-CPU Instances

Instances of this family have proportionally more CPU resources than memory (RAM) and are well suited for compute-intensive applications.

High-CPU Medium Instance 1.7 GiB of memory, 5 EC2 Compute Units (2 virtual cores with 2.5 EC2 Compute Units each), 350 GB of local instance storage, 32-bit or 64-bit platform
High-CPU Extra Large Instance 7 GiB of memory, 20 EC2 Compute Units (8 virtual cores with 2.5 EC2 Compute Units each), 1690 GB of local instance storage, 64-bit platform

Cluster Compute Instances

Instances of this family provide proportionally high CPU resources with increased network performance and are well suited for High Performance Compute (HPC) applications and other demanding network-bound applications. You can learn more about Cluster instance concepts by reading the Amazon EC2 documentation. For more information about specific use cases and cluster management options for HPC, please visit the HPC solutions page.

Cluster Compute Eight Extra Large 60.5 GiB memory, 88 EC2 Compute Units, 3370 GB of local instance storage, 64-bit platform, 10 Gigabit Ethernet

High Memory Cluster Instances

Instances of this family provide proportionally high CPU and memory resources with increased network performance, and are well suited for memory-intensive applications including in-memory analytics, graph analysis, and scientific computing. You can learn more about Cluster instance concepts by reading the Amazon EC2 documentation. For more information about specific use cases and cluster management options for HPC, please visit the HPC solutions page.

High Memory Cluster Eight Extra Large 244 GiB memory, 88 EC2 Compute Units, 240 GB of local instance storage, 64-bit platform, 10 Gigabit Ethernet

Cluster GPU Instances

Instances of this family provide general-purpose graphics processing units (GPUs) with proportionally high CPU and increased network performance for applications benefitting from highly parallelized processing, including HPC, rendering and media processing applications. While Cluster Compute Instances provide the ability to create clusters of instances connected by a low latency, high throughput network, Cluster GPU Instances provide an additional option for applications that can benefit from the efficiency gains of the parallel computing power of GPUs over what can be achieved with traditional processors. Learn more about use of this instance type for HPC applications.

Cluster GPU Quadruple Extra Large 22 GiB memory, 33.5 EC2 Compute Units, 2 x NVIDIA Tesla “Fermi” M2050 GPUs, 1690 GB of local instance storage, 64-bit platform, 10 Gigabit Ethernet

High I/O Instances

Instances of this family provide very high disk I/O performance and are ideally suited for many high performance database workloads. High I/O instances provide SSD-based local instance storage, and also provide high levels of CPU, memory and network performance. For more information about specific use cases and Big Data options on AWS, please visit the Big Data solutions page.

High I/O Quadruple Extra Large 60.5 GiB memory, 35 EC2 Compute Units, 2 * 1024 GB of SSD-based local instance storage, 64-bit platform, 10 Gigabit Ethernet

High Storage Instances

Instances of this family provide proportionally higher storage density per instance, and are ideally suited for applications that benefit from high sequential I/O performance across very large data sets. High Storage instances also provide high levels of CPU, memory and network performance.

High Storage Eight Extra Large 117 GiB memory, 35 EC2 Compute Units, 24 * 2 TB of hard disk drive local instance storage, 64-bit platform, 10 Gigabit Ethernet

EC2 Compute Unit (ECU) – One EC2 Compute Unit (ECU) provides the equivalent CPU capacity of a 1.0-1.2 GHz 2007 Opteron or 2007 Xeon processor.

See Amazon EC2 Pricing for details on costs for each instance type.

See Amazon EC2 Instance Types for a more detailed description of the differences between the available instance types, as well as a complete description of an EC2 Compute Unit.

Operating Systems and Software
Operating Systems

Amazon Machine Images (AMIs) are preconfigured with an ever-growing list of operating systems. We work with our partners and community to provide you with the most choice possible. You are also empowered to use our bundling tools to upload your own operating systems. The operating systems currently available to use with your Amazon EC2 instances include:
Operating Systems
Red Hat Enterprise Linux Windows Server Oracle Enterprise Linux
SUSE Linux Enterprise Amazon Linux AMI Ubuntu
Fedora Gentoo Linux Debian

Software

AWS Marketplace features a wide selection of commercial and free software from well-known vendors, designed to run on your EC2 instances. A sample of products are below. To see the full selection, visit AWS Marketplace.
Databases Application Servers
Microsoft SQL Server Standard Amazon EC2 Running IBM WebSphere Application Server
MongoDB Tomcat Java Web Application Deployment provided by JumpBox
Acunu Storage Platform Standard Edition w/ Apache Cassandra Tomcat on Apache – Java Servlet and JSP Platform by TurnKey Linux
TurnKey PostgreSQL – Object-relational Database System Zend Server (Clustered) w/Gold Support
Couchbase Server – Enterprise Standard

Content Management Business Intelligence
WordPress provided by BitNami SAP BusinessObjects 10 Named User License
Drupal 6 – Content Management Framework provided by TurnKey Linux JasperReports Server Community Edition
MediaWiki Wiki System provided by JumpBox