Spot instances can play an important role in reducing your cloud costs. AWS, Microsoft Azure, and Google Cloud all offer spot instances, which are a way to purchase spare computing capacity and receive substantial discounts compared to regular on-demand prices.

Most cloud providers divide their pricing models into three categories:

• On demand instances—let you pay for computing power per hour or per second, without long-term commitments or upfront payments. You can increase or decrease resources available to your applications at will.
• Reserved instances—for workloads that run on the cloud in the long term, you can commit to compute instances for a period of betweenone to three years, with the option of paying some of the amount upfront. Reserved instances grant discounts of between 40-75 percent, depending on the cloud provider and the level of commitment.
• Spot instances—spot instances let you bid on unused capacity in the cloud provider's data center. They provide the highest level of discount, up to 90 percent compared to on demand costs.

Spot instances have an important caveat—reliability is not guaranteed, and the cloud provider can interrupt these instances at short notice to reclaim capacity. However, with careful management, spot instances can be useful for batch processing and high-performance computing (HPC) clusters, web server clusters, and many other workloads.

1 AWS spot instances

AWS Spot Instances are unused Elastic Compute Cloud (EC2) instances, with a variable price that depends on supply and demand in the Amazon spot market, and is lower than the on demand price for the same instance type.

The hourly price of a spot instance is called the spot price. Amazon determines an auction price for each instance type in each availability zone, based on the long-term supply and demand of spot instances. When users request a spot instance, they specify a quantity of instances and a bidding price. Spot instances are provisioned when there is available capacity for the specified instance type and AZ, and the bidding price is equal to or higher than the current auction price.

How AWS spot instances work

The first step in this process is to create a spot instance request. You specify how many instances you want to run, the availability zone (AZ), the type of spot instance (from the same instance types available for on-demand instances), and the maximum bid.

If the maximum bid exceeds the current price for the required type of spot instances, and capacity is available, Amazon EC2 launches the instances. If these requirements are not met, EC2 waits with the request until it can be fulfilled, or you cancel it.

A spot instance request can remain active until any of the following occurs:

• Amazon EC2 fulfills the request
• The request expires
• You cancel the request

A spot instance may be terminated and the spot instance request is closed if the following occurs:

• The spot price for the required instance type exceeds your maximum bid
• Capacity is no longer available

A spot instance may be interrupted, when a Spot price exceeds the maximum bid or capacity is not available. When interrupted, the instance can be either stopped or hibernated. However, when the price changes or capacity is available again, your spot instance is started or resumed.

You can stop and start a spot instance manually. If your spot instance was terminated, you can open the request and EC2 will launch a new instance.

Note that the above applies to one-time spot instance requests. A persistent spot instance request remains active until you cancel it or it expires—regardless of whether the request is fulfilled or not.

AWS spot instance limitations

Amazon spot instances have several important limitations:

• AWS spot instances are only available if EC2 has capacity for the required instance types in the requested availability zone.
• Spot instances can be terminated if the market spot price is higher than your bid. However, to make spot instances more reliable, Amazon has recently provided an option to keep spot instances running at the same price as long as capacity is available, reducing volatility.
• For most workloads, you will need to configure mechanisms such as Amazon Auto Scaling and Amazon Spot Fleet to manage spot instances and balance capacity between spot, on-demand and reserved instances.

2 Azure Spot Instances

Azure spot VMs, like AWS spot instances, provide significant discounts for unused compute capacity on Azure. You can purchase individual spot VMs or VM Scale Sets (VMSS) with several spot instances. Spot virtual machines offer the same features as pay-as-you-go virtual machines. Like on Amazon, Azure can “evict” a spot instance if it needs to reclaim capacity.

Spot VM pricing on Azure is designed to change slowly, to maintain stability, so you can manage your budget. The Azure portal provides access to current spot pricing, so you can determine the region, availability zone or VM size that best suits your budget.

Azure provides several ways to manage spot VMs—through the Azure portal, Azure PowerShell commands, the Azure CLI, or Azure Resource Manager templates.

Azure provides another option for lower-priced VMs, called lower priority VMs—this is reduced pricing for types of VMs that are in low demand on Amazon. Low priority VMs have a consistent low price. Unlike spot VMs, which have a variable price depending on supply and demand.

How do Azure spot instances work?

When creating a Spot VM, you define an eviction strategy and bidding price. The default eviction mode “stopped-deallocated”, meaning the VM can be resumed later when spot instances become available. If you change the eviction policy to “delete”, VMs are permanently deleted when spot instances are evicted.

On Azure, you can select whether to evict spot VMs based on capacity or maximum bidding price:

• Eviction based on capacity—the spot VM will only shut down if Azure runs out of spot capacity for the requested VM size. The spot instance will continue running even if spot price goes above your bidding price.
• Eviction based on capacity and price—the spot VM will be shut down if there is no capacity available and/or if the spot price goes above your bidding price.

Azure Spot VMs Limitations

Keep in mind the following limitations of spot VMs on Azure:

• Azure does not provide an SLA for spot VMs.
• Azure spot instances are shut down after a 30 second warning (compared to 2 minutes on Amazon).
• Available capacity may vary by region, size and time of day, and may be difficult to predict.
• Azure currently does not support running scale sets combining spot VMs and regular on-demand VMs.

3 Google Cloud preemptible VM instances

Google Compute Engine provides preemptible VMs, yet another name for spot instances. Like the other providers, Google offers these instances at radically reduced rates, but if Compute Engine needs them to perform other tasks, it will stop (preempt) the instances.

As with other cloud providers, there is no guarantee as to available capacity. Because Google Cloud is the smallest of the big three cloud providers, statistically, you are less likely to find spare capacity for the instance types you are interested in.

How Preemptible VMs work

You can easily manage a collection of VMs together, by creating preemptible VMs in a managed instance group. When a virtual machine is preempted (terminated), it will be automatically recreated according to instance group specifications. Alternatively, you can automatically recreate preempted VMs using Google Kubernetes Engine (GKE) container clusters.

In Google Compute Engine, spot prices are fixed and do not change after you start an instance. You can view preemptive prices along with on-demand prices in Google's Cloud’s pricing calculator. On Google Cloud, preemptive instances typically offer prices 70-80% lower than on demand rates, and over 50% discount compared to 3-year reserved VMs.

Importantly, Google Cloud does not guarantee to notify users when VMs are preempted. Typically it notifies with 1 minute notice, and if not, you will not be charged for the preemptive VM.

Preemptible VM Instance limitations

Preemptible instances work like regular instances but come with a set of limitations, including:

• Interruptions—Compute Engine can stop a preemptible instance due to system events, at any time. The chances of this occurring are relatively low, and may vary from depending on existing conditions.
• Time limitation—Compute Engine stops preemptible instances after 24 hours of work.
• Availability challenges—a preemptible instance is a finite Compute Engine resource, and might not be available at all times.
• Less flexibility—it is not possible to migrate preemptible instances to a regular VM instance. Additionally, you cannot configure preemptible instances to automatically restart during maintenance events.
• No SLA—preemptible instances are excluded from the Compute Engine Service Level Agreement (SLA) and are not covered by any other SLA.
• No Free Tier—you cannot use Google Cloud Free Tier credits for preemptible instances.

Conclusion

Spot instances are an excellent way to reduce cloud costs, but they can be complex to manage. In this article I showed how spot instances work on the big three cloud providers, and what you should watch out for in terms of limitations and hidden catches.

At the end of the day, if you’re willing to get your hands dirty and set up a system that handles spot instance interruptions, you can get many workloads to run on spot instances on any of these cloud providers. The effort will pay off—every workload you manage to adapt to the spot instance mindset can run on the cloud at up to 1/10 of the regular on-demand price.