Managing Job Priority

Understanding VMs
On the Research Analysis Platform, analyses are executed on Virtual Machines (VMs) using the Amazon Elastic Compute Cloud (Amazon EC2).
When a new job is submitted, the Platform requests a new VM from  EC2. Two types of VMs can be used:
On-demand VMs:  These VMs cost more, but are always available.
Spot VMs: These VMs cost less, but may not be available at the time of the request, so the system may have to wait until they become available. Note also that even after becoming available, the availability of spot VMs can be interrupted, which leads to interruptions in your job execution.
Setting Job Priority
On the Research Analysis Platform, each job has a priority level. The priority level of a job can be set from the following ways.
To set a job’s priority level when launching JupyterLab from the Tools menu:
1.Click on the Tools tab from the global menu on the top of your screen on the Platform and click on JupyterLab.
2.In the New JupyterLab modal that appears, fill out the required fields for your JupyterLab job. For the Priority field,  choose a priority level for your job (high, normal, or low).
3.After filling out the rest of the required fields, click Start Environment.
Setting priority levels from the JupyterLab modal.

 To select a job’s priority level from the Manage tab:
1.From your project’s Manage tab, click Start Analysis.
2.Select the tool you want to run and click the Run Selected button.
3.In the Analysis Settings tab, fill out the required fields. For the Priority field,  choose a priority level for your job (high, normal, or low).
4.Fill out the rest of the steps and then click Start Analysis.
Setting job priority from a project's Start Analysis button.
Priority Levels
High Priority
High priority jobs run with an on-demand VM. The use of high priority is recommended for workloads that need to be executed as soon as possible including any interactive web applications such as JupyterLab or Cloud Workstation. Since the JupyterLab app takes time to initiate, users who wish to execute their analyses without interruptions or restarts may want to use high priority to execute these jobs.
Pricing for these jobs can be found in the "On-demand GBP/hr" column on the Research Analysis Platform rate card.
Normal Priority
For normal priority jobs, the Platform first requests spot VMs from  EC2, then waits up to 15 minutes for these to become available. If spot VMs become available within that time frame, the job will start executing on those spot VMs, but this execution may be interrupted. If spot VMs do not become available within 15 minutes, the Platform requests on-demand VMs from EC2. 
For normal priority jobs, the final outcome (on-demand vs. spot) depends on spot availability. 
Use normal priority for analyses for which you are willing to tolerate spot risks in exchange for a lower price, but don't want to wait more than 15 minutes for spot VMs. 
Pricing depends on which type of VM is used to execute the job. See pricing details in the "On-demand GBP/hr" and "Spot GBP/hr" columns of the Research Analysis Platform rate card.
Low Priority
For a low priority job, the Platform always asks EC2 for spot VMs. If spot VMs are available, the job will start executing right away. Otherwise, the job will remain in a "runnable" state for an  indefinite period while waiting for a spot VM to become available. Once running, jobs can become interrupted if a spot VM becomes unavailable in the middle of execution, usually due to increased cloud demand.
Use low priority for nonurgent workloads that do not have to run right away, or for workloads that do not need to be uninterrupted.
Pricing for these jobs can be found in the "Spot GBP/hr" column named on the Research Analysis Platform rate card.
Spot VM Interruptions
When jobs run on spot VMs, there is a small risk of interruption. An interruption occurs when a spot VM becomes unavailable in the middle of execution, usually due to increased cloud demand. In that case, the job is marked as failed due to an unresponsive VM. What happens next depends on the job's "restart policy." By default, the Platform retries any job that fails due to unresponsive VMs, up to two times. When a job is retried, a new VM is allocated depending on priority. Low priority jobs are retried again on spot, whereas normal priority jobs are retried on spot for 15 minutes and then switched to on-demand.
Note that if your job is interrupted, you will still be billed for usage charges incurred prior to the interruption.

Working on the Research Analysis Platform - Previous

Using Spark to Analyze Tabular Data

Next - Working on the Research Analysis Platform

Managing Usage and Storage Costs

Last modified 7mo ago

Export as PDF

Copy link

Outline

Spot VM Interruptions