​
Intro

Prices for VM instances with the same GPU can vary widely between cloud providers. Some of the variance is due to other hardware specs such as memory, storage, vCPU count, network speed, region, and more. However, a lot of the variance is also due to non-technical factors such as cloud provider brand, marketing, and pricing strategy.

Often times, the GPU configuration in an instance is far more important than any other details. In such cases, we provide an API that can help search for the most affordable GPU instance given the GPU type.

​
Using the Instance Types API

We can find the most affordable GPU instances by applying filtering and sorting query params to the Instance Types API. Without any query parameters, the API returns all of the instance types supported by Shadeform in a random order. By using the following query parameters, we can filter down the results and sort them as needed.

Query ParameterDescriptionExample Value
cloudFilter by cloud namemassedcompute
regionFilter by regionus-central-3
num_gpusFilter by number of GPUs in the instance1
gpu_typeFilter by GPU typeA100_80G
availableFilter by availabilitytrue
sortSort by a parameterprice

​
Find the Most Affordable A100_80G Instance Available

This is an example API call to find the most affordable A100_80G instance available right now.

From array of results returned, we can see that the A100_80G instance from RunPod is the most affordable instance available right now at $2.03/hr.

Response
{
  "instance_types": [
    {
      "cloud": "runpod",
      "shade_instance_type": "A100_80G",
      "cloud_instance_type": "NVIDIA A100 80GB PCIe",
      "configuration": {
        "memory_in_gb": 125,
        "storage_in_gb": 250,
        "vcpus": 8,
        "num_gpus": 1,
        "gpu_type": "A100_80G",
        "interconnect": "pcie",
        "os_options": [
          "ubuntu22.04_cuda12.2_shade_os"
        ],
        "vram_per_gpu_in_gb": 80
      },
      "memory_in_gb": 125,
      "storage_in_gb": 250,
      "vcpus": 8,
      "num_gpus": 1,
      "gpu_type": "A100_80G",
      "interconnect": "pcie",
      "hourly_price": 203,
      "availability": [
        {
          "region": "any",
          "available": true
        }
      ],
      "boot_time": {
        "min_boot_in_sec": 180,
        "max_boot_in_sec": 300
      }
    },
    ...
  ]
}

​
Find the Most Affordable 8xH100 Instance Available

This is an example API call to find the most affordable H100 instance available right now.

From array of results returned, we can see that the 8xH100 instance from Oblivus is the most affordable instance available right now at $32.80/hr.

Response
{
  "instance_types": [
    {
      "cloud": "oblivus",
      "shade_instance_type": "H100x8",
      "cloud_instance_type": "H100_PCIE_80GB_x8",
      "configuration": {
        "memory_in_gb": 1440,
        "storage_in_gb": 6600,
        "vcpus": 252,
        "num_gpus": 8,
        "gpu_type": "H100",
        "interconnect": "pcie",
        "os_options": [
          "ubuntu22.04_cuda12.2_shade_os",
          "ubuntu20.04_cuda12.2_shade_os"
        ],
        "vram_per_gpu_in_gb": 80
      },
      "memory_in_gb": 1440,
      "storage_in_gb": 6600,
      "vcpus": 252,
      "num_gpus": 8,
      "gpu_type": "H100",
      "interconnect": "pcie",
      "hourly_price": 3280,
      "availability": [
        {
          "region": "MON1",
          "available": true,
          "display_name": "CA, Montreal"
        }
      ],
      "boot_time": {
        "min_boot_in_sec": 180,
        "max_boot_in_sec": 300
      }
    },
  ]
  ...
}

​
Find the Most Affordable 8xH100 Instance Regardless of Availability

To find the most affordable 8xH100 instance regardless of availability, we can use the previous API call and remove the available=true parameter.

After we remove the available=true parameter, we now see the top result as Massed Compute’s 8xH100 instance at $23.92/hr. However, it is not available at the time of this API call.

Response
{
  "instance_types": [
    {
      "cloud": "massedcompute",
      "shade_instance_type": "H100x8",
      "cloud_instance_type": "gpu_8x_h100",
      "configuration": {
        "memory_in_gb": 512,
        "storage_in_gb": 5000,
        "vcpus": 128,
        "num_gpus": 8,
        "gpu_type": "H100",
        "interconnect": "pcie",
        "os_options": [
          "ubuntu22.04_cuda12.2_shade_os"
        ],
        "vram_per_gpu_in_gb": 80
      },
      "memory_in_gb": 512,
      "storage_in_gb": 5000,
      "vcpus": 128,
      "num_gpus": 8,
      "gpu_type": "H100",
      "interconnect": "pcie",
      "hourly_price": 2392,
      "availability": [
        {
          "region": "us-central-2",
          "available": false,
          "display_name": "US, Kansas City, KS"
        }
      ],
      "boot_time": {
        "min_boot_in_sec": 180,
        "max_boot_in_sec": 300
      }
    }
  ]
  ...
}

​
Find the most Affordable A6000 Instance Available

This is an example API call to find the most affordable A6000 instance available right now.

From array of results returned, we can see that the A6000 instance from RunPod is the most affordable instance available right now at 0.57/hr.

Response
{
  "instance_types": [
    {
      "cloud": "massedcompute",
      "shade_instance_type": "A6000",
      "cloud_instance_type": "gpu_1x_a6000",
      "configuration": {
        "memory_in_gb": 48,
        "storage_in_gb": 256,
        "vcpus": 6,
        "num_gpus": 1,
        "gpu_type": "A6000",
        "interconnect": "pcie",
        "os_options": [
          "ubuntu22.04_cuda12.2_shade_os"
        ],
        "vram_per_gpu_in_gb": 48
      },
      "memory_in_gb": 48,
      "storage_in_gb": 256,
      "vcpus": 6,
      "num_gpus": 1,
      "gpu_type": "A6000",
      "interconnect": "pcie",
      "hourly_price": 57,
      "availability": [
        {
          "region": "us-central-2",
          "available": true,
          "display_name": "US, Kansas City, KS"
        }
      ],
      "boot_time": {
        "min_boot_in_sec": 180,
        "max_boot_in_sec": 300
      }
    },
  ...
  ]
}

​
Launching the Instance

After you have found the most affordable instance, you can launch the instance by using the Create Instance API. In the example below, we will launch the most affordable A6000 GPU based on the data retrieved from the API call in the previous section. We will take the following properties from the previous response:

cloud shade_instance_type region

Using the values of these properties, we can create the following Create request:

Bring Your Own SSH KeysServe Model Using vLLM