Migrating a Nimbus Instance to Another Cloud Platform

Owned by Jeremy Phillips
Nov 12, 2024
7 min read

Introduction

If you are looking to migrate your instance out of Nimbus and onto another cloud platform (such as Nectar), you can export the root disk of each instance as a raw file. Normally the preferred process should be to create a clean new instance on the new cloud platform, however we understand that in some cases it is desirable to preserve the complete setup of one or more instances. How you export the root disk depends on whether the instance was created with an ephemeral disk or a dedicated root volume (i.e., whether you selected "Create New Volume" at instance creation time). Both options are detailed below.

The following procedure is for migrating a root disk from Nimbus to another cloud platform. It is not intended for the transferring of data files or storage volumes. It is much more efficient to transfer the data through more traditional means; refer to Transfer Your Data for more information on this.

This documentation is provided as-is, to assist those people who need to export their instance out of Nimbus. The steps below have been tested and confirmed to work. If you encounter quota issues at either the Nimbus or Nectar end, please contact the Pawsey or ARDC helpdesks respectively for assistance. Outside of that, we will be unable to provide further assistance.

Export a Root Disk From Nimbus

Before proceeding, make sure that you have sufficient quota in your Nimbus project to create both a snapshot and an image of your instance (plus a copy of the root volume, depending on the steps required).

  1. Log in to the Nimbus dashboard.

  2. Go to "Compute" -> "Instances", and shut down the instance you are looking to export. Wait until the instance’s status changes to "shutoff".

The next few steps depend on whether your instance has a dedicated root volume or an ephemeral disk.

Dedicated Root Volume

For an instance with a dedicated root volume:

  1. Go to "Volumes" -> "Volumes", and look for the volume attached as either /dev/vda or /dev/sda on the desired instance.

  2. Select "Create Snapshot" from the drop-down menu to the right of the volume. This should take you to the "Volume Snapshots" page, where the snapshot you just created will be listed with status "Available".

  3. Select "Create Volume" to the right of the snapshot. This should take you back to the "Volumes" page, where the volume you just created will be listed with status "Creating". Give it a couple of minutes to create the new volume.

  4. Once the new volume changes to status "Available", select "Upload to Image" from the drop-down menu to the right of the volume. Make sure the disk format is set to "Raw", give it a suitable image name, and click on "upload". The status of the volume will change to "Uploading".
    (If you're wondering why you can't just select "Upload to Image" directly from the original root disk of the instance, you cannot create an image from a volume still attached to an instance, ie one with a status of "in-use". As a result, the extra steps are required.)

  5. Go to "Compute" -> "Images", where you can see your new image being saved. The "Saving" status will take some time, depending on the size of the root disk.

  6. Once the status of the image changes to "Active", click on the image itself to look at the image properties. Under the "Image" block, make a note of the "ID" value (this will be in UUID format), you will need it for later. Then go to step 11 (under the "Export Image" sub-heading below).

Ephemeral Disk

For an instance with an ephemeral root disk:

  1. From the "Instances" page, select "Create Snapshot" from the drop-down menu to the right of the instance. This will then take you to the "Images" page, where you can see the snapshot being saved. The "Saving" status will take some time, depending upon the size of the root disk.

  2. Once the status of the image changes to "Active", click on the image itself to look at the image properties. Under the "Image" block, make a note of the "ID" value (this will be in UUID format), you will need it for later. Then go to step 11 (under the "Export Image" sub-heading below).

Export Image

The next steps cannot be done in the Nimbus dashboard, and must be performed using the OpenStack command-line tool. For further information on how to install the OpenStack client, please refer to Use the OpenStack Client . Note that you will also need to download and run your OpenStack RC file, so that you can access your Nimbus project from the command line. Also note, at this point you can start your Nimbus instance back up.

  1. Before proceeding, make sure you have enough disk space on your machine to save the image as a local file. Also make sure that you can see the image listed that you wish to save by running:

    openstack image list

  2. Export your image to a local file by running the following command:

    openstack image save --file <local_file.raw> <image_id>

    Where <local_file.raw> is the filename you wish to give the file you are saving the image to, and <image_id> is the UUID value you recorded from the previous step. Depending on your network connection, this will take a while, so leave it to run.

  3. Once the command finishes running, check to make sure that the new file size is the same as the size of the root disk of your instance. For most instances, the default size of the root disk will be 40GB (unless you changed the root disk size at instance creation time).

After following these steps, you will now have a locally saved file copy of the entire root disk of your instance. From here, you can import it into another cloud platform. If you specifically wish to use it in Nectar (provided you already have a project there with sufficient resources), continue reading.

 

Import a Root Disk Into Nectar

This section is for those users who already have a Nectar allocation. More information on accessing Nectar and applying for an allocation can be found at https://ardc.edu.au/services/ardc-nectar-research-cloud/how-to-access-the-ardc-nectar-research-cloud/ .

Before proceeding, make sure that your Nectar project has sufficient resources to create one or more instances.

  1. To start off, log in to the Nectar dashboard.

  2. From the left-hand navigation bar, go to "Identity" -> "Application Credentials", and select "Create Application Credential" from the right-hand side of the dashboard. Give it a suitable name related to yourself, and select just the "member" role. You do not need to fill out any other fields, so just click on "Create Application Credential".

  3. You will be presented with a summary including your credential name, and an auto-generated ID and secret. Click on "download clouds.yaml" so save a local copy of clouds.yaml. Make sure that this file is saved to the same directory where you are running the OpenStack command-line tool from, which should also be where the local copy of your Nimbus instance's root disk was saved.

Before proceeding, make sure that your Nimbus credentials from the previous section are not still loaded into memory, as you may otherwise encounter errors. Either start a new shell session for the next steps, or "unset" all of the old Nimbus environment variables by running:

for i in $(env | grep OS_ | cut -d "=" -f 1); do unset $i; done

  1. Instruct the OpenStack client to use the credentials stored in clouds.yaml:

  2. Commence uploading the local copy of your Nimbus instance's root disk to your Nectar project as an image:

    Where <local_file.raw> is the filename of the local copy, and <image_name> is the name you give the image file in your Nectar project. Note that this upload may take a long time, and there is the risk of a network interruption to the upload that will require you to re-run the command and start over (it will not pick up from where it got up to previously). Please be patient.

  3. Once the upload is completed, you can confirm that the image is listed in your Nectar project:

  4. Return to the Nectar dashboard, and go to "Project" -> "Compute" -> "Images".

  5. Select "create volume" from the drop-down menu to the right of the image you just uploaded. Make sure "Type" and "Availability Zone" are set if required (otherwise go with the default values), then click on "Create Volume".

  6. Go to "Project" -> "Volumes" -> "Volumes", where your new volume may still be creating. Once its status is "Available", select "Launch as Instance" from the drop-down menu to the right of the volume. From here, follow the normal steps of creating an instance in Nectar. Do not change the settings in the "Source" section, in order to use the volume you just created.

Once the instance is created, your root disk should now be migrated across from Nimbus to Nectar. If there are any attached data volumes on your Nimbus instance, you should now create and attach a similar sized data volume on your Nectar instance, and transfer the data across.