Overview of cloud app types

Prior reading: Cloud apps overview

Purpose: This document provides an overview about cloud app options available in Verily Workbench.

Introduction

A cloud app is a configurable pool of cloud computing resources. Cloud apps consist of a virtual machine and a persistent disk, with some useful libraries and tools preinstalled. They’re ideal for interactive analysis and data visualization, and can be finely tuned to suit analysis needs.

Cost is incurred while the app is running, based on your configuration. You can pause the app when it’s not in use, but there’s still a charge for maintaining your disk.

You can create and manage multiple cloud apps per workspace. The apps can have different base images, machine configurations, and numbers of attached GPUs. You might set up a many-core VM to prototype on-node ML training or run complex analysis. For workflows that use Dataproc clusters, a lightweight and lower-cost VM is often sufficient, since the heavy computation happens on the Dataproc cluster itself, and the notebook that launches the cluster performs minimal processing.

Cloud app options

When creating a new app in a Workbench workspace, you have a few application options to choose from:

JupyterLab (Compute Engine instance)

To create a new cloud app with JupyterLab Compute Engine, see Create a new cloud app (JupyterLab Compute Engine instance).

JupyterLab Spark cluster (Dataproc cluster)

See Use Dataproc and Hail to get started with a Dataproc cluster.

R Analysis Environment and Visual Studio Code (Compute Engine instances)

To learn more about using these app options, see Use R Analysis apps and Use Visual Studio Code apps.

Custom (Compute Engine instance)

Create a custom cloud app and share it with other Workbench users in your workspace. See Use custom apps for more details.

Configure & use a cloud app

After an app reaches the Running state, you can select the app's name to launch it in a new browser window.

Depending on the type of cloud app, it will launch an instance of JupyterLab, R Analysis, or Visual Studio Code. From this UI, you can create and run notebooks and use the terminal to work from the command line.

Access the wb command-line tool from your app

The wb command-line utility is automatically installed and configured in all Workbench cloud apps. From the terminal window or from a notebook cell, you can use this utility to get information about your account, workspaces, and workspace resources. Below are a few examples.

Show details of the currently authorized account with wb auth status:

$ wb auth status
User email: xxxx@google.com
Proxy group email: PROXY_xxxxxxxxxxxxxxxxxxxxx@verily-bvdp.com
Service account email for current workspace: pet-xxxxxxxxxxxxxxxxxxxxx@wb-quick-rhubarb-111.iam.gserviceaccount.com
LOGGED IN

wb resource list lists all the resources defined for the current workspace:

$ wb resource list
ID                              RESOURCE TYPE         STEWARDSHIP TYPE      DESCRIPTION
nb-repo                         GIT_REPO              REFERENCED            (unset)
nextflow_tests                  AI_NOTEBOOK           CONTROLLED            (unset)
nf-core-sample-data-repo        GIT_REPO              REFERENCED            (unset)
rnaseq-nf-repo                  GIT_REPO              REFERENCED            Respository containing a Nextflow RNA...
tabular_data_autodelete_aft...  BQ_DATASET            CONTROLLED            BigQuery dataset for temporary storag...
workbench-examples              GIT_REPO              REFERENCED            (unset)
ws_files                        GCS_BUCKET            CONTROLLED            Bucket for reports and provenance rec...
ws_files_autodelete_after_t...  GCS_BUCKET            CONTROLLED            Bucket for temporary storage of file ...

You can see details of a resource given its name:

$ wb resource describe --id ws_files
ID:           ws_files
Description:  Bucket for reports and provenance records.
Type:         GCS_BUCKET
Stewardship:  CONTROLLED
Cloning:      COPY_NOTHING
Access scope: SHARED_ACCESS
Managed by:   USER
Properties:   class Properties {
    []
}
GCS bucket name: ws-files-wb-quick-rhubarb-111
Location: US-CENTRAL1
# Objects: 2

You can use the wb resource resolve command to find the underlying resource that a name points to. You will often see this command used in example notebooks. This makes it straightforward to work with easily-remembered resource names and to access the underlying URI when needed.

$ wb resource resolve --id ws_files
gs://wb-quick-rhubarb-111-ws-files

View and manage your cloud apps via the Cloud console

In addition to viewing the status of your apps in the Workbench UI, you can also view it in the Google Cloud console. This provides another interface for launching, stopping, and starting your apps, as well as making some configuration changes. (However, you must create and delete your apps via Workbench.)

You can visit https://console.cloud.google.com/compute/instances to see your apps. Alternatively, you can follow the project link in a workspace description page to visit the Cloud console for the workspace project, and then navigate to Compute Engine >> VM instances in the Cloud console.