Installing and running the Workbench CLI

Getting started with installing and running the Workbench CLI.
Tags:
Categories:

Prior reading: Overview of the Command Line Interface

Purpose: This document provides detailed instructions for installing the Workbench CLI package and using the Verily Workbench Command Line Interface.

Installation

Important note: If you are using the CLI on a cloud environment in Verily Workbench, you do not need to install anything; all relevant packages are already installed by default in your environment.

Software requirements

For installation

  1. Java 17 or greater
  2. Docker 20.10.2 (Must be running if installing in DOCKER_AVAILABLE mode)
  3. curl, tar, gcloud (For install only)

Note: The CLI doesn’t use gcloud directly either during installation or normal operation. However, docker pull may use gcloud under the hood to pull the default Docker image from the Google Container Repository (GCR); therefore, gcloud is required for installation.

Tool dependencies

When running terra app commands from anywhere other than a cloud environment in Workbench, it’s necessary to install various tools locally. The following instructions are for both MacOS and Linux.

  • gcloud - Make sure you have Python installed, then download the .tar.gz archive file from the installation page. Run gcloud version to verify the installation.
  • gsutil - This is included in the gcloud CLI, or available separately here. Verify its installation with gsutil version (also printed as part of gcloud version)
  • bq - This is included with gcloud. More details are available here. Similarly, verify its installation with bq version.
  • nextflow - Install by downloading a bash script and running it locally. Create a nextflow directory somewhere convenient (e.g. $HOME/nextflow) and switch to it. Then run curl -s https://get.nextflow.io | bash. Finally, move the nextflow executable script to a location on the $PATH by running sudo mv nextflow /usr/local/bin/. Verify the installation with nextflow -version.
  • git - Follow instruction here for installing Git on your platform.

Installing those packages will make the applications available in terra by running, for example, terra gsutil ls. When these tools are run through terra, environment variables are set based on resources in the active workspace, and context such as the active GCP project is set up automatically.

Note: As noted above, if you are using the CLI on a cloud environment in Workbench, you do not need to run any of those commands; those packages are already installed by default.

Docker support

By default, the CLI will be installed without support for Docker, meaning it won’t pull the Docker image.

If you wish to install with Docker support, set the TERRA_CLI_DOCKER_MODE environment variable to one of the following:

  • DOCKER_NOT_AVAILABLE (default) to skip pulling the Docker image.
  • DOCKER_AVAILABLE to pull the image (requires Docker to be installed and running)

First installation

To install the latest version:

curl -L https://storage.googleapis.com/workbench-public/workbench-cli/download-install.sh | bash && export SUPPRESS_GCLOUD_CREDS_WARNING=true

# Optional: Move to somewhere in PATH
sudo mv terra /usr/local/bin

# Set the Workbench CLI to talk to the Verily production environment (as opposed to a test environment):
terra server set --name=verily --quiet

Re-installing or updating to the latest version

To update the Workbench CLI package, simply run the initial installation commands again.

Re-installing the Workbench CLI package will overwrite any existing installation, meaning that all JARs and scripts will be overwritten, but it will not modify your $PATH. If you have added the location of the Workbench CLI to your $PATH, you will therefore need to ensure its location is up to date in your path again after each install.

Using a specific version

To install a specific version, run this command before running the ‘first installation’ commands, first editing it for the version that you wan to use:

export TERRA_CLI_VERSION=0.418.0

Make sure to change the version number provided in the example to the specific version you want to install.

Updating to the latest version in your cloud environment

To update the Workbench CLI package in your cloud environment, you’ll need to open a terminal. Then, run these commands:

curl -L https://storage.googleapis.com/workbench-public/workbench-cli/download-install.sh | bash && export SUPPRESS_GCLOUD_CREDS_WARNING=true

# Optional: Move to somewhere in PATH
mv terra $HOME/.local/bin

# Set the authentication mechanism to 'manual' as your cloud environment does not have an internet browser.
terra config set browser MANUAL

# Re-authenticate, following the prompt.
terra auth login

# Provide a workspace to which to point the workpace context, which has been reset.
terra workspace set --id=<YOUR_WORKSPACE_ID>

Running

Usage

For a list of available commands, please see the List of Commands. You can find context-specific CLI usage examples in the CLI Usage Examples section.

Exit codes

The CLI sets the process exit code as follows.

  • 0 = Successful program execution
  • 1 = User-actionable error (e.g. missing parameter, workspace not defined in the current context)
  • 2 = System or internal error (e.g. error making a request to a Workbench service)
  • 3 = Unexpected error (e.g. null pointer exception)

Application exit codes will be passed through to the caller. For example, if gcloud --malformedOption returns exit code 2, then terra gcloud --malformedOption will also return exit code 2.

Access requirements

Login (authentication)

If you are running the Workbench CLI from anywhere other than a cloud environment in Workbench, you must authenticate yourself.

The following command launches an OAuth flow that creates a new tab in your web browser window where you will complete the login process:

terra auth login

If the machine where you’re running the CLI does not have a web browser available to it, run the following commands instead:

terra config set browser MANUAL
terra auth login

This will trigger the use of a manual login flow that does not involve the web browser.

Note: If you are using the CLI on a cloud environment in Workbench, you do not need to run any of those commands; you are already logged in by default when you open the cloud environment. You can verify this by running terra auth status.

Billing

In order to perform any operations that incur a cost on Workbench (e.g. creating a workspace and resources within it), you need access to a billing account via a spend profile. An administrator (user with admin permissions) can grant you access.

External data

In order to read data from or write data to a private external resource from Workbench, you must grant the appropriate data access permissions to your proxy-group. Run terra auth status to view the email address of your proxy group.

To learn more about sharing resources with other users, see Access control and sharing.

Troubleshooting

Clear context

Clear the context file and all credentials. This will require you to login and select a workspace again.

cd $HOME/.terra
rm context.json
rm StoredCredential

Uninstalling

There is not yet an uninstaller. You can clear the entire context directory, which includes the context file, all credentials, and all JARs. This will then require a re-install (see above).

rm -R $HOME/.terra

Last Modified: 16 November 2023