Using git repos in Verily Workbench notebooks with the Workbench CLI

Introduction

Source control is the practice of tracking and managing changes to code. There are many reasons to use source control to keep track of changes to notebooks, Python and R scripts, or workflows on Verily Workbench.

• Collaborating: Source control lets many people work on the same code base at the same time. Any conflicts are resolved in a controlled and trackable way, and it’s easy to revert to previous versions.
• Sharing: Source control allows others to view and use your work, and see the history of changes to a code base.
• Managing different versions of work: When experimenting with or prototyping different approaches to an analysis, source control lets you easily review differences between versions.
• Creating a history of changes that you (and your collaborators) can refer to later.
• As a backup mechanism: Easily revert to previous versions, when needed.

When multiple researchers collaborate in the same Workbench workspace, use of source control becomes particularly important.

Workbench supports a referenced resource for git repositories. Users can add repositories relevant to a given workspace. Then, when someone creates a cloud environment in that workspace, the referenced repositories are automatically cloned to the cloud environment. Researchers can then use the Git extensions on the cloud environment to commit and push their changes. On their local machine, researchers can use terra git clone --all to clone all the git repo references to their preferred directory.

Setup

Install the Workbench CLI and create a workspace

First, install the Workbench CLI and authenticate with your account as necessary.

Then, check terra status. If no workspace is set, or you are not logged in, first log in and set the workspace in which you want to work. Otherwise, create a new workspace.

terra status

Then log in if necessary, and list your workspaces:

terra auth login  # if need be
terra workspace list

To create a new workspace:

terra workspace create --id=<workspace-name>

To set the Workbench CLI to use an existing workspace:

terra workspace set –-id=<workspace-id>

Set up your Workbench SSHkey (as necessary)

If you want to access private git repos, then you will first need to associate your Workbench SSH public key with your GitHub account using this process.

To create a GCS bucket resource via the web UI, see the instructions here or here.

Add a git repo reference

Create a couple of git repo references:

terra resource add-ref git-repo --name="wsm" --repo-url=https://github.com/DataBiosphere/terra-workspace-manager.git
terra resource add-ref git-repo --name="example_notebook" --repo-url=https://github.com/DataBiosphere/terra-example-notebooks.git

Create a gcp notebook and clone the git-repo referenced resources to which the user has access

1. create a cloud environment

terra resource create gcp-notebook --name="my-cloud-env"

2. get the cloud environment proxy url from the output of running the command below.

terra resource describe –-name="my-cloud-env"

3. See the git repo cloned in the cloud environment:

Post creation

If more git-repo resources are subsequently added to the workspace and you want to clone them into a cloud environment that you have created (e.g., “my-cloud-env”), you can run:

terra git clone --all