Troubleshooting

Purpose: This document provides troubleshooting tips for issues you may come across in Verily Workbench.

Workbench CLI (command-line interface)

The Workbench CLI tool rename

The Workbench CLI has been renamed from terra to wb. In the Verily Workbench cloud apps, wb is copied to terra, so that you may use either name for the tool. Backwards compatibility is preserved.

When you do an installation to a local machine, only the wb executable is created; however, if you like, you can copy it to terra and add that to your path as well. This may be helpful if you have existing scripts or notebooks that reference terra.

Check and set the current workspace

Especially if you have created multiple workspaces, it is useful to double-check your current workspace before you run any Workbench CLI commands that affect it (such as creating a new resource).

Run wb status to see information about your current workspace. If one is not set, or if you want to change workspaces, run:

wb workspace set --id=<your-workspace-id>

To see a list of your workspaces and their IDs, run:

wb workspace list

Clear context

If you appear to be having credentials-related issues, it may help to clear the context file and all credentials. This will require you to log in and select a workspace again.

cd $HOME/.workbench
rm context.json
rm StoredCredential

You'll then need to rerun wb auth login, and select a workspace as described above.

Manual uninstall of the Workbench CLI

There is not yet an uninstaller. You can clear the entire context directory, which includes the context file, all credentials, and all JARs. You will then need to re-install.

rm -R $HOME/.workbench

Workspaces

Unable to create workspaces

If you're unable to create new workspaces, it's possible the billing account has been deactivated or you've reached your Google project quota.

You can see your project-per-user quota by navigating to the Create Project page. If you're the billing account owner or admin, you can see the projects-per-billing-account quota in Google Cloud console.

You can request to increase the project quota associated with your Google Cloud billing account by contacting Google via this form. The requester must be the billing account owner or admin.

Complete the form, following the guidelines below:

• Email addresses: Use the Google Cloud Billing account owner's email address
• How many projects are being requested?: Enter 5 (entering a high amount may trigger an abuse investigation)
• What kind of services will these projects use?: Select Paid services
• Billing account IDs: Enter the 18-digit Google Cloud Billing account ID found on the Cloud billing accounts page.
• Any other things we need to be aware of to help us understand the request?: Enter a brief explanation that you're using the Workbench platform, which creates Google projects via an API for data processing. Mention that you'll need enough quota to transfer Workbench projects to your Billing account.

Google should send you a confirmation email within the next two days, and they may ask for additional information before increasing the quota.

Contact Workbench support for further assistance.

Update workspace billing account

All Workbench workspaces are linked to a pod, which is associated with a Cloud billing account. If the billing account is inactive or unlinked from the pod for any reason, you'll see an alert at the top of your workspace's page.

If you're the pod owner, the alert will direct you to create a Cloud billing account and link it to your billing pod. Select the links in the error message to complete these steps.

If you're a pod user, you'll see an alert directing you to contact your billing administrator to create a Cloud billing account.

The Resources, Apps, and Workflows tabs for your workspace will be disabled until the billing information is updated.

Cloud apps

App won't start

A cloud app may not start if the VM's data disk is full.

To prevent this, data files should be stored in cloud storage buckets, which will be automatically mounted to your apps. For more information, see Access workspace files and folders from your cloud app.

Workspace buckets mounted in cloud apps

See the Troubleshooting section in Access workspace files and folders from your cloud app for issues with bucket automounting.

Dataproc (managed Spark) clusters

If you have any secondary workers running as part of a Dataproc cluster, the cluster cannot be stopped (paused). You will first need to edit your cluster configuration, as described in the next section, to set the secondary workers to 0 nodes. Note that if the cluster is "UPDATING" (e.g., if autoscaling is engaged to turn nodes up or down), you will need to wait until the update is finished to make edits.