Access GCP platform APIs from your app

Prior reading: Cloud apps overview

Purpose: This document provides details for accessing APIs for GCP Dataflow and Vertex AI in Verily Workbench.

Access GCP Dataflow from your app

Workspaces are configured with permissions to call the Dataflow APIs in GCP. This example demonstrates calling a Dataflow sample from a notebook cell.

First, create a storage bucket in Workbench that will be used for temporary storage and output from the Dataflow job. Once created, record the gsutil URI of the storage bucket.

You can then run commands in your notebook cell to collect information needed to run a Dataflow job:

# prints the service account used by the notebook
!gcloud auth list

# prints information about the workspace, including the default region and Google project name
!wb workspace describe

You can then run a Dataflow pipeline from the notebook. For example, you can run the QuickStart WordCount pipeline using the following commands:

# install apache beam
!pip3 install 'apache-beam[gcp]' --user --quiet

# run the example
!python3 -m apache_beam.examples.wordcount \
--region [YOUR_DEFAULT_REGION] \
--output [YOUR_GSUTIL_URI]/output \
--temp_location [YOUR_GSUTIL_URI]/temp \
--service_account_email "[YOUR_SA_EMAIL]" \
--project [YOUR_GCP_PROJECT_NAME] \
--input gs://dataflow-samples/shakespeare/kinglear.txt \
--runner DataflowRunner \
--network=network \
--subnetwork=regions/us-central1/subnetworks/subnetwork \
--no_use_public_ips

Access Vertex AI from your app

Users may want to call GCP Vertex AI APIs from within their Workspace projects to leverage Google Cloud’s machine learning capabilities. All workspace projects are configured with permissions to call these APIs. This example demonstrates calling the Vertex AI embeddings sample from a notebook cell.

First, load the sample within your app by selecting the menu item File > Open from URL… and provide the sample URL:

https://raw.githubusercontent.com/GoogleCloudPlatform/vertex-ai-samples/refs/heads/main/notebooks/official/generative_ai/text_embedding_new_api.ipynb

Once the sample is loaded, run the cell under Installation to install google-cloud-aiplatform.

Skip the Colab cells since the app is not running in the Colab environment.

Note down your workspace’s GCP project ID. This can be found in the Overview section of your workspace within Workbench or by running the following command within a terminal in your app:

wb workspace describe

Replace your project ID into the Initialize Vertex AI SDK cell and run the cell.

PROJECT_ID = "[your-project-id]"  # @param {type:"string"}

Since the app is running in a GCP virtual machine, you can skip the Authenticate your Google Cloud account section.

Run the last two cells to complete running the sample.