How to reference and work with Requester Pays buckets
Categories:
Prior reading: Cloud cost management
Purpose: This document provides information about working with Requester Pays buckets in Verily Workbench.
Introduction
Access to data in Cloud Storage may incur cloud charges. Google Cloud does not charge users for accessing data in Cloud Storage as long as that data stays within its storage region. For example, moving data from a bucket in us-central1
to a cloud environment in us-central1
is free. Charges are incurred for moving data out of the storage region.
When a researcher copies data out of a Cloud Storage region, including downloading the data, the bucket owner is charged for all data transfer costs associated with the bucket. However, buckets can be configured to be a Requester Pays bucket. Requester Pays allows you to control Cloud Storage billing by shifting access costs to the users who request your data. For more on cloud charges and management see Cloud cost management.
Adding and browsing a reference to a Requester Pays bucket
Adding a data reference via the Web UI or CLI is described in How to work with data. When a Requester Pays bucket is added through the UI, an info message about costs related to the Requester Pays bucket is shown.
![Screenshot of the Adding Cloud Storage bucket dialog showing a blue banner indicating you're adding a Requester Pays bucket.](/images/resources/add_rp_bucket.png)
As you browse a Requester Pays bucket, costs for reading and downloading data are billed to the workspace’s Pod. Learn more in Cloud Storage Documentation.
Mounting a Requester Pays bucket
In a cloud environment, buckets mount on each startup. Mounting buckets set to Requester Pays may incur cost. See more about mounted workspace buckets in Accessing workspace files and folders from your Cloud Environment.
Accessing a Requester Pays bucket using gsutil
or gcloud storage
To use gsutil
to access a Requester Pays bucket, use the -u
flag and supply a user project to be billed.
gsutil -u PROJECT_ID ls gs://BUCKET_NAME
To use gcloud storage
to access a Requester Pays bucket, use the --billing-project
flag:
gcloud storage ls gs://BUCKET_NAME --billing-project PROJECT_ID
Last Modified: 22 August 2024