Cloud basics

Purpose: This document provides a high-level overview of major cloud concepts to help you get better acquainted with the features of Verily Workbench.

New to cloud?

Your work is waiting for you in the cloud. It may seem like a daunting move, but you don’t need a complete understanding of cloud architecture to use Workbench. With a few basic building blocks in hand, you’ll be able to dive directly into your work backed by all the flexibility the cloud offers.

Workbench uses a streamlined interface to dispel many complexities of cloud computing. With these key concepts, you’ll be able to effectively forge connections to your data and tools, understand project administration, and get started with analysis work in Workbench.

Cloud building blocks

Let’s strip the cloud down to its basic building blocks.

Buckets

Buckets are used to store, organize, and control access to file data in cloud storage services. Files stored in a bucket are referred to as objects. A bucket is like a shared file server that can be accessed from your computer or virtual machines in the cloud. There’s no limit on the number of files a bucket can store. Each bucket has a globally unique name, and a storage region that’s specified on creation.

Containers

A container is a software unit that enables quick and easy use of an application in a variety of computing environments. Within a container is an application and everything needed to run it, including code, runtime, system tools, and system libraries. By using containers, the same application can be used consistently in many environments without installing other packages.

Virtual machines

A virtual machine (VM) is an emulation of a physical computer and can perform the same functions, such as running applications and other software. VM instances are created from pools of resources in cloud data centers. You can specify your VM’s geographic region, compute, and storage resources to meet your job’s requirements.

You can create and destroy virtual machines at will to run applications of interest to you, such as interactive analysis environments or data processing algorithms. Virtual machines underlie Verily Workbench’s cloud environments.

Workbench key concepts

Learn how the building blocks of the cloud apply to Workbench.

Workspaces

Workspaces are where researchers and teams connect, collaborate, and organize all the elements of their research, including data, documentation, code, and analysis. Workspaces are also a way for data and tool providers to deliver data and tools to researchers, along with helpful resources such as documentation, code, and examples.

Resources

Resources comprise a variety of entities whose chief purpose is to facilitate analysis. In many cases, resources are simply multimodal data that can be managed within the workspace, but they aren’t limited to data exclusively. Inside a workspace, the “Resources” tab is where the data resources associated with that project are found.

Cloud environments

A cloud environment is a configurable pool of cloud computing resources. Cloud environments consist of a virtual machine and a persistent disk, with some useful libraries and tools preinstalled. They’re ideal for interactive analysis and data visualization, and can be finely tuned to suit analysis needs.

Cost is incurred while the cloud environment is running, based on your configuration. You can pause the environment when it’s not in use, but there’s still a charge for maintaining your disk.

Cloud storage considerations

Default resource region

Moving data between cloud storage regions can incur expensive egress charges. When you create a Workbench workspace, you’ll be prompted to select a default resource region, where Workbench automatically keeps cloud resources and environments to prevent any unexpected egress fees.

Deletion

In most cases, deleted cloud resources and environments can’t be recovered and are gone forever. If you delete an entity that contains objects, such as a BigQuery dataset or Cloud Storage bucket, all of its contents are deleted. When you delete a workspace, all of its associated resources will be deleted permanently, excluding references. Workbench will always inform you of the outcome and request confirmation before deletion.

References are a special type of resource that point to data outside of Workbench. Deleting a reference has no effect on the source you’re pointing to. To recover a deleted reference, simply create a new reference to the source.

Before deleting, always be certain. Workspaces are collaborative environments. If you have partners or a shared workspace with others, it’s important to understand how deleting resources and environments could affect others’ work.

Project administration

Cloud costs

Cloud costs scale with load and duration. Your cloud platform charges for the amount of data you’re storing, egress for moving or downloading that data, and the virtual machines used during analysis. Workbench provides you with clear information when things cost money.