wb resource create dataproc-cluster

Categories:

Name

wb-resource-create-dataproc-cluster - Add a controlled GCP Dataproc cluster resource with Jupyter.

Synopsis

wb resource create dataproc-cluster [--quiet] [--autoscaling-policy=<autoscalingPolicy>] [--bucket=<configBucket>] [--cluster-id=<clusterId>] [--description=<description>] [--format=<format>] [--idle-delete-ttl=<idleDeleteTtl>] [--image-version=<imageVersion>] [--region=<region>] [--software-framework=<softwareFrameworkType _ >] [--temp-bucket=<tempBucket>] [--workspace=<id>] [--components=<components>[, <components>...]]... [--initialization-actions=<initializationAct_ _ ions>[,<initializationActions>...]]... [-M=<String=String>[, <String=String>...]]... [--properties=<String=String>[, <String=String>...]]... (--id=<id>) [[--manager-machine-type=<machineType>] [--manager-image-uri=<imageUri>] [[--manager-accelerator-type=<type>] [--manager-accelerator-count=<count>]] [[--manager-boot-disk-type=<bootDiskType>] [--manager-boot-disk-size=<bootDiskSizeGb>] [--manager-num-local-ssds=<numLocalSsds>] [--manager-local-ssd-interface=<localSsdInte_ _ rface>]]] [[--num-workers=<numNodes>] [--worker-machine-type=<machineType>] [--worker-image-uri=<imageUri>] [[--worker-accelerator-type=<type>] [--worker-accelerator-count=<count>]] [[--worker-boot-disk-type=<bootDiskType>] [--worker-boot-disk-size=<bootDiskSizeGb>] [--worker-num-local-ssds=<numLocalSsds>] [--worker-local-ssd-interface=<localSsdInter_ _ face>]]] [[--num-secondary-workers=<numNodes>] [--secondary-worker-machine-type=<machineTyp_ _ e>] [--secondary-worker-image-uri=<imageUri>] [--secondary-worker-type=<type>] [[--secondary-worker-accelerator-type=<type>_ ] [--secondary-worker-accelerator-count=<count _ >]] [[--secondary-worker-boot-disk-type=<bootDis_ _ kType>] [--secondary-worker-boot-disk-size=<bootDisk_ _ SizeGb>] [--secondary-worker-num-local-ssds=<numLocal_ _ Ssds>] [--secondary-worker-local-ssd-interface=<loc_ _ alSsdInterface>_]]]

Description

Add a controlled GCP Dataproc cluster resource with Jupyter. For a detailed explanation of parameters, see the documentation for Resource: Cluster.

Options

  • --id=<id>
    ID of the resource, scoped to the workspace. Only use letters, numbers, dashes, and underscores.

  • --description=<description>
    Description of the resource.

  • --workspace=<id>
    Workspace ID to use for this command only.

  • --format=<format>
    Set the format for printing command output. Defaults to the config format property.

    Valid values: JSON, TEXT

    Default: null

  • --quiet
    Suppress interactive prompt.

  • --cluster-id=<clusterId>
    The unique name to give to the Dataproc cluster. It cannot be changed later. The instance name must be 1–52 characters long and contain only lowercase letters, numeric characters, and dashes. The first character must be a lowercase letter and the last character cannot be a dash. If not specified, a value will be auto-generated for you.

  • --region=<region>
    The Google Cloud region of the cluster.

  • --image-version=<imageVersion>
    The Dataproc cluster image version containing versions of its software components. See Dataproc cluster image version lists for the full list of image versions and their bundled software components.

  • --initialization-actions=<initializationActions>[,<initializationActions>...]
    A comma-separated list of initialization scripts to run during cluster creation. The path must be a URL or Cloud Storage path, e.g., gs://path-to-file/file-name.

  • --components=<components>[,<components>...]
    A comma-separated list of components.

  • --properties=<String=String>[,<String=String>...]
    Properties in the format key=value.

  • --software-framework=<softwareFrameworkType>
    Software framework for the cluster.

    Valid values: NONE, HAIL

    Default: NONE

  • --bucket=<configBucket>
    Resource name of the cluster staging bucket. If not specified, a default staging bucket will be created.

  • --temp-bucket=<tempBucket>
    Resource name of the cluster temp bucket. If not specified, a default temp bucket will be created.

  • --autoscaling-policy=<autoscalingPolicy>
    Autoscaling policy ID to attach to the cluster.

  • -M, --metadata=<String=String>[,<String=String>...]
    Custom metadata to apply to this cluster.

    Specify multiple metadata in the format of --metadata="key1=value1" -key2=value2.

    It allows multiple metadata entries split by "," like --metadata=key1=value1,key2=value2.

    By default, set Workbench CLI server (terra-cli-server=[CLI_SERVER_ID]) and the Workbench workspace ID (terra-workspace-id=[WORKSPACE_ID]).

  • --idle-delete-ttl=<idleDeleteTtl>
    Time-to-live after which the resource becomes idle and is deleted.

Manager node configurations

  • --manager-machine-type=<machineType>
    The machine type of the manager node.

    Default: n2-standard-2

  • --manager-image-uri=<imageUri>
    The image URI for the manager node.

  • --manager-accelerator-type=<type>
    The type of accelerator for the manager.

  • --manager-accelerator-count=<count>
    The count of accelerators for the manager.

    Default: 0

  • --manager-boot-disk-type=<bootDiskType>
    The type of boot disk for the manager node.

  • --manager-boot-disk-size=<bootDiskSizeGb>
    The size of the boot disk in GB for the manager node.

    Default: 500

  • --manager-num-local-ssds=<numLocalSsds>
    The number of local SSDs for the manager node.

    Default: 0

  • --manager-local-ssd-interface=<localSsdInterface>
    The interface type of local SSDs for the manager node.

    Default: scsi

Worker node configurations

  • --num-workers=<numNodes>
    The number of worker nodes.

    Default: 2

  • --worker-machine-type=<machineType>
    The machine type of the worker node.

    Default: n2-standard-2

  • --worker-image-uri=<imageUri>
    The image URI for the worker node.

  • --worker-accelerator-type=<type>
    The type of accelerator for the worker.

  • --worker-accelerator-count=<count>
    The count of accelerators for the worker.

    Default: 0

  • --worker-boot-disk-type=<bootDiskType>
    The type of boot disk for the worker node.

  • --worker-boot-disk-size=<bootDiskSizeGb>
    The size of the boot disk in GB for the worker node.

    Default: 500

  • --worker-num-local-ssds=<numLocalSsds>
    The number of local SSDs for the worker node.

    Default: 0

  • --worker-local-ssd-interface=<localSsdInterface>
    The interface type of local SSDs for the worker node.

    Default: scsi

Secondary worker node configurations

  • --num-secondary-workers=<numNodes>
    The number of secondary worker nodes.

    Default: 0

  • --secondary-worker-machine-type=<machineType>
    The machine type of the secondary worker node.

    Default: n2-standard-2

  • --secondary-worker-image-uri=<imageUri>
    The image URI for the secondary worker node.

  • --secondary-worker-type=<type>
    The type of the secondary worker.

    Valid values: preemptible, non-preemptible, spot

    Default: spot

  • --secondary-worker-accelerator-type=<type>
    The type of accelerator for the secondary worker.

  • --secondary-worker-accelerator-count=<count>
    The count of accelerators for the secondary worker.

    Default: 0

  • --secondary-worker-boot-disk-type=<bootDiskType>
    The type of boot disk for the secondary worker node.

  • --secondary-worker-boot-disk-size=<bootDiskSizeGb>
    The size of the boot disk in GB for the secondary worker node.

    Default: 500

  • --secondary-worker-num-local-ssds=<numLocalSsds>
    The number of local SSDs for the secondary worker node.

    Default: 0

  • --secondary-worker-local-ssd-interface=<localSsdInterface>
    The interface type of local SSDs for the secondary worker node.

    Default: scsi

Last Modified: 13 November 2025