Skip to content

Google Cloud

"Google Cloud SDK is a set of tools that you can use to manage resources and applications hosted on Google Cloud Platform. These include the gcloud, gsutil, and bq command line tools. The gcloud command-line tool is downloaded along with the Cloud SDK" - https://cloud.google.com/sdk/docs/

gcloud CLI Examples

Working with gcloud configurations

When working with several projects, it's best to use multiple configurations, with one for each project. Run gcloud init to create a new configuration. This interactive prompt will let you re-initialize the default, or create a new named config. Once you have done that, you can run gcloud config configurations list to show what you have configured. To activate a different configuration, run gcloud config configurations activate <new-config-name>.

$ gcloud config configurations list
NAME     IS_ACTIVE  ACCOUNT          PROJECT          COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
default  False      user@example.io  example-prod
staging  True       user@example.io  example-staging  us-west4-a            us-west4
$ gcloud config configurations activate default
Activated [default].

Alternately, you can specify the config you want to use as an environment variable.

$ gcloud config configurations list
NAME     IS_ACTIVE  ACCOUNT                PROJECT                  COMPUTE_DEFAULT_ZONE  COMPUTE_DEFAULT_REGION
dev      False      danielh@example.com    example-dev-369821
staging  True       danielh@example.com    example-staging-550891   us-east4-c            us-east4
$ CLOUDSDK_ACTIVE_CONFIG_NAME=dev gcloud compute instances list
NAME                                                 ZONE           MACHINE_TYPE    PREEMPTIBLE  INTERNAL_IP  EXTERNAL_IP     STATUS
cat-pic-download-1                                   us-central1-a  e2-medium                    10.128.0.2                   TERMINATED
gibson-access                                        us-east4-a     e2-micro                     10.0.0.69    256.24.927.306  RUNNING
gke-dev-cluster-terraform-20090525165-d1058ae2-c0mc  us-east4-a     n1-standard-16               10.0.0.13                    RUNNING
gke-dev-cluster-terraform-20210624231-9d581566-1oo9  us-east4-a     n1-standard-8                10.0.0.56                    RUNNING

Launch cloud shell from CLI

Google Cloud Shell is a free VM in GCP that you can use to access GCP cli tools, or do whatever else you might want to do in a free VM. The home directory is kept for like 6 months or something, so configuring your shell and storing useful scripts isn't a bad idea.

gcloud cloud-shell ssh --authorize-session

Use vscode with cloud-shell

First, make sure your cloud-shell is started by logging into it via gcloud cloud-shell ssh --authorize-session --command=uptime

Generate an ssh_config entry for your cloudshell:

gcloud cloud-shell ssh --authorize-session --dry-run |
while read -r _ _ _ port _ _ _ _ creds _ ; do
  printf "Host cloud-shell\n  User %s\n  Port %s\n  Hostname %s\n" ${creds/@*/} $port ${creds/*@/}
done

This will print something like:

Host cloud-shell
  User dade_murphy
  Port 53000
  Hostname 5.4.3.1

You'll have to put this into your ssh_config somehow. I recommend using Include statement in ~/.ssh/config and having the above command redirect to a file with only that content so it can be updated in-place. Then use vscode like you usually do with remote ssh, or by running

code --folder-uri \
vscode-remote://ssh-remote+cloud-shell/home/dade_murphy/git_workspace/garbage_file_recovery_work

List google cloud projects

gcloud projects list

Switch to a different project

gcloud config set project "$project_name"

Grant a user permission to a docker registry

gsutil iam ch 'user:user@example.com:objectViewer' 'gs://artifacts.example.appspot.com/'

List google compute zones

gcloud compute zones list

List compute nodes

gcloud compute instances list

List all disks

gcloud compute disks list

Generate ssh commands for all nodes

gcloud compute instances list | awk 'NR>1 {printf "gcloud compute ssh --zone %s %s\n", $2, $1;}'

ssh to a compute node

This is useful for getting system level access to an EKS node.

ssh-to-gke-node() {
  [ "$#" -gt 0 ] || { echo "Usage: ssh-to-gke-node <node_name> [command]" ; return 1 ; }
  read -r zone host < <(gcloud compute instances list --filter="$1" | awk 'NR==2 {print $2, $1 ;}') ;
  shift
  gcloud compute ssh --tunnel-through-iap --zone "$zone" "$host" -- "${@}" ;
}

Loop through all gcloud instances and perform a command

gcloud compute instances list |
awk 'NR>1 {printf "gcloud compute ssh --zone %s %s\n", $2, $1;}' |
while read -r ssh_cmd ; do
  $ssh_cmd -- "docker images" </dev/null
done |
sort -u

Create a one-off compute node

gcloud compute instances create $USER-temp-node --zone=us-west4-a --network-interface=no-address

Leave off the --network-interface=no-address if you want a public IP address.

Delete a compute node

Sometimes autoscalers have a hard time scaling down, requiring manual termination of idle nodes. The following commands are equivalent:

gcloud compute instances delete "projects/$project_name/zones/$zone/instances/$node_name"
gcloud compute instances delete --project "$project_name" --zone="$zone" "$node_name"

To connect to VMs that don't have a public ip address, you need to give --tunnel-through-iap on the CLI and also have IAP-secured Tunnel User permission.

Add an EKS context to kubectl

https://cloud.google.com/sdk/gcloud/reference/container/clusters/get-credentials

This adds the cluster to your .kube/config with authentication done via an access token.

CLUSTER_NAME='foo-dev'
PROJECT_NAME='some-project'
REGION='us-west42'
gcloud container clusters get-credentials "${CLUSTER_NAME}" \
  --region "${REGION}" \
  --project "${PROJECT_NAME}"

Add Separate kubectl configs for different EKS clusters

This keeps each config in a different file, which is useful for requiring explicit enabling of a given environment, vs the normal behavior of inheriting the last used context.

# Set up individual kube config files for dev, prod and staging

KUBECONFIG="$HOME/.kube/foo-dev-config"
gcloud container clusters get-credentials dev-cluster --region vn-west4 --project foo-dev
kubectl config rename-context $(kubectl config current-context) foo-dev

KUBECONFIG="$HOME/.kube/foo-prod-config"
gcloud container clusters get-credentials prod-cluster --region vn-west4 --project foo-prod
kubectl config rename-context $(kubectl config current-context) foo-prod

KUBECONFIG="$HOME/.kube/foo-staging-config"
gcloud container clusters get-credentials staging-cluster --region vn-west4 --project foo-staging
kubectl config rename-context $(kubectl config current-context) foo-staging

Then setup aliases like

# ~/.bash_aliases
alias foo-k-dev="export KUBECONFIG=$HOME/.kube/foo-dev-config ; kubectl config set-context foo-dev --namespace=default ;"
alias foo-k-prod="export KUBECONFIG=$HOME/.kube/foo-prod-config ; kubectl config set-context foo-prod --namespace=default ;"
alias foo-k-stage="export KUBECONFIG=$HOME/.kube/foo-staging-config ; kubectl config set-context foo-staging --namespace=default ;"

List images available in Google Container Registry

gcloud container images list

Pull a docker container from Google Container Registry

gcloud docker -- pull gcr.io/project-id/hello-world

Control access to registries

"Container Registry uses a Cloud Storage bucket as the backend for serving container images. You can control who has access to your Container Registry images by adjusting permissions for the Cloud Storage bucket.

Caution: Container Registry only recognizes permissions set on the Cloud Storage bucket. Container Registry will ignore permissions set on individual objects within the Cloud Storage bucket.

You manage access control in Cloud Storage by using the GCP Console or the gsutil command-line tool. Refer to the gsutil acl and gsutil defacl documentation for more information." - https://cloud.google.com/container-registry/docs/access-control

Authenticate a private GCR registry in kubernetes

This is likely not copy/paste material, but the flow is generally correct.

PARTNER=other_company
PROJECT="our_company-$PARTNER"
USER=service-account-user-for-$PARTNER
EMAIL="$USER@$PROJECT.iam.gserviceaccount.com"
gcloud iam service-accounts create $USER
gcloud iam service-accounts keys create \
  --display-name "$USER" \
  --iam-account "$EMAIL" \
  key.json
gcloud projects add-iam-policy-binding "$PROJECT" \
  --member "serviceAccount:$EMAIL" \
  --role "roles/storage.objectAdmin"
done
kubectl create secret "docker-pull-$PROJECT" "$PROJECT" \
  --docker-server "https://gcr.io" \
  --docker-username _json_key \
  --docker-email "$EMAIL" \
  --docker-password "$(cat key.json)"

Then use the value of docker-pull-${PROJECT} as your ImagePullSecret.

Set cache expiration of GCP bucket items to 5 minutes

By default, GCP bucket items have 1 hour of public cache, which means items can be cached outside of the control of the GCP admin. This means that within that cache time window, any requests for the item will have unpredictable results. Set your Cache-Control max-age to something low for files that change, like page content and indexes, but long for files that never change, like images and archives.

gsutil setmeta -h "Cache-Control: public, max-age=300" gs://helm-repo.example.org/index.yaml

More info: https://medium.com/@val_deleplace/public-cache-dont-panic-11038a073a9

View the metadata of a GCP bucket item

gsutil stat gs://helm-repo.example.org/index.yaml

The output will be something like:

gs://helm-repo.example.org/index.yaml:
    Creation time:          Fri, 22 Jul 2020 23:20:11 GMT
    Update time:            Mon, 23 Jul 2020 19:17:53 GMT
    Storage class:          MULTI_REGIONAL
    Cache-Control:          public, max-age=300
    Content-Length:         3195714
    Content-Type:           application/octet-stream
    Hash (crc32c):          vA/Awm==
    Hash (md5):             2AJ32cECSriE0UQStsXxyw==
    ETag:                   COP7ew7D5+CAEoI=
    Generation:             1595460011829230
    Metageneration:         5

Show a bar chart of disk usage of gcp bucket contents

The general idea here is you use gsutil du gs://whatever/somepath, use swap the first and second columns, and pipe that to termgraph. In this example I use awk to do the column swap, filter out all files so we're only checking full directory sizes, and trim the directory name and part of the filename:

$ gsutil du gs://cat-pic-downloader-backups/backups/*full* | awk '/\/$/ {gsub(/.*velero-/, "", $2) ; print $2,$1 ;}' | termgraph

full-back-up-20190128040005/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 307.60M
full-back-up-20190129040006/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 305.13M
full-back-up-20190130040007/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 298.71M
full-back-up-20190201040008/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 298.80M
full-back-up-20190202040009/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 297.77M
full-back-up-20190203040010/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 297.64M
full-back-up-20190204040011/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 297.61M
full-back-up-20190205040012/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 296.78M
full-back-up-20190206040013/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 294.78M
full-back-up-20190207040014/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 292.45M
full-back-up-20190208040015/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 292.05M
full-back-up-20190209040016/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 292.74M
full-back-up-20190210040017/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 291.43M
full-back-up-20190211040018/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 291.73M
full-back-up-20190212040019/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 291.55M
full-back-up-20190213040020/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 289.23M
full-back-up-20190214040022/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 287.54M
full-back-up-20190215040023/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 286.56M
full-back-up-20190216040024/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 286.87M
full-back-up-20190217040025/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 286.58M
full-back-up-20190218040026/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 286.01M
full-back-up-20190219040027/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 286.23M
full-back-up-20190220040028/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 285.57M
full-back-up-20190221040029/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.66M
full-back-up-20190222040030/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.62M
full-back-up-20190223040031/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.29M
full-back-up-20190224040032/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.27M
full-back-up-20190225040033/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.31M
full-back-up-20190226040034/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.19M
full-back-up-20190227040035/: ▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇▇ 283.44M

Set maintenance window on a GKE cluster

This syntax is a bit difficult to work with. It must be given in RFC-5545 (iCal), but GCP only supports a subset of the spec. Most frustratingly, GCP doesn't support timezones. They warn you about this, saying the "timezone will not be stored", so if you want to start something at Wednesday 03:00:00+0600 you have to specify that as Tuesday 21:00:00. 🙄

N.B.: in my testing while writing this, local TZ was used if not TZ was sent, so it looks like their docs may be wrong and maybe the conversion step can be avoided.

Convert local time to rfc-5545 UTC

So if you want to start your maintenance on Thursday at 5am Pacific time and end at Thursday 5pm Pacific time, the first step is to figure out what the rfc-5545 UTC format would be:

$ TZ=Z gdate '+%Y%m%dT%H%M%SZ %A' -d 2021-05-20T05:00:00-0800
20210520T130000Z Thursday
$ TZ=Z gdate '+%Y%m%dT%H%M%SZ %A' -d 2021-05-20T17:00:00-0800
20210521T010000Z Friday

Set and verify the maintenance schedule:

gcloud container clusters update \
  --project="$PROJECT_NAME" \
  --zone="$ZONE" \
  "$CLUSTER_NAME" \
  --maintenance-window-start="20210520T130000Z" \
  --maintenance-window-end="20210521T010000Z" \
  --maintenance-window-recurrence="FREQ=WEEKLY;BYDAY=TH"

You should see Updating <cluster_name>...done. followed by some hyperlinks. Next you should validate that it was set right:

gcloud container clusters describe \
  --format=json \
  --project="$PROJECT_NAME" \
  --zone="$ZONE" \
  "$CLUSTER_NAME" |
jq .maintenancePolicy.window.recurringWindow

You should see a small json blob like:

{
  "recurrence": "FREQ=WEEKLY;BYDAY=TH",
  "window": {
    "endTime": "2021-05-21T02:00:00Z",
    "startTime": "2021-05-20T14:00:00Z"
  }
}

Grab the start time and feed it back into gdate to validate that your desired local time is set:

gdate -d "2021-05-20T14:00:00Z"  # comes out to: Thu May 20 07:00:00 PDT 2021

gcloud web console examples

Logs Explorer examples

Show namespace deletions

protoPayload.methodName="io.k8s.core.v1.namespaces.delete"

Show all images pulled by GKE

This will show pulls of all images except with substrings matching the references on line 3. The OR appears to be case sensitive.

resource.type="k8s_pod"
jsonPayload.reason="Pulling"
-jsonPayload.message : ( "Pulling image \"dade-murphy/leet-image\"" OR "Pulling image \"mr-the-plague/da-vinci\"" )

Show all k8s control plane upgrades

You can search using the same syntax at the CLI, which is great for local saving of search results so you can search them more quickly using grep and easily archive them for postmortem documentation.

gcloud beta logging read 'timestamp>="2021-01-01T00" AND protoPayload.metadata.operationType: UPGRADE_MASTER AND operation.producer: container.googleapis.com' > control-plane-upgrades.yaml

Cloud SQL examples

Docs are here: https://cloud.google.com/sql/docs/postgres/

List instances

gcloud sql instances list

Create a new postgres db instance

In the following example, the tier is custom, with 24 cpu cores and 122880mb of memory. You can see the standard tiers that are available with gcloud sql tiers list. Also, this creates an instance of POSTGRES_14, but other versions are listed here: https://cloud.google.com/sql/docs/postgres/create-instance#gcloud

gcloud sql instances create \
  --project=${GCP_PROJECT} \
  --region=${INSTANCE_REGION} \
  --database-version=POSTGRES_14 \
  --tier=db-custom-24-122880 \
  ${INSTANCE_NAME}

Upgrade postgres to a new version in-place

https://cloud.google.com/sql/docs/postgres/upgrade-major-db-version-inplace

gcloud beta sql instances patch "${INSTANCE_NAME}" --database-version=POSTGRES_14

List backups for an instance

gcloud sql backups list --instance="${INSTANCE_NAME}"

Create a backup

gcloud sql backups create \
  --instance="${INSTANCE_NAME}" \
  --description="Pre pg14 upgrade"

Restore a backup from one instance to another

This has to be done within the same GCP project

gcloud sql backups restore \
  --backup-instance="${OLD_INSTANCE}" \
  --restore-instance="${NEW_INSTANCE}" \
  "${BACKUP_ID}"

Restore a backup from a sql file stored in a bucket

Depending on the contents of your sql backup, you may have to create the target database first. If the db does not exist and the sql file does not working CREATE statements you'll see the ambiguous error system error occurred pre-import and an unhelpful python stack trace.

gcloud sql import sql ${INSTANCE_NAME} \
  --user "${DB_USERNAME}" \
  --database "${TARGET_DB}" \
  gs://some-bucket-name/some_backup.sql \
  --verbosity debug

Add --async if you just want it to go to the background and return you to your shell prompt.