Connect Your Google Cloud Organization

Prerequisites

Registered Spot account
You’ll need these permissions for the Google Cloud account:
- roles/iam.organizationRoleAdmin to create and manage roles at the organization level.
- roles/iam.serviceAccountCreator to create service accounts.
- roles/iam.serviceAccountAdmin to manage service accounts.
- roles/resourcemanager.projectIamAdmin to manage IAM policies for projects.

Connect a Google Cloud billing account

In the Spot console, go to Eco and select Google Cloud > Log in to your Google Cloud account. After you sign in to Google Cloud, keep that browser tab open, then in the Spot console, click Next.
If you don't already have projects and datasets for your Cloud billing data, pricing, and recommendations, you can set them up. Your recommendations and billing export must be configured in the same Google Cloud project.
- Export Cloud billing and pricing data to BigQuery
- Export recommendations to BigQuery

Create a service account and grant permissions to your Google Cloud environment:

Open Cloud Shell.
Set the project property in the core section to your project ID. This makes the project the default project, where the service account is bound. Make sure your data exports to this account.

For example, gcloud config set project YOUR_MAIN_PROJECT_ID.
Grant permissions to your account by running these commands in Cloud Shell:

gcloud organizations add-iam-policy-binding YOUR_ORGANIZATION_ID --member="user:YOUR_EMAIL" --role="roles/iam.organizationRoleAdmin"

gcloud projects add-iam-policy-binding YOUR_MAIN_PROJECT_ID --member="user:YOUR_EMAIL" --role="roles/iam.serviceAccountCreator"

gcloud projects add-iam-policy-binding YOUR_MAIN_PROJECT_ID --member="user:YOUR_EMAIL" --role="roles/iam.serviceAccountAdmin"

gcloud projects add-iam-policy-binding YOUR_MAIN_PROJECT_ID --member="user:YOUR_EMAIL" --role="roles/resourcemanager.projectIamAdmin"

Create a file in Cloud Shell using a text editor, such as nano or vi. For example: nano setup_gcloud_iam_roles_and_service_accounts.sh.

Copy this script and paste it into the file you just created:

View the script

set -uo pipefail

FAILED=0
log_error() {
  echo "ERROR: $1" >&2
}

log_success() {
  echo "SUCCESS: $1"
}

validate_command() {
  local err_msg="$1"
  local success_msg="$2"
  local cmd="$3"
  shift 3

  echo "Running: $cmd $*"

  if ! "$cmd" "$@"; then
    log_error "$err_msg"
    FAILED=$((FAILED+1))
  else
    log_success "$success_msg"
  fi
}

echo "Fetching org ID..."
ANALYSIS_ORG_ID="$(gcloud projects get-ancestors $(gcloud config get-value project --quiet) | awk '/TYPE: organization/{print id} {id=$2}')"
SERVICE_ACCOUNT_ORG_ID="$ANALYSIS_ORG_ID"

CURRENT_PROJECT_ID=$(gcloud config get-value project --quiet)
ANALYSIS_PROJECTS=("$CURRENT_PROJECT_ID")
SERVICE_ACCOUNT_PROJECT_LIST=("$CURRENT_PROJECT_ID")

ANALYSIS_ORG_ROLES=("roles/billing.viewer" "roles/browser")
ANALYSIS_EMAILS=("greg.kuderna@flexera.com")
ANALYSIS_PROJECT_ROLES=("roles/bigquery.dataViewer" "roles/bigquery.jobUser")
ANALYSIS_CUSTOM_ROLE_NAME="spot_read_only_custom_role"
ANALYSIS_CUSTOM_ROLE_TITLE="Spot Read-Only Custom Role"
ANALYSIS_CUSTOM_ROLE_DESCRIPTION="Spot Read-Only Permissions needed for programmatic visibility into commitment and cost data"
ANALYSIS_CUSTOM_ROLE_PERMISSIONS="bigquery.capacityCommitments.get,bigquery.capacityCommitments.list,bigquery.jobs.listAll,cloudasset.assets.exportComputeCommitments,cloudasset.assets.listComputeCommitments,compute.commitments.get,compute.commitments.list,compute.instances.get,compute.instances.list,recommender.bigqueryCapacityCommitmentsInsights.get,recommender.bigqueryCapacityCommitmentsInsights.list,recommender.bigqueryCapacityCommitmentsRecommendations.get,recommender.bigqueryCapacityCommitmentsRecommendations.list,recommender.commitmentUtilizationInsights.get,recommender.commitmentUtilizationInsights.list,recommender.spendBasedCommitmentInsights.get,recommender.spendBasedCommitmentInsights.list,recommender.spendBasedCommitmentRecommendations.get,recommender.spendBasedCommitmentRecommendations.list,recommender.spendBasedCommitmentRecommenderConfig.get,recommender.usageCommitmentRecommendations.get,recommender.usageCommitmentRecommendations.list"
SERVICE_ACCOUNT_PROJECT_ROLES=("roles/bigquery.dataViewer" "roles/bigquery.jobUser" "roles/bigquery.readSessionUser")
SERVICE_ACCOUNT_NAME="spot-programmatic-access-sa" #between 6 and 30 characters
SERVICE_ACCOUNT_DESCRIPTION="Spot Service Account created for Programmatic Access to Resources"
SERVICE_ACCOUNT_DISPLAY_NAME="spot-programmatic-access-service-account"
SERVICE_ACCOUNT_CUSTOM_ROLE_NAME="spot_programmatic_access_role"
SERVICE_ACCOUNT_CUSTOM_ROLE_TITLE="Spot Programmatic Access Role"
SERVICE_ACCOUNT_CUSTOM_ROLE_DESCRIPTION="Spot Custom Role for Programmatic Access"
SERVICE_ACCOUNT_CUSTOM_ROLE_PERMISSIONS="monitoring.timeSeries.list,cloudquotas.quotas.get,cloudquotas.quotas.update,serviceusage.quotas.get,serviceusage.quotas.update,serviceusage.services.get,serviceusage.services.list,bigquery.jobs.create,bigquery.readsessions.create"

echo "Validating service account IAM roles..."

echo "Checking if service account $SERVICE_ACCOUNT_NAME exists..."
if gcloud iam service-accounts describe "$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com" >/dev/null 2>&1; then
  echo "Validating service account IAM roles..."
  for ROLE in "roles/bigquery.dataViewer" "roles/bigquery.jobUser" "roles/bigquery.readSessionUser"; do
    echo "Checking if $SERVICE_ACCOUNT_NAME has $ROLE..."
    if [[ -z $(gcloud projects get-iam-policy "$CURRENT_PROJECT_ID" \
        --flatten="bindings[].members" \
        --filter="bindings.members:serviceAccount:$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com AND bindings.role:$ROLE" \
        --format="value(bindings.role)") ]]; then
      log_error "Service account does not have $ROLE"
    else
      log_success "Service account has $ROLE"
    fi
  done
else
  log_success "Service account $SERVICE_ACCOUNT_NAME does not exist yet - skipping IAM role check"
fi

for ROLE in "${ANALYSIS_ORG_ROLES[@]}"; do
  for EMAIL in "${ANALYSIS_EMAILS[@]}"; do
    echo "Adding member: user:$EMAIL to role $ROLE ..."
    # Note: This requires permissions on the billing account, not the project.
    # It will fail if the user running the script is not a Billing Account Administrator.
    BILLING_ACCOUNT_ID=$(gcloud billing projects describe "$CURRENT_PROJECT_ID" --format="value(billingAccountName)" | cut -d'/' -f2)
    if [[ -n "$BILLING_ACCOUNT_ID" ]]; then
      validate_command \
        "Failed to add user:$EMAIL to billing account role $ROLE" \
        "Added user:$EMAIL to billing account role $ROLE" \
        gcloud organizations add-iam-policy-binding $ANALYSIS_ORG_ID \
            --role=$ROLE \
            --member="user:$EMAIL"
    else
      log_error "Could not determine Billing Account ID for project $CURRENT_PROJECT_ID."
      FAILED=$((FAILED+1))
    fi
  done
done

for PROJECT in "${ANALYSIS_PROJECTS[@]}"; do
  for ROLE in "${ANALYSIS_PROJECT_ROLES[@]}"; do
    for EMAIL in "${ANALYSIS_EMAILS[@]}"; do
      echo "Adding user:$EMAIL to project role $ROLE in project $PROJECT..."
      validate_command \
        "Failed to add user:$EMAIL to project role $ROLE" \
        "Added user:$EMAIL to project role $ROLE in $PROJECT" \
        gcloud projects add-iam-policy-binding $PROJECT \
        --role=$ROLE \
        --member="user:$EMAIL"
    done
  done
done

# Check for the role at the PROJECT level, and create or update it.
if gcloud iam roles describe "$ANALYSIS_CUSTOM_ROLE_NAME" --project="$CURRENT_PROJECT_ID" >/dev/null 2>&1; then
    # Role exists, now check if it's deleted
    if gcloud iam roles describe "$ANALYSIS_CUSTOM_ROLE_NAME" --project="$CURRENT_PROJECT_ID" --format="value(deleted)" | grep -q "True"; then
        echo "Custom role $ANALYSIS_CUSTOM_ROLE_NAME is in a deleted state. Undeleting..."
        validate_command \
            "Failed to undelete custom role $ANALYSIS_CUSTOM_ROLE_NAME" \
            "Undeleted custom role $ANALYSIS_CUSTOM_ROLE_NAME" \
            gcloud iam roles undelete "$ANALYSIS_CUSTOM_ROLE_NAME" --project="$CURRENT_PROJECT_ID"
    fi

    # Now that we know the role is active, update it to ensure permissions are correct.
    echo "Custom role $ANALYSIS_CUSTOM_ROLE_NAME exists, updating permissions..."
    validate_command \
        "Failed to update custom role $ANALYSIS_CUSTOM_ROLE_NAME" \
        "Updated custom role $ANALYSIS_CUSTOM_ROLE_NAME" \
        gcloud iam roles update "$ANALYSIS_CUSTOM_ROLE_NAME" \
            --project="$CURRENT_PROJECT_ID" \
            --permissions="$ANALYSIS_CUSTOM_ROLE_PERMISSIONS"

else
    # Role does not exist in any state, so create it.
    echo "Creating custom role $ANALYSIS_CUSTOM_ROLE_NAME..."
    validate_command \
        "Failed to create custom role" \
        "Created custom role $ANALYSIS_CUSTOM_ROLE_NAME" \
        gcloud iam roles create "$ANALYSIS_CUSTOM_ROLE_NAME" \
            --project="$CURRENT_PROJECT_ID" \
            --description="$ANALYSIS_CUSTOM_ROLE_DESCRIPTION" \
            --permissions="$ANALYSIS_CUSTOM_ROLE_PERMISSIONS" \
            --stage="GA" \
            --title="$ANALYSIS_CUSTOM_ROLE_TITLE"
fi

for EMAIL in "${ANALYSIS_EMAILS[@]}"; do
  echo "Granting custom role $ANALYSIS_CUSTOM_ROLE_NAME to $EMAIL..."
  validate_command \
    "Failed to grant custom role to $EMAIL" \
    "Granted custom role to $EMAIL" \
    gcloud projects add-iam-policy-binding "$CURRENT_PROJECT_ID" \
      --member="user:$EMAIL" \
      --role="projects/$CURRENT_PROJECT_ID/roles/$ANALYSIS_CUSTOM_ROLE_NAME"
done

# You will need roles/iam.serviceAccountCreator to create a service account
# To Grant the service account access to the project, you need roles/resourcemanager.projectIamAdmin

if gcloud iam service-accounts describe "$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com" >/dev/null 2>&1; then
  log_success "Service account $SERVICE_ACCOUNT_NAME already exists"
else
  echo "Creating service account $SERVICE_ACCOUNT_NAME..."
  validate_command \
  "Failed to create service account" \
  "Created service account $SERVICE_ACCOUNT_NAME" \
  gcloud iam service-accounts create $SERVICE_ACCOUNT_NAME \
    --description="$SERVICE_ACCOUNT_DESCRIPTION" \
    --display-name="$SERVICE_ACCOUNT_DISPLAY_NAME"
fi

echo "Waiting for service account to propagate..."
for i in {1..5}; do
  if gcloud iam service-accounts describe "$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com" >/dev/null 2>&1; then
    break
  fi
  echo "Still waiting..."
  sleep 2
done

for PROJECT in "${SERVICE_ACCOUNT_PROJECT_LIST[@]}"; do
  for ROLE in "${SERVICE_ACCOUNT_PROJECT_ROLES[@]}"; do
    echo "Adding member: serviceAccount:$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com to role $ROLE ..."
    validate_command \
      "Failed to add service account to project role $ROLE" \
      "Added service account to project role $ROLE" \
      gcloud projects add-iam-policy-binding $PROJECT \
        --role="$ROLE" \
        --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com"
  done
done

# Check if the role exists at all (active or deleted)
if gcloud iam roles describe "$SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" --project="$CURRENT_PROJECT_ID" >/dev/null 2>&1; then
    # Role exists, now check if it's deleted
    if gcloud iam roles describe "$SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" --project="$CURRENT_PROJECT_ID" --format="value(deleted)" | grep -q "True"; then
        echo "Custom role $SERVICE_ACCOUNT_CUSTOM_ROLE_NAME is in a deleted state. Undeleting..."
        validate_command \
            "Failed to undelete custom role $SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" \
            "Undeleted custom role $SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" \
            gcloud iam roles undelete "$SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" --project="$CURRENT_PROJECT_ID"
    fi

    # Now that we know the role is active, update it to ensure permissions are correct.
    echo "Custom role $SERVICE_ACCOUNT_CUSTOM_ROLE_NAME exists, updating permissions..."
    validate_command \
        "Failed to update service account custom role" \
        "Updated service account custom role $SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" \
        gcloud iam roles update "$SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" \
            --project="$CURRENT_PROJECT_ID" \
            --permissions="$SERVICE_ACCOUNT_CUSTOM_ROLE_PERMISSIONS"

else
    # Role does not exist in any state, so create it.
    echo "Creating custom role for service account..."
    validate_command \
        "Failed to create service account custom role" \
        "Created custom role $SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" \
        gcloud iam roles create "$SERVICE_ACCOUNT_CUSTOM_ROLE_NAME" \
            --project="$CURRENT_PROJECT_ID" \
            --description="$SERVICE_ACCOUNT_CUSTOM_ROLE_DESCRIPTION" \
            --permissions="$SERVICE_ACCOUNT_CUSTOM_ROLE_PERMISSIONS" \
            --stage="GA" \
            --title="$SERVICE_ACCOUNT_CUSTOM_ROLE_TITLE"
fi

echo "Granting custom role to service account..."
validate_command \
  "Failed to grant custom role to service account" \
  "Granted custom role to service account" \
  gcloud projects add-iam-policy-binding $CURRENT_PROJECT_ID \
    --member="serviceAccount:$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com" \
    --role="projects/$CURRENT_PROJECT_ID/roles/$SERVICE_ACCOUNT_CUSTOM_ROLE_NAME"

# You will need roles/iam.serviceAccountAdmin to create this service account key...
# Or a relevant custom role with iam.serviceAccountKeys.create

echo "Creating service account key..."
validate_command \
  "Failed to create service account key" \
  "Created service account key" \
  gcloud iam service-accounts keys create ~/my-sa-key.json \
    --iam-account="$SERVICE_ACCOUNT_NAME@$CURRENT_PROJECT_ID.iam.gserviceaccount.com"

echo "Downloading service account key..."
validate_command \
  "Failed to download service account key" \
  "Downloaded service account key" \
  cloudshell download my-sa-key.json

echo
read -p "Press Enter once the file has finished downloading..."

echo "Removing local key file..."
validate_command \
  "Failed to remove local key file" \
  "Removed local key file" \
  rm ~/my-sa-key.json

if [[ "$FAILED" -gt 0 ]]; then
  echo "Script completed with $FAILED errors"
else
  echo "Onboarding script completed successfully."
fi        

What the script does

The script automates setting up IAM roles and service accounts in Google Cloud. It creates secure, read-only access for users and programmatic access for automation. The script:

Retrieves the organization ID and project ID for the project where BigQuery exports are located.
Assigns predefined, read-only IAM roles to specific email addresses at the organization and project levels.
Creates a custom IAM role with permissions specific for data visibility and analysis. It then assigns it to specific users.
Sets up a separate, custom IAM role for a service account to monitor analysis metrics. It then sets up a service account and assigns it the custom IAM role with programmatic access.
Generates and downloads a service account key for use in automation.

Save and close the file you just created with the script. If you’re using nano, Ctrl+O > Enter > Ctrl+X.
Make the script executable by running this command: chmod +x setup_gcloud_iam_roles_and_service_accounts.sh.
Run the script: ./setup_gcloud_iam_roles_and_service_accounts.sh.
Click Download to save the service account key to your default downloads folder.

Click Upload Saved JSON File, browse to find your service key: spot-programmatic-access-sa-key.json and click Next.
Enter your BigQuery dataset and table IDs for your detailed billing export without the full path, then select Next.

For example, if your table path is project.detailed_billing_export.gcp_billing_export_resource_v1_xyz, enter dataset ID: detailed_billing_export and table ID: gcp_billing_export_resource_v1_xyz.
Click Connect to Eco. It can take a few hours for your data to show up in the dashboard.

Prerequisites​

Connect a Google Cloud billing account​

Prerequisites

Connect a Google Cloud billing account