Skip to content

Add GCS bucket for external TestGrid configurations#9169

Open
ameukam wants to merge 1 commit intokubernetes:mainfrom
ameukam:testgrid-bucket
Open

Add GCS bucket for external TestGrid configurations#9169
ameukam wants to merge 1 commit intokubernetes:mainfrom
ameukam:testgrid-bucket

Conversation

@ameukam
Copy link
Member

@ameukam ameukam commented Mar 3, 2026

Add a GCS bucket so we can migrate testgrid dashboards from Google to the Kubernetes infrastructure

Ref:

@ameukam
Copy link
Member Author

ameukam commented Mar 3, 2026

/hold

@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Mar 3, 2026
@k8s-ci-robot k8s-ci-robot requested review from hakman and upodroid March 3, 2026 18:58
@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. area/infra Infrastructure management, infrastructure design, code in infra/ area/infra/gcp Issues or PRs related to Kubernetes GCP infrastructure area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. labels Mar 3, 2026
@k8s-triage-robot
Copy link

Unknown CLA label state. Rechecking for CLA labels.

Send feedback to sig-contributor-experience at kubernetes/community.

/check-cla
/easycla

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Mar 3, 2026
@ameukam
Copy link
Member Author

ameukam commented Mar 4, 2026

/test pull-k8sio-verify

@ameukam
Copy link
Member Author

ameukam commented Mar 4, 2026

atlantis plan

@k8s-infra-ci-robot

This comment has been minimized.

@k8s-ci-robot k8s-ci-robot added area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects area/prow Setting up or working with prow in general, prow.k8s.io, prow build clusters area/registry.k8s.io Code in registry.k8s.io/ area/release-eng Issues or PRs related to the Release Engineering subproject sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 4, 2026
@k8s-infra-ci-robot
Copy link
Contributor

k8s-infra-ci-robot commented Mar 4, 2026

Argo CD Diff Preview

Summary:

No changes found

No changes found

Stats:
[Applications: 76], [Full Run: 1m35s], [Rendering: 1m29s], [Cluster: 0s], [Argo CD: 2s]

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ameukam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Mar 4, 2026
@k8s-infra-ci-robot

This comment has been minimized.

@ameukam
Copy link
Member Author

ameukam commented Mar 4, 2026

@upodroid look's like some bindings are not defined in Terraform ?

@upodroid
Copy link
Member

upodroid commented Mar 4, 2026

someone merged a broken change to the module

terraform-google-modules/terraform-google-cloud-storage#442

@ameukam
Copy link
Member Author

ameukam commented Mar 9, 2026

@upodroid can we manually actuate this until a fix lands ?

@upodroid
Copy link
Member

upodroid commented Mar 9, 2026

Yes. Pin to the previous version

@k8s-infra-ci-robot

This comment has been minimized.

Add a GCS bucket so we can migrate testgrid dashboards from Google to
the Kubernetes infrastructure

Ref:
  - kubernetes#8973

Signed-off-by: Arnaud Meukam <ameukam@gmail.com>
@k8s-infra-ci-robot
Copy link
Contributor

Ran Plan for dir: infra/gcp/terraform/k8s-infra-prow workspace: default

Show Output
Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the
last "terraform apply" which may have affected this plan:

  # module.prow.google_container_cluster.primary has changed
~ resource "google_container_cluster" "primary" {
        id                                       = "projects/k8s-infra-prow/locations/us-central1/clusters/prow"
      ~ master_version                           = "1.33.5-gke.1308000" -> "1.33.5-gke.2469000"
        name                                     = "prow"
      ~ node_version                             = "1.33.5-gke.1162000" -> "1.33.5-gke.2392000"
        # (36 unchanged attributes hidden)

      ~ node_pool {
            name                        = "prod-v1"
          ~ version                     = "1.33.5-gke.1162000" -> "1.33.5-gke.2392000"
            # (7 unchanged attributes hidden)

            # (5 unchanged blocks hidden)
        }

        # (33 unchanged blocks hidden)
    }

  # module.prow_bucket.google_storage_bucket.bucket has changed
~ resource "google_storage_bucket" "bucket" {
        id                          = "kubernetes-ci-logs"
        name                        = "kubernetes-ci-logs"
      ~ updated                     = "2025-03-16T02:05:17.746Z" -> "2026-01-08T14:46:52.898Z"
        # (16 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.testgrid_config_bucket.google_storage_bucket.bucket has changed
~ resource "google_storage_bucket" "bucket" {
        id                          = "k8s-testgrid-config"
        name                        = "k8s-testgrid-config"
      ~ updated                     = "2024-07-27T01:01:29.112Z" -> "2026-01-01T12:58:10.762Z"
        # (17 unchanged attributes hidden)

        # (4 unchanged blocks hidden)
    }

  # module.utility_cluster.google_container_cluster.primary has changed
~ resource "google_container_cluster" "primary" {
        id                                       = "projects/k8s-infra-prow/locations/us-central1/clusters/utility"
      ~ master_version                           = "1.33.5-gke.1308000" -> "1.33.5-gke.2469000"
        name                                     = "utility"
      ~ node_version                             = "1.33.5-gke.1162000" -> "1.33.5-gke.2392000"
        # (36 unchanged attributes hidden)

      ~ node_pool {
            name                        = "prod-v1"
          ~ version                     = "1.33.5-gke.1162000" -> "1.33.5-gke.2392000"
            # (7 unchanged attributes hidden)

            # (5 unchanged blocks hidden)
        }

        # (33 unchanged blocks hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the
relevant attributes using ignore_changes, the following plan may include
actions to undo or respond to these changes.

─────────────────────────────────────────────────────────────────────────────

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
+ create
~ update in-place
- destroy

Terraform will perform the following actions:

  # google_iam_workload_identity_pool_provider.s390x will be updated in-place
~ resource "google_iam_workload_identity_pool_provider" "s390x" {
        id                                 = "projects/k8s-infra-prow/locations/global/workloadIdentityPools/ibm-clusters/providers/s390x"
        name                               = "projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/providers/s390x"
        # (9 unchanged attributes hidden)

      ~ oidc {
          ~ jwks_json         = jsonencode( # whitespace changes
                {
                    keys = [
                        {
                            alg = "RS256"
                            e   = "AQAB"
                            kid = "pW5IxvjkcZZfO3wT4fj_DPsrl_-CEsl_NxYrDWN0q0w"
                            kty = "RSA"
                            n   = "lwAa1E91aQRA23MQ6AoSDIrxDqJVVVYrKicKc_xhvuIrjarK5-Oz7NDitY9xcHHhF1TK3RbQErQEFjzudb-AEuDqJCLGJuKj0dPyPDwiRUgU6mbC5U0e2z0k0IPnIe__53ty1N2s6SY_Ra1PBppXKp53OhUj62UqZhRs606fnk2aCO-MMNNJ-hkj2kht36cMhU-xQzIBrFVbohSJ6Y1_6ATOIa8F8ExyVoPIJmM0-9VEdMVQKpWu1zzjUxz0R1VHmCMT2YpAnZ-TVe_4rfOGHyCQNaMOKwh6cKy17QD1gTCPQZQTkSW2HnA8m5tB0LS5cbhzscBvdvprzWSnc1ZnUQ"
                            use = "sig"
                        },
                    ]
                }
            )
            # (2 unchanged attributes hidden)
        }
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["0"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "kubernetes-ci-logs"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:prow-build@k8s-infra-prow-build.iam.gserviceaccount.com"
      + role   = "roles/storage.objectAdmin"
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["1"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "kubernetes-ci-logs"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:prow-control-plane@k8s-infra-prow.iam.gserviceaccount.com"
      + role   = "roles/storage.objectAdmin"
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["2"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "kubernetes-ci-logs"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:prow-build-trusted@k8s-infra-prow-build-trusted.iam.gserviceaccount.com"
      + role   = "roles/storage.objectAdmin"
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["3"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "kubernetes-ci-logs"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "principalSet://iam.googleapis.com/projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/attribute.namespace/test-pods"
      + role   = "roles/storage.objectAdmin"
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["4"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "kubernetes-ci-logs"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "principalSet://iam.googleapis.com/projects/773781448124/locations/global/workloadIdentityPools/prow-aks/attribute.namespace/test-pods"
      + role   = "roles/storage.objectAdmin"
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["5"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "kubernetes-ci-logs"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "allUsers"
      + role   = "roles/storage.objectViewer"
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["roles/storage.objectAdmin principalSet://iam.googleapis.com/projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/attribute.namespace/test-pods"] will be destroyed
  # (because key ["roles/storage.objectAdmin principalSet://iam.googleapis.com/projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/attribute.namespace/test-pods"] is not in for_each map)
- resource "google_storage_bucket_iam_member" "members" {
      - bucket = "b/kubernetes-ci-logs" -> null
      - etag   = "CBE=" -> null
      - id     = "b/kubernetes-ci-logs/roles/storage.objectAdmin/principalSet://iam.googleapis.com/projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/attribute.namespace/test-pods" -> null
      - member = "principalSet://iam.googleapis.com/projects/16065310909/locations/global/workloadIdentityPools/ibm-clusters/attribute.namespace/test-pods" -> null
      - role   = "roles/storage.objectAdmin" -> null
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["roles/storage.objectAdmin serviceAccount:prow-build-trusted@k8s-infra-prow-build-trusted.iam.gserviceaccount.com"] will be destroyed
  # (because key ["roles/storage.objectAdmin serviceAccount:prow-build-trusted@k8s-infra-prow-build-trusted.iam.gserviceaccount.com"] is not in for_each map)
- resource "google_storage_bucket_iam_member" "members" {
      - bucket = "b/kubernetes-ci-logs" -> null
      - etag   = "CBE=" -> null
      - id     = "b/kubernetes-ci-logs/roles/storage.objectAdmin/serviceAccount:prow-build-trusted@k8s-infra-prow-build-trusted.iam.gserviceaccount.com" -> null
      - member = "serviceAccount:prow-build-trusted@k8s-infra-prow-build-trusted.iam.gserviceaccount.com" -> null
      - role   = "roles/storage.objectAdmin" -> null
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["roles/storage.objectAdmin serviceAccount:prow-build@k8s-infra-prow-build.iam.gserviceaccount.com"] will be destroyed
  # (because key ["roles/storage.objectAdmin serviceAccount:prow-build@k8s-infra-prow-build.iam.gserviceaccount.com"] is not in for_each map)
- resource "google_storage_bucket_iam_member" "members" {
      - bucket = "b/kubernetes-ci-logs" -> null
      - etag   = "CBE=" -> null
      - id     = "b/kubernetes-ci-logs/roles/storage.objectAdmin/serviceAccount:prow-build@k8s-infra-prow-build.iam.gserviceaccount.com" -> null
      - member = "serviceAccount:prow-build@k8s-infra-prow-build.iam.gserviceaccount.com" -> null
      - role   = "roles/storage.objectAdmin" -> null
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["roles/storage.objectAdmin serviceAccount:prow-control-plane@k8s-infra-prow.iam.gserviceaccount.com"] will be destroyed
  # (because key ["roles/storage.objectAdmin serviceAccount:prow-control-plane@k8s-infra-prow.iam.gserviceaccount.com"] is not in for_each map)
- resource "google_storage_bucket_iam_member" "members" {
      - bucket = "b/kubernetes-ci-logs" -> null
      - etag   = "CBE=" -> null
      - id     = "b/kubernetes-ci-logs/roles/storage.objectAdmin/serviceAccount:prow-control-plane@k8s-infra-prow.iam.gserviceaccount.com" -> null
      - member = "serviceAccount:prow-control-plane@k8s-infra-prow.iam.gserviceaccount.com" -> null
      - role   = "roles/storage.objectAdmin" -> null
    }

  # module.prow_bucket.google_storage_bucket_iam_member.members["roles/storage.objectViewer allUsers"] will be destroyed
  # (because key ["roles/storage.objectViewer allUsers"] is not in for_each map)
- resource "google_storage_bucket_iam_member" "members" {
      - bucket = "b/kubernetes-ci-logs" -> null
      - etag   = "CBE=" -> null
      - id     = "b/kubernetes-ci-logs/roles/storage.objectViewer/allUsers" -> null
      - member = "allUsers" -> null
      - role   = "roles/storage.objectViewer" -> null
    }

  # module.testgrid_config_external_bucket.google_storage_bucket.bucket will be created
+ resource "google_storage_bucket" "bucket" {
      + effective_labels            = {
          + "goog-terraform-provisioned" = "true"
        }
      + force_destroy               = false
      + id                          = (known after apply)
      + location                    = "US-CENTRAL1"
      + name                        = "k8s-testgrid-config-external"
      + project                     = "k8s-infra-prow"
      + project_number              = (known after apply)
      + public_access_prevention    = "inherited"
      + rpo                         = (known after apply)
      + self_link                   = (known after apply)
      + storage_class               = "STANDARD"
      + terraform_labels            = {
          + "goog-terraform-provisioned" = "true"
        }
      + time_created                = (known after apply)
      + uniform_bucket_level_access = true
      + updated                     = (known after apply)
      + url                         = (known after apply)

      + autoclass {
          + enabled                = false
          + terminal_storage_class = (known after apply)
        }

      + hierarchical_namespace {
          + enabled = false
        }

      + soft_delete_policy {
          + effective_time             = (known after apply)
          + retention_duration_seconds = 604800
        }

      + versioning {
          + enabled = true
        }

      + website (known after apply)
    }

  # module.testgrid_config_external_bucket.google_storage_bucket_iam_member.members["0"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "k8s-testgrid-config-external"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:k8s-testgrid-config-updater@k8s-infra-prow-build-trusted.iam.gserviceaccount.com"
      + role   = "roles/storage.objectAdmin"
    }

  # module.testgrid_config_external_bucket.google_storage_bucket_iam_member.members["1"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "k8s-testgrid-config-external"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:testgrid-canary@k8s-testgrid.iam.gserviceaccount.com"
      + role   = "roles/storage.objectViewer"
    }

  # module.testgrid_config_external_bucket.google_storage_bucket_iam_member.members["2"] will be created
+ resource "google_storage_bucket_iam_member" "members" {
      + bucket = "k8s-testgrid-config-external"
      + etag   = (known after apply)
      + id     = (known after apply)
      + member = "serviceAccount:updater@k8s-testgrid.iam.gserviceaccount.com"
      + role   = "roles/storage.objectViewer"
    }

Plan: 10 to add, 1 to change, 5 to destroy.
  • ▶️ To apply this plan, comment:
    atlantis apply -d infra/gcp/terraform/k8s-infra-prow
  • 🚮 To delete this plan and lock, click here
  • 🔁 To plan this project again, comment:
    atlantis plan -d infra/gcp/terraform/k8s-infra-prow

Note: Objects have changed outside of Terraform
Plan: 10 to add, 1 to change, 5 to destroy.


  • ⏩ To apply all unapplied plans from this Pull Request, comment:
    atlantis apply
  • 🚮 To delete all plans and locks from this Pull Request, comment:
    atlantis unlock

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. area/artifacts Issues or PRs related to the hosting of release artifacts for subprojects area/infra/gcp Issues or PRs related to Kubernetes GCP infrastructure area/infra Infrastructure management, infrastructure design, code in infra/ area/prow Setting up or working with prow in general, prow.k8s.io, prow build clusters area/registry.k8s.io Code in registry.k8s.io/ area/release-eng Issues or PRs related to the Release Engineering subproject area/terraform Terraform modules, testing them, writing more of them, code in infra/gcp/clusters/ cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. sig/apps Categorizes an issue or PR as relevant to SIG Apps. sig/cluster-lifecycle Categorizes an issue or PR as relevant to SIG Cluster Lifecycle. sig/k8s-infra Categorizes an issue or PR as relevant to SIG K8s Infra. sig/release Categorizes an issue or PR as relevant to SIG Release. sig/scheduling Categorizes an issue or PR as relevant to SIG Scheduling. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants