What happened
Google Cloud: IAM configuration change propagated globally past canary. Scope: Global.
CHYP.ai · ORBIT // Cloudy_Day Security Suite
OUTAGE DISSECTOR · CASE FILE · Meltdown Index · vendor post-incident report
Google Cloud: IAM configuration change propagated globally past canary. Scope: Global.
Meltdown
Drag or press Play to replay what was public at each moment. Each dot links to its evidence. Lanes are kept separate on purpose.
Google Cloud: IAM configuration change propagated globally past canary. Scope: Global.
Cloud IAM, Cloud SQL, BigQuery streaming, dependent Cloud APIs.
75 catalogued services run on Google Cloud (inferred). Hit: Global.
Resolved after 50m.
Center: Google Cloud. Inner ring: what it runs on. Outer ring: catalogued services that run on it, coloured by their official status right now. Dependencies are inferred from public information.
Places come from names in the public text (cities, PoP codes, countries, regions). Approximate.
Failure modes as stated in the vendor's own post-incident report (Meltdown Index).
Correlation is not causation
Five independent lanes across the window. Brighter = stronger item. Items landing in the same column are coincidence unless the provider says otherwise.
A latent issue with our canarying system allowed this specific type of configuration change to propagate globally in a rapid manner. As the configuration was pushed to production, the global Cloud IAM service quickly began to experience internal errors.
Engineers were alerted at 06:52 US/Pacific; the change was identified at 07:27 and the mitigation finished propagating by 07:42 for a majority of internal services. Some services (external Cloud IAM API, high-availability Cloud SQL, BigQuery streaming) took longer. The later end is not used as a single official close time here.
Verbatim excerpt from Google Cloud Service Health — IAM / Zanzibar incident, 8 Apr 2020.
90% similar · same provider
Network-control-plane configuration update dropped GCVE private traffic
Config pushControl plane
85% similar · same provider
Service Control crash loop after a blank quota-policy field went global
Control planeConfig pushCapacity
85% similar · same provider
Automated quota system shrank User ID Service capacity
CapacityConfig pushControl plane
Nearest matches in the Control-Plane Meltdown Index (1.0.0, vendor-sourced, tracked since 2024 with earlier landmark cases). Similarity = provider + failure modes + scope.
Automated take: rules written by Cloudy_Day, applied to public data. Not a human verdict on this incident, not a root-cause finding, not affiliated with Google Cloud.
Everything above as one shareable, citable capsule with a content hash.
JSON capsuleMarkdownShare card (SVG)
Open Google Cloud in the War Room →Status page →
<a href="https://www.chyp.ai/dissect/m/gcp-iam-zanzibar-2020-04-08/"><img src="https://www.chyp.ai/dissect/m/gcp-iam-zanzibar-2020-04-08/card.svg" alt="IAM configuration change propagated globally past canary · ORBIT Outage Dissector" width="600"></a>