Summary
On March 11, 2025, between 14:59 and 16:26 PDT, some users experienced issues loading the login page and/or completing password-based account creation. During this period, a portion of traffic was unintentionally routed to deprecated OpenAI infrastructure, resulting in login pages returning errors or failing to load for a subset of users. We resolved the issue by removing the affected infrastructure from service, which immediately restored normal functionality. The issue was fully mitigated by 16:26 PDT.
Impact
A significant number of login page requests failed during the incident window.
A subset of authentication attempts were unsuccessful.
A subset of account creation attempts did not complete successfully.
Users experienced blank pages or 404 errors when attempting to log in or sign up.
Root Cause
The issue was caused by a configuration change that unintentionally restored traffic to deprecated backend origins
Contributing Factors
The workflow used to restore traffic included a mode that increased traffic to all configured origins, including deprecated ones.
Deprecated infrastructure had not yet been fully removed from configuration, allowing traffic to be routed to it.
Existing monitoring did not clearly surface traffic being routed to these deprecated or non-functional origins, delaying detection.
Some requests were routed to endpoints that did not return valid responses, which were not fully captured by backend error monitoring.
Resolution
We mitigated the issue by removing the deprecated origins from the traffic routing configuration, which promptly restored normal login and account creation behavior.
Prevention and Next Steps
To reduce the risk of similar issues in the future, we have implemented the following improvements:
Updated internal tooling to support traffic restoration workflows not enabling deprecated infrastructure.
Introduced clearer safeguards and confirmations in operational workflows.
Improved monitoring to detect traffic routed to non-functional or deprecated systems.
Removed unused or deprecated infrastructure more proactively.
We apologize for the disruption and are continuing to improve the safety and reliability of our systems.
