Some fine-tuned snapshots are missing sampling permissions after finishing training

Resolved·Degraded performance

This incident has been resolved.

Tue, Sep 24, 2024, 05:04 AM

(8 months ago)

Affected components

Sep 24, 2024, 03:08 AM

05:04 AM

Updates

Resolved

This incident has been resolved.

Tue, Sep 24, 2024, 05:04 AM

Monitoring

A fix has been fully deployed and fine-tuned snapshots can be sampled from normally again. Impacted fine-tunes were those created after 3:48pm PT on 9/23.

Tue, Sep 24, 2024, 05:04 AM

Identified

A fix is being deployed, 404 errors should subside before 9:30pm PT.

Tue, Sep 24, 2024, 04:19 AM(45 minutes earlier)

Investigating

Issue has been identified and reproduced internally, we are preparing a fix.

Tue, Sep 24, 2024, 03:50 AM(28 minutes earlier)

Investigating

We are continuing to investigate this issue. It appears to impact some fine-tuned models created after 4:10pm PT today (9/23)

Tue, Sep 24, 2024, 03:39 AM(11 minutes earlier)

Investigating

We are investigating reports that sampling requests to fine-tuned models are returning 404 due to lack of sampling permissions.

Tue, Sep 24, 2024, 03:08 AM(31 minutes earlier)

Availability metrics are reported at an aggregate level across all tiers, models and error types. Individual customer availability may vary depending on their subscription tier as well as the specific model and API features in use.