Summary
Between 15:31 UTC and 15:55 UTC on October 29th, runs for customers using ARM runners failed to start with the following error message:

couldn't start worker: couldn't start docker container: error populating worker binary: unexpected response code when querying for worker binary at https://downloads.spacelift.io/worker-arm64/752bab1d/spacelift-worker: 404

Incident Timeline

• 15:31 UTC - A new deployment of the Spacelift backend was triggered.
• 15:45 UTC - We triggered a rollback to the previous version.
• 15:50 UTC - The fix began to take effect.
• 15:55 UTC - The fix was fully rolled out.

Root Cause
The problem was caused by our build pipeline failing silently to upload that single file. When we triggered a new deployment of the Spacelift backend, new runs switched over to the new worker version and began to fail because the file was missing.

Mitigations

• Immediate: we rolled back to a previous working version.
• Permanent: we have updated our deployment pipeline to ensure that it cannot silently fail to upload assets like this in future.