SaaS Builds Stuck In Pending due to 3rd party outage

Incident Report for Codefresh

Resolved

The incident of Builds Stuck in Pending is now resolved. Builds are being scheduled and running normally.
Posted Dec 08, 2021 - 02:15 UTC

Monitoring

Builds are being scheduled normally and running. We continue to monitor as the backlog of builds starts to run and return to normal.
Posted Dec 08, 2021 - 01:14 UTC

Update

We are continually working on the issue with builds stuck in pending. We see an increase in builds running, and our capacity is going back to normal.
Posted Dec 08, 2021 - 00:42 UTC

Update

We continue to work with AWS to get capacity back to our normal capacity as we understand builds are slowly moving through. If you have seen a builds terminated due to max retry attempts, you can restart the build to try again. Alternatively, you can use the Hybrid Runner to run builds. Feel free to reach out to Support if you have questions about installing and using the Hybrid Runner.
Posted Dec 07, 2021 - 21:36 UTC

Update

We are currently seeing builds starting to run. We are continuing to monitor this issue.
Posted Dec 07, 2021 - 18:34 UTC

Update

We are currently working with AWS to resolve this issue. At the same time, we are working on moving builds to use another Cloud Provider. If you have essential builds that need to run, the Hybrid Runners will continue to work. Reach out to Support if you need assistance using a Hybrid Runner.
Posted Dec 07, 2021 - 17:54 UTC

Identified

We have Identified the issue. There is an issue with AWS with the us-east-1 region.
Posted Dec 07, 2021 - 16:27 UTC

Investigating

Some builds are currently stuck in pending. We are investigating this issue and will continue to update this page with further information
Posted Dec 07, 2021 - 16:11 UTC
This incident affected: Codefresh Systems (codefresh.io).