We're experiencing an issue with Codefresh pipelines

Incident Report for Codefresh

Postmortem

Impact:

We had a 64 minute window where the CI pipeline view had no data in our SAAS platform.

Detection:

Our monitoring systems immediately alerted us to the issue.

Root Cause:

One of our database collections went offline.

Resolution:

We were able to sync and restore connectivity to this database collection. Some parts of this process took longer than expected and we have implemented improvements to a number of processes to avoid a similar incident in the future.

Posted Jul 05, 2023 - 20:42 UTC

Resolved

This incident has been resolved.

Posted Jun 20, 2023 - 11:09 UTC

Monitoring

A fix has been implemented and we're currently monitoring results. All systems are operational now.

Posted Jun 20, 2023 - 10:48 UTC

Identified

In order to speed up the issue resolution, Codefresh platform went into maintenance mode. Expected resolution time is 30 minutes.

Posted Jun 20, 2023 - 10:25 UTC

Update

In order to speed up the issue resolution, Codefresh platform went into maintenance mode. Expected resolution time is 15 minutes.

Posted Jun 20, 2023 - 10:25 UTC

Update

Codefresh platform is currently under maintenance, expected downtime is up to one hour

Posted Jun 20, 2023 - 10:05 UTC

Update

We are continuing to investigate this issue.

Posted Jun 20, 2023 - 09:59 UTC

Investigating

Pipelines are missing from the Pipelines list. We are currently investigating this issue.

Posted Jun 20, 2023 - 09:49 UTC

This incident affected: Codefresh Systems (codefresh.io).