SEDNA Not Loading Messages
Incident Report for SEDNA Systems
Postmortem

Incident Report

SEDNA Incident 

11 Jan 2022

Summary

On January 11, 2022 users on our default cluster experienced issues with SEDNA not loading messages being received by teams. 

Why this happened

As part of improvements SEDNA was making to our Search feature there was a temporary issue with importing data which caused a spike in load on our servers which resulted in the degraded performance users experienced.

Actions and opportunities for improvement

  1. Improve logging and reporting on the search functionality - COMPLETE
  2. Revisit and improve change management process - IN PROGRESS
Posted Jan 27, 2022 - 14:56 UTC
Resolved
This incident has been resolved.
Posted Jan 11, 2022 - 15:58 UTC
Update
Our Engineering team has resolved the issue and all messages have now been loaded in SEDNA.
Posted Jan 11, 2022 - 15:58 UTC
Update
We are continuing to monitor the situation with SEDNA not loading messages.
Posted Jan 11, 2022 - 14:57 UTC
Monitoring
Our Engineering team has deployed a fix and are seeing that SEDNA is stabilising. We are continuing to monitor the situation.
Posted Jan 11, 2022 - 14:28 UTC
Identified
Our team has identified the issue that is causing the message to not load in SEDNA and are actively looking at ways to resolve this.
Posted Jan 11, 2022 - 14:17 UTC
Update
Our team continues to investigate the issue which is causing a long delay in messages being loaded in SEDNA
Posted Jan 11, 2022 - 13:51 UTC
Investigating
Our team is actively investigating an issue that is causing messages not to load in SEDNA.
Posted Jan 11, 2022 - 13:22 UTC
This incident affected: SEDNA - www.sednanetwork.com and SEDNA - Private Hosting.
← Current Status Powered by Atlassian Statuspage