Slow Performance Issues
Incident Report for SEDNA Systems
Postmortem

14 May 2019

Summary

On May 14th at approximately 11:00 customers on our European cluster experienced reduced functionality due to SEDNA’s third-party search provider failing. This caused SEDNA to enter into reduced functionality mode with search being slower than normal.

Why this happened

This occurred due to a setting on the server coupled with a long query that caused the server to become overloaded

Actions and opportunities for improvement

Update the server to ensure this does not happen again - COMPLETE

Ensure correct alerts are set for early detection - COMPLETE

Posted May 16, 2019 - 14:25 UTC

Resolved
The team have confirmed SEDNA is back to regular speed and search and confirm this incident resolved.
Posted May 14, 2019 - 10:28 UTC
Monitoring
Our team have identified the issue and have resolved this. Loading and searching is back to normal and the team continue to monitor system performance.
Posted May 14, 2019 - 10:23 UTC
Investigating
Our team are currently investigating issues with loading messages - performance will be degraded while this is being worked on. This is only affecting clients on our European Cluster.
Posted May 14, 2019 - 10:13 UTC