Slow Performance Issues

Incident Report for Sedna Systems

Postmortem

14 May 2019

Summary

On May 14th at approximately 11:00 customers on our European cluster experienced reduced functionality due to SEDNA’s third-party search provider failing. This caused SEDNA to enter into reduced functionality mode with search being slower than normal.

Why this happened

This occurred due to a setting on the server coupled with a long query that caused the server to become overloaded

Actions and opportunities for improvement

Update the server to ensure this does not happen again - COMPLETE

Ensure correct alerts are set for early detection - COMPLETE

Posted May 16, 2019 - 15:25 BST

Resolved

The team have confirmed SEDNA is back to regular speed and search and confirm this incident resolved.

Posted May 14, 2019 - 11:28 BST

Monitoring

Our team have identified the issue and have resolved this. Loading and searching is back to normal and the team continue to monitor system performance.

Posted May 14, 2019 - 11:23 BST

Investigating

Our team are currently investigating issues with loading messages - performance will be degraded while this is being worked on. This is only affecting clients on our European Cluster.

Posted May 14, 2019 - 11:13 BST