24 November 2020
On November 24th 2020 between the hours 8am GMT and 10am GMT SEDNA was inaccessible to some users and some users were experiencing degraded services.
SEDNA relies on a cloud provider for our search functionality and they began experiencing difficulties which in turn had an impact on SEDNA’s ability to load messages and function as expected. One node in a regional search cluster became inaccessible in a way that led to inconsistent and delayed responses. We worked with AWS to resolve the problem and in the mean-time initiated a redeployment of the search cluster - that redeployment succeeded and service was fully restored.
Actions and opportunities for improvement
3. Enhanced alarms around search service - COMPLETED