Thursday 4th May 2023

General communicate service [SEWAN] Unexpected disturbances - 04.05.2023 (SOLVED)

INCIDENT REPORT

 

At 1:06 PM on the 4th of May 23, one of our cached internal routing tables got corrupted. These tables are used to decide to which internal platform a number must be routed towards. The event was immediately identified by our monitoring systems, triggering alerts toward our IT team.

This resulted that part of our DIDs* were no longer reachable. The first impact was felt at 1:10 PM.

We instantly took measures to mitigate this issue.

By 1:38 PM, 90% of the impacted services were already going through the alternate table, located into a separate datacenter.

As a corrective action, we replaced the routing tables with a stronger version, and improved the recovery procedure in case of such issue. Not all platforms were impacted*. During the timeframe, the problem became less impacting throughout the resolution.

We are confident our resolution is profound and future-proof.

 

*Our major “V2” VoIP platform was not impacted.


UPDATE - 1.45 PM

 

Service back to normal. Technical investigation on going.

May you have any remaining issue after 1.45 PM today, please contact us : support@sewan.be


UPDATE - 1.38 PM

 

Service back to normal by 90%.


UPDATE - 1.31 PM

 

Restoration in progress. ETA : 1.45 PM


UPDATE - 1.26 PM

 

Redis issue identify. On our way for fixing.

More info will follow.


   

Beginning: 04.05.2023 – 1.10 PM

Ending: 04.05.2023 – 1.45 PM

Impact: inbound fixed telephony impacted

 

Dear customer, dear partner,

 

We are experiencing a problem on our inbound calls.

Outbound calls are not impacted.

Technical team is on it. More info will follow.

 

Best regards;

SEWAN technical team