All Systems Operational
FRA1 - XDR ? Operational
Ingestion Operational
Threat Intelligence for research & triage Operational
Automation Operational
Event storage Operational
Detection Operational
Hunting Operational
Case management Operational
Web application Operational
FRA1 - CTI ? Operational
Search Operational
API consumption Operational
TAXII consumption Operational
MISP consumption Operational
Enrichers Operational
Web application Operational
FRA2 - XDR (SecNumCloud / PCI DSS region) ? Operational
MCO1 - XDR ? Operational
EUR1 - XDR ? Operational
UAE1 - XDR ? Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Jul 27, 2024

No incidents reported today.

Jul 26, 2024

No incidents reported.

Jul 25, 2024
Resolved - An overload network component got overloaded, which resulted in a high error rate on our APIs and frontend between 12:27 and 12:47. Our team has been working on decommissioning this component while limiting side effects in the meantime, which means this won't be an issue in the near future.
We are sorry for the inconvenience.

Jul 25, 12:30 CEST
Jul 24, 2024
Resolved - This incident has been resolved, nothing new happened since the last update. A detailed communication is being prepared by our customer facing teams.
Jul 24, 16:31 CEST
Monitoring - The OVHcloud API and control plane is back online, which allowed the OVH support team to fix their DDoS protection mechanism. FRA1 is now fully reacheable and operational.
We are seeing a nominal volume of incoming events, our API error rates are low and our external TCP probes are reporting no errors.
Our team is still actively monitoring the situation to ensure FRA1 remains stable.

We are closely working with OVHcloud to produce a post-mortem and ensure their DDoS protection does not erroneously trigger again.

Jul 24, 12:56 CEST
Update - OVHcloud is having a parallel incident which made their administration console and API unavailable. This means that both OVH support and Sekoia.io incident handlers are unable to perform any changes to our FRA1 infrastructure.

We are waiting for the OVH control plane to get back online for them to disable the faulty DDoS protection rules.

In the meantime, our team is looking for workarounds.

Jul 24, 12:10 CEST
Identified - Our cloud provider for FRA1 (OVHcloud) has allegedly flagged our cloud load balancer traffic as a DDoS attempt, resulting in a complete loss of access for the web application, APIs, and intakes. Their DDoS protection is currently flapping on and off, resulting in an unstable access to FRA1.
We are working with our OVH Technical Account Manager to resolve this as soon as possible.

Jul 24, 10:51 CEST
Investigating - We are currently facing a major outage due to a problem on our cloud provider side, impacting our reverse proxies.
Our team is getting in touch with them to get more information and try to fix this.

Jul 24, 10:32 CEST
Jul 23, 2024

No incidents reported.

Jul 22, 2024
Resolved - This incident has been resolved.
Jul 22, 23:52 CEST
Monitoring - A fix has been implemented, we should catch up quickly.
Jul 22, 21:44 CEST
Identified - We identified a performance issue on event storage. This is causing a delay on events being stored and available on the UI.
Detection is still in real time and no data loss.

Jul 22, 21:19 CEST
Jul 21, 2024

No incidents reported.

Jul 20, 2024

No incidents reported.

Jul 19, 2024

No incidents reported.

Jul 18, 2024

No incidents reported.

Jul 17, 2024
Resolved - This incident has been resolved.
Jul 17, 17:19 CEST
Monitoring - Situation is back to normal. We are monitoring the situation in case there is an other problem.
Jul 17, 15:47 CEST
Identified - We identified the root cause. All servers disks are much slower than usual due to a problem on the cloud provider part.
We are trying to find a work around. We are also waiting for a intervention from our cloud provider to improve the situation.

Jul 17, 15:45 CEST
Investigating - We are currently investigating on the issue.
Some events may not be received properly on the region.
We suspect an issue on our cloud provider system.

Jul 17, 14:58 CEST
Jul 16, 2024

No incidents reported.

Jul 15, 2024

No incidents reported.

Jul 14, 2024

No incidents reported.

Jul 13, 2024

No incidents reported.