Issues on AWS have been resolved and all our tests passed.
The issue has been resolved and all our services are operating normally.
Posted Dec 08, 2021 - 00:37 CET
Our services have been recovered to operational status. All automatic tests passed and now we are testing all services manually to confirm that everything is working correctly.
Information from AWS: "We have executed a mitigation which is showing significant recovery in the US-EAST-1 Region. We are continuing to closely monitor the health of the network devices and we expect to continue to make progress towards full recovery. We still do not have an ETA for full recovery at this time.".
So, on our side, we are monitoring the situation and testing everything manually to ensure that our system is stable.
Posted Dec 07, 2021 - 23:15 CET
We've noticed that our services slowly return to operating.
We are checking, testing, recovery all services.
Posted Dec 07, 2021 - 23:01 CET
We are monitoring the situation and waiting for recovery AWS services. The last update from AWS informs that there is no ETA for full recovery - "While we have observed some early signs of recovery, we do not have an ETA for full recovery.".
We've noticed that we slowly receive access to more and more AWS services but still, the most important services for us don't work properly.
Posted Dec 07, 2021 - 22:50 CET
AWS us-east-1 is experiencing problems with several network devices and this situation impacts multiple global services. Our infrastructure is dependent on some services located in this region and some of our services don't work properly.
Collaboration is partially working properly but it can be problems with storing document data, import/export documents, and webhooks.
Part of our services like collaboration start working. However, we notice that we still have a problem with important parts of our system like webhooks or storing. We are investigating and actively working towards recovery.
Posted Dec 07, 2021 - 20:07 CET
Still, we don't have any reliable information from AWS. Waiting for more information and access to services.
Posted Dec 07, 2021 - 19:08 CET
We are investing a problem and trying to estimate time to recovery our services. At this moment, it is hard due to the fact a huge problem with our cloud provider - AWS. Unfornotutnelty, we are not able to access our servers, and our next steps are quite limited due to a problem with AWS availability. We will inform you about the progress. We are doing what we can to recover our system as soon as it is possible.
Posted Dec 07, 2021 - 18:29 CET
Problem with the availability of the servers infrastructure.
Posted Dec 07, 2021 - 17:56 CET
This incident affected: API (Webhooks, REST API), Export (HTML to PDF, HTML to DOCX), and Collaboration, Easy Image.