Latency on Podio web app
Incident Report for Podio
Resolved
We will close the issue as resolved, as we have seen improved performance in the early morning hours of CET the last 9 days since the root cause was identified and fixes applied last week. We have additional actions to reduce load on specific API calls in the future, in order to prevent similar bottlenecks to appear. We will also be announcing a larger database maintenance event soon, which will be performed over a weekend to help improve performance further. Thanks for your patience.
Posted 6 months ago. Mar 22, 2019 - 09:56 EDT
Update
We are continuing to monitor however Podio has been stable since 10.15 EST. We have seen better performance in the morning of CET and is now investigating what caused the issues today in the morning of EST (US time). Thanks for your patience. We will keep this incident open until both issues have been fully confirmed resolved.
Posted 6 months ago. Mar 18, 2019 - 17:25 EDT
Monitoring
We are monitoring performance as things have been stable the last 30 minutes.
Posted 6 months ago. Mar 18, 2019 - 10:37 EDT
Investigating
We are looking into reports of Podio being unavailable at the moment.
Posted 6 months ago. Mar 18, 2019 - 10:10 EDT
Update
We are investigating errors on Podio at the moment.
Posted 6 months ago. Mar 18, 2019 - 09:37 EDT
Update
Podio is still fully operational however we are keeping the incident open until we confirm the spikes in early morning CET has been resolved on Mondays and Tuesdays as well, this is our peak time for load on Podio. We will close the incident when resolution has been confirmed this week.
Posted 6 months ago. Mar 17, 2019 - 20:51 EDT
Monitoring
Podio has been operational the last 2 days and we believe to have identified cause of the early spikes in performance in mornings CET. We will have follow up actions to prevent similar case to occur in the future. We are still monitoring results and will have additional actions the following weeks to improve further.
Posted 6 months ago. Mar 14, 2019 - 10:52 EDT
Update
We have seen good initial results from the cause analysis and changes done from yesterday, however we are still monitoring results and will keep the incident open for another few days to confirm full resolution on a daily basis. Podio is still fully operational and have seen good performance today so far.
Posted 6 months ago. Mar 13, 2019 - 08:05 EDT
Update
Update: We are still investigating different routes of cause for the spike in latency in the morning hours of CET. We had issues today from around 8.30 CET to 9.10 CET and have seen similar symptoms in the morning hours of CET over the last 2 weeks. We are doing everything we can to find cause and longterm solution. We will share additional updates in the morning CET. We will keep this incident open until we have root cause defined and action plan in place.
Posted 6 months ago. Mar 12, 2019 - 11:30 EDT
Update
We are getting closer to understanding the root cause and will update as we have more confirmation. Performance has been stable the last 2 hours.
Posted 6 months ago. Mar 12, 2019 - 06:50 EDT
Identified
Performance is improving, we continue to investigate the issue.
Posted 6 months ago. Mar 12, 2019 - 04:20 EDT
Investigating
We are currently investigating the issue
Posted 6 months ago. Mar 12, 2019 - 04:10 EDT
This incident affected: Web and API.