Thunderhead ONE Status
Degraded Performance on NA5 Production Environment
Update - Customers may still be experiencing high response times on our North American environment. We are continuing to make adjustments in an effort to increase stability. We will be watching closely over the weekend for any changes.
Dec 6, 17:37 EST
Identified - As of 5:30 EST, we have taken measures to restore availability and performance and as such there will be short periods of increased API & Interaction error rates over the next few hours. We will provide further updates when more information becomes available.
Dec 4, 21:59 EST
Investigating - Customers may be experiencing high response times on our North American production environment as of 22:25 EST on December 3rd, 2019. We are investigating the issue as our top priority and will provide a status update when more information is available.
Dec 4, 14:43 EST
Degraded Performance on EU2 Production Environment
Update - Our engineers have identified an issue causing our heightened response times and have applied a mitigation to address the issue. Response times have returned to normal levels and remain stable. We will continue monitoring over the weekend.
Dec 6, 17:35 EST
Update - We are continuing to monitor our European production environment. After applying the updates to our database configuration, we have seen response times remain stable. We are noting a status of degraded performance until we are absolutely certain that the issue has been resolved.
Dec 4, 22:20 EST
Monitoring - After further consultation with our database vendor we have applied additional configuration changes to our database. Full stability has been restored to our European production environment as of 19:22 UTC. We continue to have slower than normal response times and are actively monitoring it.
Dec 3, 17:50 EST
Investigating - Customers may be experiencing high response times on our European production environment as of 16:29 UTC on December 3rd, 2019. Our engineers are currently investigating the issue and we will provide updates as soon as we have more information.
Dec 3, 11:53 EST
Data Exports
Monitoring - During the previously noted degraded performance on our European production environment, customer data extracts were suspended. After resolution was achieved through changes to our database infrastructure over this past weekend, the extract processing has been recovering past extracts while also running regularly scheduled extracts, resulting in longer delivery times than normal. We allocated a team of engineers to assist with the processing of past extracts throughout the week and the team will continue to process these until all are recovered and available. We are also investigating alternative processes to remediate these impacts going forward. Please contact your Thunderhead Customer Success team with any specific questions or needs and we will work together to assist you.
Dec 6, 17:11 EST

About This Site

This page provides real-time status and performance information for Thunderhead services. For additional information or to report a problem, please visit the ONE Customer Support Portal at https://thunderhead.jira.com/servicedesk/customer/portal/1

Uptime over the past 90 days. View historical uptime.
Thunderhead ONE Europe (EU2) ? Degraded Performance
90 days ago
99.91 % uptime
Today
Thunderhead ONE North America (NA5) ? Degraded Performance
90 days ago
100.0 % uptime
Today
Thunderhead ONE Asia Pacific (AP1) ? Operational
90 days ago
99.99 % uptime
Today
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
had a major outage
had a partial outage

No incidents or maintenance related to this downtime.

Past Incidents
Dec 7, 2019

No incidents reported today.

Dec 5, 2019
"Royal Blood" Production Release for Thunderhead European Customers
Completed - The scheduled maintenance has been completed as of 1:09AM CEST.
Dec 5, 18:19 EST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Dec 5, 18:00 EST
Scheduled - Thunderhead will be deploying the latest release of ONE called Royal Blood, on December 6th 2019, beginning at 12AM CET. This deployment will impact all customers being served from the Thunderhead European production environment.
Nov 26, 05:30 EST
Dec 3, 2019
"Royal Blood" Production Release for Thunderhead North American Customers
Completed - The scheduled maintenance has been completed as of 10:22PM EDT.
Dec 3, 22:29 EST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Dec 3, 22:00 EST
Scheduled - Thunderhead will be deploying the latest release of ONE, called Royal Blood on December 3rd 2019, beginning at 10PM EST. This deployment will impact all customers being served from the Thunderhead North American production environment.
Nov 26, 05:23 EST
"Royal Blood" Production Release for Thunderhead Asia Pacific Customers
Completed - The scheduled maintenance has been completed as of 10:15PM SGT.
Dec 3, 09:30 EST
In progress - Scheduled maintenance is currently in progress. We will provide updates as necessary.
Dec 3, 09:00 EST
Scheduled - Thunderhead will be deploying the latest release of ONE, called Royal Blood, on December 3rd 2019, beginning at 10PM SGT. This deployment will impact all customers being served from the Thunderhead Asia Pacific production environment.
Nov 26, 05:16 EST
Dec 2, 2019

No incidents reported.

Dec 1, 2019
Degraded Performance on EU2 Production Environment
Resolved - We have implemented configuration changes and updated our database infrastructure over the weekend, and full stability has been restored to our European production environment (as of December 1st, 4:15 CET). We will provide a postmortem with more details about the root cause of the issue during this coming week.
Dec 1, 23:52 EST
Update - Customers are currently experiencing degraded performance on our European production environment (as of 09:30 UTC on November 29th, 2019). We are taking measures to restore availability and performance and as such there will be short periods of increased API & Interaction error rates over the next few hours. We will provide further updates during the day and when more information becomes available.
Nov 29, 08:27 EST
Update - Customers are currently experiencing degraded performance on our European production environment (as of 09:30 UTC on November 29th, 2019). We will provide further updates during the day and when more information becomes available.
Nov 29, 06:26 EST
Update - The European production environment has remained stable throughout the night. We continue to have slower than normal response times and are continuing to actively monitor it.
Nov 28, 09:10 EST
Monitoring - We have implemented a temporary configuration change in our database servers and performance of the European production environment has stabilized. In the meantime, we are continuing to work with vendors to find the root cause of this incident and will continue to provide updates until the issue is resolved.
Nov 27, 18:02 EST
Update - We currently have a ticket open with our database vendor to address the issue and they are actively investigating our nodes to help identify where the issue lies. In addition, they are also reviewing our configuration files to provide feedback.

We have also reached out to our cloud provider's critical support team to help investigate if the issue lies within our hosted cloud platform since the issue is isolated to a subset of servers.
Nov 27, 14:28 EST
Identified - Customers may be experiencing high response times on our European production environment (as of 11:35 UTC on November 27th, 2019). Our engineers have identified the cause of the disruption as an issue with some of our database nodes and are working to resolve. We will provide updates during the day.
Nov 27, 09:11 EST
Nov 30, 2019

No incidents reported.

Nov 26, 2019
Degraded Performance on EU2 Production Environment
Resolved - This incident has been resolved (as of 04:10 UTC on November 27th, 2019) and the European production environment is operating as expected.
Nov 26, 23:15 EST
Investigating - Customers may currently be experiencing high response times on our European production environment (as of 22:00 UTC on November 26th, 2019). Our engineers are currently investigating the issue and we will provide updates as soon as we have more information.
Nov 26, 17:00 EST
Degraded Performance on NA5 Production Environment
Resolved - This incident has been resolved as of 12:37 EDT on November 26, 2019 and the North American production environment is operating as expected.
Nov 26, 12:56 EST
Investigating - Customers may currently be experiencing degraded performance on our North American production environment (as of 11:50 EST on November 26th, 2019). Our engineers are currently investigating the issue and we will provide updates as soon as we have more information.
Nov 26, 12:29 EST
Nov 25, 2019
Degraded Performance on EU2 Production Environment
Resolved - This incident has been resolved as of 10:48 UTC and the European production environment is operating as expected.
Nov 25, 18:52 EST
Investigating - All customers may currently be experiencing degraded performance on our European production environment (as of 09:04 - 10:48 UTC on November 25th, 2019). Our engineers are currently investigating the issue and we will provide updates as soon as we have more information.
Nov 25, 06:22 EST
Nov 24, 2019

No incidents reported.

Nov 23, 2019

No incidents reported.

← Incident History Powered by Statuspage