Resolved -
Amazon has deployed a server-side update this morning to correct the issue and we've tested and validated internally. As such, beta participates will need to disable and re-enable the skill in order to restore service. Instructions were sent to all beta participants at 08:00 PDT this morning.
We will continue to monitor this incident until we see traffic and performance return to normal for all users.
Again, please accept our sincerest apologize for this extended delay. We will issue a comprehensive postmortem in the coming days that details the root causes, as well as the procedures Amazon, MiOS, and Vera are putting in place to ensure an outage of this duration does not take place again in the future.
Thank you for your patience and continued support.
Aug 10, 07:59 PDT
Monitoring -
The MiOS and Amazon engineering teams worked collaboratively throughout the day yesterday to conduct additional diagnostics and testing to properly address this issue.
Amazon is deploying a server-side update this morning to correct the issue and restore service.
This change will take up to 48 hours to propagate completely throughout their production environments, worldwide.
We will continue to monitor this incident until we see traffic and performance return to normal for all users.
Again, please accept our sincerest apologize for this extended delay. We will issue a comprehensive postmortem in the coming days that details the root causes, as well as the procedures Amazon, MiOS, and Vera are putting in place to ensure an outage of this duration does not take place again in the future.
Thank you for your patience and continued support.
Aug 9, 09:26 PDT
Update -
Amazon team has indicated that further diagnoses are required and their earliest update will come tomorrow.
We apologize for this added delay and thank you for your patience and continued support as we work with our partner to restore service.
Aug 7, 21:49 PDT
Update -
We are awaiting a follow on update from the team at Amazon. Will provide information as soon as it is available.
They are targeting to have service restored before the end of the day, at the absolute latest.
Aug 7, 16:33 PDT
Identified -
The team at Amazon has diagnosed and isolated the issue surrounding access tokens. We will update this incident again in the next 90 minutes with an ETA to service restoration.
Thank you for your patience and continued support as we work with our partner to restore service.
Aug 7, 14:17 PDT
Update -
The team at Amazon is continuing diagnostics on their end, but has found nothing conclusive thus far. Their current ETA for an update, and or, remediation is Monday morning (PT), at the earliest. We understand the impact of this extended outage and will continue to pursue escalation.
Thank you for your patience and continued support as we work with our partner to restore service.
Aug 5, 00:45 PDT
Investigating -
Beginning around 2am PT today we started receiving reports from beta participants in the Alexa program (US market) that device control commands were not executing successfully. Our QA Team has subsequently been able to reproduce the behavior and confirm the issue internally.
We’ve escalated to the Amazon Alexa Team and are working diligently with them to diagnose the root cause and restore service.
We will keep you posted as we work aggressively to remediate this issue ASAP.
Our apologies for any inconvenience this disruption in service may cause.
As always, should you have any questions, comments, or concerns, please email us at
support@getvera.com.
Sincerely,
Network Operations Team
Vera Control, Ltd. ▾ Smarter Home Control
www.getvera.com ▾
status@getvera.com ▾ +1 (866) 966-2272
Aug 4, 15:34 PDT