Summary of event: On October 19, doxy.me encountered an issue that prevented patients from checking in on doxy.me 2.0. This outage resulted in 8 hours and 28 minutes of unplanned downtime between 8:01 AM ET and 4:30 PM ET.
Business impacts: Patients were unable to complete the check-in process. After entering their name and clicking the Check-In button, they remained stuck on the check-in screen instead of joining their provider’s waiting room. While providers were able to log in during this incident and receive patient check-in notifications, patients didn’t appear in their waiting room queue. This prevented a provider from starting a video call with the patient.
Root cause: Security improvements implemented in our real-time messaging system and doxy.me 2.0 platform led to an issue that arose approximately 48 hours after the release. The continuous release cycle of doxy.me prevented the issue from appearing in our non-production environments used for pre-release testing.
How was the issue resolved: Engineering identified real-time messaging as the root cause of the outage and updated accordingly while maintaining the security enhancements implemented.
Preventative next steps: We’re enhancing our testing framework to include additional scenarios that will help identify unexpected real-time issues.