Lead Center and Dashboard Outage
Incident Report for CallRail
Postmortem

Yesterday (Tuesday, 9/7/2021) we made a routine change to our application database that had the unintended side effect of impacting database connectivity for several of our services. The total downtime was 20 minutes. This incident impacted functionality around:

  • The CallRail and Lead Center web interfaces
  • Call Routing for Lead Center

This did not impact Call Tracking or Dynamic Number Insertion services, although there may have been a small delay in processing that information.

Our team's immediate response was to enable a brief maintenance window for the application and safely remediate the database connectivity issue. We are monitoring all product functionality and are implementing policies around moving database-related changes to take place outside of business hours in order to mitigate such events in the future.

Posted Sep 08, 2021 - 12:27 EDT

Resolved
This incident has been resolved.
Posted Sep 07, 2021 - 15:30 EDT
Monitoring
A fix for this issue has been implemented and we are monitoring the results. The application is up and available.
Posted Sep 07, 2021 - 12:17 EDT
Investigating
We are currently investigating degraded performance in multiple applications impacting web, mobile, and Lead Center call routing functionality.
Posted Sep 07, 2021 - 12:07 EDT
This incident affected: Application Dashboard and Lead Center.