Call Data Processing Delay
Incident Report for CallRail
Postmortem

A change on September 7th, 2021 to our application caused cache load for one service to begin increasing. It reached its limit on September 15, 2021, resulting in an impact to CallRail and Lead Center web interface functionality. This did not impact Call Tracking services beyond a brief delay in processing.

In response, our team increased cache load capacity for the affected application. This restored full functionality in the web interface and remediated the event processing delay. We continue to monitor the affected application. We are also implementing additional policies and monitoring around cache load management.

Posted Sep 24, 2021 - 11:43 EDT

Resolved
This incident has been resolved.
Posted Sep 15, 2021 - 16:52 EDT
Update
We are investigating the cause of a spike in load on our backend services, but we have also provisioned additional cloud resources to provide an additional safety margin.
Posted Sep 15, 2021 - 13:31 EDT
Monitoring
We are monitoring an issue related to our event processing. Call routing for both CallRail and LeadCenter remain un-impacted, although there may be a slight delay in processing some call event data
Posted Sep 15, 2021 - 11:52 EDT
This incident affected: Call Tracking (CallRail Web Application), Lead Center (Lead Center General), and Call Recording.