Elevated error rate from call routing systems
Incident Report for CallRail
Resolved
This incident has been resolved.
Posted Jun 13, 2023 - 18:44 EDT
Update
Errors are clearing and calls are routing normally again. Calls that were routed during the incident may continue to show as "in progress" for up to two hours after they have completed. Recording processing and transcriptions for those calls will be delayed. Free trial sign-ups are also operational again. We will continue to monitor.
Posted Jun 13, 2023 - 17:22 EDT
Update
We are temporarily routing all calls using our backup routing system. Calls will route successfully, however an increased number of calls will temporarily show as "in progress" in the CallRail dashboard after they are complete. Processing of those recordings and transcriptions will also be delayed. Lead Center call queues are failing to route calls during this mitigating action.
Posted Jun 13, 2023 - 17:04 EDT
Update
We are continuing to monitor. We are seeing an improvement in error rates but some calls are still not successfully routing.
Posted Jun 13, 2023 - 16:45 EDT
Update
We are continuing to monitor. New CallRail trial sign ups are also effected.
Posted Jun 13, 2023 - 16:24 EDT
Update
We continue to experience elevated error rates related to this issue at our provider.
Posted Jun 13, 2023 - 16:19 EDT
Monitoring
We've identified an issue at a hosting provider, however our backup call routing system is correctly routing calls. The hosting provider has identified and deployed a fix. A small number of calls that occurred during this period may continue to show as "in progress" temporarily. We will continue to monitor.
Posted Jun 13, 2023 - 15:37 EDT
Investigating
We are currently investigating elevated error rates from our call routing system.
Posted Jun 13, 2023 - 15:12 EDT
This incident affected: Call Tracking (CallRail Web Application, Call Routing) and Lead Center (Lead Center General, Lead Center Web Application, Softphone, Lead Center Mobile App).