ACH Payment Processing
Incident Report for Autobooks
Postmortem

Summary: 

On April 6th, 2021 from 12:16 ET to 14:05ET the Autobooks platform experienced an outage with a third-party ACH provider that prevented Administrators and SMB customers from submitting ACH payments during this window. 

Details: 

Autobooks utilizes many premier cloud partners as part of providing our SaaS platform, including partners that provide capabilities like ACH (Automated Clearing House) that we then provide to our customers. 

At 12:16 ET on April 6th our automated monitoring began to alert the team to potential issues affecting ACH payments via the Autobooks platform.  Our team quickly determined that the issue was related to failing calls to a third-party ACH provider. 

At 12:26 ET, our team was able to confirm this issue with our ACH provider directly and worked to monitor the situation while this third-party provider worked to resolve the issue. 

At 14:00 ET, our team determined that ACH payments were again starting to succeed in the Autobooks platform. Continued testing reported no further issues our end and we ultimately gave the all-clear at 14:05 ET.  

Our team has used data collected as part of this incident to identify a number of improvements that will minimize the impact of similar issues going forward and will work to prioritize these improvements as part of our standard development cycle.  These improvements include technology to queue up requests in the event of a similar situation in the future, as well as tools to put ACH functionality in “maintenance mode” to gracefully signal to SMB customers that this feature is currently unavailable in the event of an extended outage or maintenance scenario.   

We apologize for any inconvenience this may have caused. Your business is very important to us, and we strive to provide you with exceptional service.

Posted Apr 20, 2021 - 14:02 EDT

Resolved
This incident has now been resolved. Our teams have completed monitoring and have confirmed the last ACH payment error occurred at 2:05 PM EST.
Posted Apr 06, 2021 - 15:17 EDT
Monitoring
A fix has been implemented by our ACH provider. We will continue to actively monitor our logs to ensure no further errors occur.
Posted Apr 06, 2021 - 14:46 EDT
Update
As our ACH provider continues to make progress on resolving this incident, we are beginning to see some successful ACH payments post within the Autobooks application. Our next update to this incident will be posted by 3:00 PM EST or sooner.
Posted Apr 06, 2021 - 13:53 EDT
Identified
We have identified an issue with our ACH provider which is resulting in payment errors in the Autobooks application. As a result, users may receive an error when attempting to submit an ACH payment at this time. We are actively working with our ACH provider until this incident is resolved.
Posted Apr 06, 2021 - 12:48 EDT
This incident affected: Autobooks Platform (Web App).