Scheduling and End-User Transactional Pages

Slow response times

Resolved

Between UTC 11:04-11:30, customers would have experienced a slowdown in booking; a few would experience errors during windows.

This was due to bad bots scraping and probing our tenant websites at a rapid pace, causing our servers to overload. We had safeguards in development that were intended to block this very thing but to prevent false positives (and blocking legitimate traffic) the mechanisms were not set to enforce, only record what would be blocked. The rules were ready for production but, in typical bad luck, a wave of scrapes flooded servers before roll-out.

The ruleset has been rolled out, which promptly blocked the bad bots. Thank you for your patience!

Monitoring

The platform was slowed down due to an unknown attacker performing a DoS on our servers. We have blocked the bots and typical service should resume.

Investigating

We are aware of very slow booker response times (and a few errors as well). Engineering is investigating the issue.