Scheduling and End-User Transactional Pages

Please be aware that a DOS is only intended to overload the servers and not breach data in any way. These are a fact of life on the internet. All zingfit customer data are secured with industry best-practices.

An update to this morning’s incident:

Background

Last week an unknown actor was DoSing us with more requests per minute than all other studios combined. Our WAF ruleset that would have blocked them in the first place was rolled out and monitored. Unfortunately, the ruleset was conflicting with users dependent on our API had been blacklisted because there was so much traffic coming from the one IP. Engineering removed the ruleset for now in fear of another false positives hitting a studio; These attacks are typically a one-off event: It was deemed unlikely to occur again until the ruleset was put in place.

Incident

At UTC 1100, the same attacker issued a similar load to our services. As our ruleset was temporarily disabled as explained above, the attacker continued unimpeded. Engineering was alerted to the incident and blacklisted the IP range permanently.

What could have been done better

Even though our hand was forced to temporarily disable this particular rule in the firewall, prudence would dictate manually adding the attacker’s IP range into a manual blacklist.

Fixing it for the future

Configuring web application firewalls to halt bad guys but do not impede the good is difficult to get right. As this incident occurred a second from what appears to be the same actor (same time, same day-of-week, same IP range), the range of IPs associated have been blacklisted indefinitely. Configuring our firewall to identify and proactively block this sort of behavior is now a top priority. We do apologize for the inconvenience.

Our ruleset for identifying and blocking bad actors on the internet needed to be tightened a little. As our ruleset becomes more robust we’ll all enjoy quicker identification and blocking of tricky bots.