What is the best way to implement rate limiting at the API gateway level to stop automated bots from scraping my cloud-hosted historical society archive?
Download run
Fan-out diagram
The searches the AI actually ran for this prompt.
SEED QUERY
What is the best way to implement rate limiting at the API gateway level to stop automated bots from scraping my cloud-hosted historical society archive?
1 real search · 1 real-search pass
OWASP API Security rate limiting automated scraping API gateway guidance AWS WAF rate-based rules Cloudflare rate limitingGoogle ↗
OBSERVEDInstruction
Badge and edge weight = share of passes that produced the query. Hover a query for its intent and routing format.
Query classes
Google's semantic query class for every sub-query: what kind of answer each search is after, simulated vs real.
ChatGPT real searches (1)
Classes come from the query-classifier (a local Laya model). 1 of 1 calls are low-confidence (under 50%) — hover a query in the table to see its confidence.
Searches and sources
ChatGPT1 real searches · gpt-6
OWASP API Security rate limiting automated scraping API gateway guidance AWS WAF rate-based rules Cloudflare rate limiting Google ↗
- Rate limiting best practices · developers.cloudflare.com
- Rate limiting rules · developers.cloudflare.com
- Applying rate limiting to requests in AWS WAF - AWS WAF, AWS Firewall Manager, AWS Shield Advanced, and AWS Shield network security director · docs.aws.amazon.com
- Runtime Application Protection: WAF, WAAP, and RASP · owasp.github.io
- Bot Management and Anti-Automation · cheatsheetseries.owasp.org
- REL05-BP02 Throttle requests · docs.aws.amazon.com
- Options for rate limiting in rate-based rules and targeted Bot Control rules - AWS WAF, AWS Firewall Manager, AWS Shield Advanced, and AWS Shield network security director · docs.aws.amazon.com
- Choosing and configuring Bot Control for your use case - AWS WAF, AWS Firewall Manager, AWS Shield Advanced, and AWS Shield network security director · docs.aws.amazon.com