Bad Bot MitigationΒΆ

XtendWeb includes the following nginx module

testcookie_access

If the module is installed and the settings below in XtendWeb >> Server Settings section is activated

test_cookie

The server accepts connections only from a user agent that support cookies or prevents bots from accessing the domain

To prevent good bots like google,yahoo etc from being denied ,the server whitelists a set of UserAgents for bypassing the cookie challenge test which by default is one below

adsbot-google|googlebot|mediapartners-google|teoma|slurp|bingbot|bingpreview|duckduckgo|yandex|msnbot|aolbuild|baidu|facebot|ia_archiver|wget|curl

A server administrator can add/remove any other UserAgents to this list to fine tune test_cookie setting