Good Bots, Bad Bots

Nearly half of all web traffic is from “bots.” That statistic alone should grab your attention, but the one that should worry you more is that “bad” bots are growing in number and reach. They’re also getting smarter.

So what are “bots?” A bot … “is an automated program that is programmed for certain actions and executes them either regularly or reactively. The bot does this without needing human activation. It analyzes the environment and ‘decides’ which actions to take depending on the situation.”

Bots are “good” and “bad.” Some good bots include:  

“Crawlers/Spiders (e.g. Googlebot, Yandex bot, Bingbot) – Used by search engines and online services to discover and index website content, making it easier for internet users to find it. 

“Traders (Bitcoin trading bots) – Used by Ecommerce businesses to act like agents on behalf of humans, interacting with external systems to accomplish a specific transaction, moving data from one platform to another. Based on the given pricing criteria, they search for the best deals and then automatically buy or sell. 

“Monitoring Bots (e.g. Pingdom, Keynote) – Monitor health system of the website, evaluate its accessibility, report on page load times & downtime duration, keeping it healthy and responsive. 

“Feedfetcher/Informational Bots (e.g. Pinterest bot, Twitter bot) – Collect information from different websites to keep the users or subscribers up-to-date on the news, events or blog articles. They cover different forms of content fetching, from updating weather conditions to censoring language in comments and chat rooms. 

“Chat Bots (e.g. Messenger, Slack, Xiaoice) – A service that enables interacting with a user via a chat interface regarding a number of things, ranging from functional to fun.” 

Some bad bots include:  

“Impersonators – Designed to mimic human behavior to bypass the security and by following offsite commands, steal or bring down the website. This category also includes propaganda bots, used by countries to manipulate public opinion. 

“Scrapers – Scrape and steal original content and relevant information. Often repost it on other websites. Scrapers can reverse-engineer pricing, product catalogues and business models or steal customers lists and email addresses for spam purposes. 

“Spammers – Post phishing links and low-quality promotional content to lure visitors away from the website and ultimately drive traffic to the spammer’s website. Often use malware or black hat SEO techniques that may lead to blacklisting the infected site. A specific type of spammer is auto-refresh bots, which generate fake traffic. 

“Click/Download Bots – intentionally interact or click on PPC and performance-based ads. Associated costs of such ads increase based on exposure to an ad – meaning the more people are reached, the more expensive they are.”

Smart Bots

Bots are everywhere working 24/7 behind the scenes, though they’re not completely stealth actors. In fact, they’re discoverable. At the same time, they’re getting smarter.  The marriage between bots, artificial intelligence (AI) and machine learning (ML) old news and is yielding smart children of all shapes and sizes. The good news? Smart bots are mostly good. But the bad news is that bad bots are getting really smart.