Inside Good Bots: Why Management of Benign Traffic is Crucial

Bad bots have often been in the headlines at Web security blogs and forums, as well as in mainstream media outlets in recent years, due to the multitude of ways in which they negatively impact online businesses and other organizations. Malicious bots have steadily evolved to become very sophisticated at evading conventional Web security measures and passing off as humans. They constitute a clear and present danger to websites, applications, and APIs, hence they get the lion’s share of attention from webmasters and security experts.

As bot management specialists serving a wide spectrum of organizations, we know that even good bots can, under certain circumstances, cause various degrees of harm. Such as, for example, when good bot traffic escalates and starts approaching or even exceeding the limits of server capacity and available bandwidth — or when they produce a high degree of skewed analytics, to mention another outcome. This is why we recommend a holistic, industry-specific approach to manage every type of bot traffic so that webmasters and security specialists can take the most appropriate measures to deal with them.

Our research report ‘Inside Good Bots‘ details the most widespread types of good bots across key industries, the functions they carry out, and recommendations to best manage them according to your specific organizational objectives. The report also graphically breaks down our data in three ways:

• Bad Bots vs. Good Bots vs. Humans
• Search Engine Crawlers vs. All Other Good Bots
• Breakdown of Good Bot Traffic (OTA, E-Commerce, Classifieds, & Media & Publishing)

We’ve observed that businesses in the Classifieds & Marketplaces industry get the highest level of Aggregator bot traffic, with Online Travel Agencies (OTAs) following in second place. E-Commerce sites and OTAs are seen to have the highest levels of Partner bot traffic. E-Commerce also attracts the greatest numbers of Social Network bots, followed by Media & Publishing sites, while OTAs come in last with a tiny 0.1% of their good bot traffic consisting of this kind of bot.

Takeaways From ‘Inside Good Bots’

It’s becoming increasingly crucial to manage both good and bad bots depending on organizational objectives. Dedicated bot management solutions are no longer an optional component for Web and app security. We are increasingly seeing the adoption of bot managers by forward-thinking organizations to augment their conventional security systems. In summary, here are the key takeaways for readers of our report:

• How good bots can, in certain circumstances, have a negative impact.
• Webmasters and security experts are well-advised to implement industry-specific approaches to deal with every type of bot traffic.
• Analysis of trends in good bot traffic can provide marketers with deep insights to refine their strategies.
• A dedicated Bot Manager is an indispensable part of a modern web and app security system, helping mitigate a wide range of threats that affect organizations of every kind.

Download ‘Inside Good Bots’ to read the report in its entirety. As always, we welcome your questions, comments, and feedback at info@shieldsquare.com.

Originally published at https://www.shieldsquare.com on February 11, 2019.