Do you remember when phonebooks were delivered to your doorstep? I do. Every time one arrived, we'd pull it inside, leaf through it for a minute or two, and put it in the cabinet under the fish tank. When we needed something, whether it was a plumber or a good photographer, we'd pull out the phone book.
That's probably my last clear memory of phonebooks. The internet was young back then, but it's certainly blossomed now. No one is looking in the phonebook for plumbers or photographs or your association. They're looking online.
To survive in the modern world, your association needs a website that makes it easy for people to find you. We've become comfortable with that concept. Now, as the faces behind our organizations, we even think we understand our website visitors and target audience.
But what if we don't? What if it's not just people that are finding our websites? What if bots are also finding us and impacting our website traffic and other key metrics?
Your website could, at this very moment, be crawling with virtual spiders and bots that are viewing your website, links, and articles. They may even be stealing your valuable content out from under you.
That's certainly a problem we never had with phonebooks.
Just like the necessity of websites, however, bots are inevitable on the internet and you can't just ignore them. Instead, the best way to deal with bots is to understand what they are and what their purpose is. You can use that knowledge to take informed, actionable steps to protect your association's website from the negative effects some bots cause.
The term "bot" is a generalized reference to automation software. Any software that does a task for you is a bot. You can order flowers on 1-800-Flowers just by texting a bot and answering a few questions, for example. Simple, easy and efficient, the ordering process is now done by software instead of a human.
That doesn't mean that all bots focus on sending flowers to loved ones. Another type of bot, a website bot, is a software program that roams the internet looking at websites. Google has a veritable army of bots that crawl your website and determine what your SEO rank is, for instance.
Search engine bots like those Google uses are considered good bots. They have a specific task that they were sent out to do, and they're usually just looking for information. Search engine bots can even help your association by giving you a good SEO rating.
In addition to search engine bots, your website also has to contend with what are commonly known as bad bots. Bad bots scour the web in order to steal content or spam websites. We've all seen the results of these bots. Just think back to that random, unrelated blog comment suggesting you "Make $1000 today!" simply by following the link. Bad bots are often responsible for those types of comments.
Yes. According to Incapsula, in 2015 almost half of all website traffic was due to bots. That means the likelihood that bots are crawling your site is extremely high. There are bots on your website, and they are a mixture of good bots and bad bots.
Unfortunately, the bad bots outnumber the good. Of all your website visitors (humans included) only 19.5% are good bots. 29% are bad bots that are sowing discord or combing through your information for something to pilfer.
In other words, nearly a third of your website traffic isn't there because of an interest in your association or to boost your SEO, it's there trying to do your website harm.
While bad bots are more concerning in terms of your internet security, both good and bad bots affect your key metrics.
Your reports won't show you the difference between humans and bots, so when bots visit your association's website, they show up as page views. Even if they're just SEO bots seeking information, your page views, normally thought of as human visits, will be artificially increased.
That means that you're not reaching as many people as you thought. If your website is in line with the statistics, you can attribute nearly half of your visits to bots. If those bot visitors spend hours on your site going through pages they may even increase the demand on your site, causing pages requested by human visitors to take longer to load.
Bots don't just crawl your website and ratchet up your page views, they can also click on your calls-to-action (CTAs). If bots do click on your offers when they visit, they'll exaggerate your conversion rate. If half the clicks on your offer are bots, for instance, then your offer may not be nearly as effective as you thought because it's not appealing to humans—just computers.
The opposite problem can occur if bots don't click on your CTAs. These bots will still show up as page views, but since they're not clicking, they make your conversion rate appear lower than it actually is.
Online advertisements on websites, social networks, and search engines can all be affected by bots. When bots visit your ads and click on paid advertising, they make it look like your promotional campaigns are getting a lot of attention. The attention might seem nice at first, but it won't result in new members. Those extra bot clicks will only drive up your costs.
If you're using content to showcase your association's expertise and value (and you should be) then that's another area where bots can do some harm. Bad bots can steal your content, publish it somewhere else, and damage your exclusivity.
When this happens, the bot will typically steal your content, be it a blog, video, or other offer, and place it on a different website that's full of ads. Your content is now freely available, decreasing the value of your association and frustrating members who think they're getting exclusive information.
Bots are a fact of life in a digital world. They're not going away and you cannot ignore them. The best way to handle the situation is to adjust how you manage and analyze your online presence, taking the effects of bots into account.
Reporting and analyses are good places to start. To be effective in evaluating your association's online performance, include the likely effects of bots. Based on current information about how active bots are online (often reported by organizations like Incapsula and Distil Networks) you can adjust your numbers.
If you had 10,000 visitors last month, you might estimate that only about 5,000 of them were humans, for example. Similar adjustments could be made for your conversion rates. The best membership management software may even make these adjustments for you. Our software platform has algorithms that do this automatically, so you know that your page visits really are from people interested in your association.
Whenever possible, take steps to protect your association from the negative effects of bots. Put your most valuable content behind a login so that bots cannot easily access it, and lock users out of your system after a certain number of failed login attempts.
The best membership management and online community software will use this method to keep out bots, allowing members who really have forgotten their passwords to reset them through email. There will be no inconvenience to you or your members, but without a valid account, bots will be denied access to your content. By using this technique, you also get the added advantages of making content an exclusive member benefit.
If putting your most valuable resources behind a login isn't enough, there is technology that will block bots without negatively affecting your SEO from search engine bots. Often called Bot Protection Software or Anti-Bot Security Software, you can research and find a program that works for your association.
Many times, your membership management or online community software will also help protect you against the damaging effects of bots. In addition to secure logins, secure software systems such as MemberCloud use CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart) to ensure that users are human before they post comments or log in. You've seen CAPTCHAs before, and likely know them as the little box with random numbers and letters that you need to verify before submitting a comment. These help keep spam off your sites, and bots out of secure areas.
How much of your website traffic is made up of bots will likely vary over time, but on the internet bots are a fact of life. The chances that they'll be eliminated any time in the foreseeable future is unlikely.
Since they're here to stay, you need to understand bots so you can accurately evaluate your association's performance and protect yourself from negative effects. Protecting your members and potential members from the frustration bots and duplicate content can cause may be one of the most important aspects of this. By protecting your content and retaining exclusivity, you'll help members feel secure and be able to provide them with a better overall experience.