It is said that despite the problems we have to access content from the websites. World Web Security Services Company, Cloud Flair, conducted a test to confirm the engine company’s stealth behavior. Researchers highlighted that not only were crawler boats because of neglecting the instructions of the websites, but they were actively hiding their identity by several sources to ensure that website owners did not track this activity. Cloud Flair also managed to find a way to close artificial intelligence (AI) company’s efforts successfully.
Cloud Flair catchs the sticky strategy of anxiety in action
In a blog post, the web security platform has claimed that the hassle is involved in the “stealth creeping” activities. The post added, “We see permanent evidence that the hassle is repeatedly modifying its user agent and changing their source ASNS to hide their crawling activity, as well as ignores – or sometimes – the robot dotted files.”
Before finding anxiety behavior, it is important to understand how the whole system works. Owners of content websites add information, and third -party services such as search engines bring these data to indicate these websites and show them when they type relevant questions. Some apps and websites also scratch websites to either level them in their interface or collect data with permission.
However, this relationship must be confident of this relationship, to work between websites and crawls. When it was set up by these boats after a set of rules while crawling on any website. These rules order that the activity of the boats should be transparent, they should fulfill a clear purpose and only perform specific activity, and they should follow the website’s guidance and priorities. Therefore, if a website stops the boot, it should not ring its website.
According to Cloud Flair researchers, uncomfortable websites are also breaking the trust model using stealth tactics to scratch the website data from these websites that clearly use its bots, the disturbing boots and anxiety. Researchers managed to confirm this activity by creating new test domains.
These domains were not arranged by any search engine or publicly accessible or discovered. In addition, researchers implemented a robot dot TX file (used by websites to instruct web crawlers) to prevent access to any part of the website.
After that, Cloud Flair researchers went into trouble and asked specific questions about these newly created domains. They found that, despite following the Internet Protocol to prevent crawling activity, anxiety can still be at the level of detailed information about these websites.
Cloud Flair claims that Several takes several steps to ignore the guidelines of troubled user agents or web crawler websites and access to data. If a declared user agent is refused to access through robots dot TST, he ignores it and keeps the data scratch. If a website has implemented a Web application Firewall (WAF) to stop the boot, the company uses a common browser agent aimed at imitating Google Chrome or Macos.
It is said that this unannounced boot is also said to use several IPS to activate the website. To hide its tracks further, it is said that these crawlers use the number of different automatic systems. In particular, Cloud Flair said that when these unannounced boats were stopped successfully, the reaction of the problem diminished, as it began to rely on other data sources to answer the question.
Cloud Flair said its BOT management system has been able to register all unannounced crawling activity from hidden user agents of anxiety and is now automatically protecting all its boot management users. In addition, the company has included signature matches for a stealth crawler in its systematic principle, which prevents AI crawling activity. It is available to all cloud flair users, including free levels.
