Crawler

Crawler-based search engines have three major elements. The first is the spider, also called the crawler, which visits a web page, reads it, and then follows links to other pages within the site.

These serve as an algorithmic way to judge the “mobileness” of a site, and similar algorithms are likely to be used by the search crawlers.