“Last.fm’s robots.txt”
robots.txt is a file used by convention to tell web spiders (systems like those at Google or Microsoft that “crawl the web,” indexing files for their search engines) to ignore certain files. It seems that someone at Last.fm has a geeky sense of humor… User-Agent: * Disallow: /music? Disallow: /widgets/radio? Disallow: /show_ads.php Disallow: /affiliate/ Disallow: …