############################### # # sample robots.txt file for this website # # addresses all robots by using wild card * # User-agent: * # list folders robots are not allowed to index Disallow: /oldpage/ Disallow: /pdfs/ Disallow: /helpfiles/ Disallow: /errors/ Disallow: /controls/admin/ Disallow: /controls/ftb/ Disallow: /controls/solpart/ Disallow: /aspnet_client/ Disallow: /templates/ Disallow: /SiteMapdotNet/ Disallow: /bin/ # # list specific files robots are not allowed to index # User-agent: Googlebot Disallow: /ProductList.html Disallow: /links.html #Disallow: /*tabid=19&id= # # End of robots.txt file # ###############################