# All robots will spider the domain User-agent: * Disallow: # Disallow directory /oldSite/ User-agent: * Disallow: /oldSite/ # Disallow directory /aspnet_client/ User-agent: * Disallow: /aspnet_client/ # Disallow directory /pageContent/ User-agent: * Disallow: /pageContent/