If I turn on branch deploy subdomains (say for something like test.mysite.com), will this be crawled and indexed for search? If so, what is the preferred method for ensuring this doesn’t happen. I’ve not done much work with SEO, but I know I don’t want a subdomain used for testing to accidentally be showing up in searches. I’m using Gatsby to build my site.
Hi @muffintheman! I’d suggest using a pattern like the one described here:
…which shows how to do “something conditional per branch during build”. You could put a robots.txt in place ONLY on branch deploys, or set a password as mentioned there - both will be effective for SEO. You could also just not link those branches anywhere google would find them 
We do AUTOMATICALLY do this at deploy preview URL’s (how we build your PR’s) for you - we send an HTTP response header X-robots-tag: noindex to prevent THOSE from being indexed directly (these have URL’s like https://somelonghash--yoursitename.netlify.com).
How can I prevent my website on netlify subdomain from being indexed on google? To clarify, my website is being indexed with its Custom URL which is what i would want, but some pages on the netlify subdomain are also showing up in search results…something i definitely DO NOT want to happen
Can you advise please
The fix for that is a bit different, but we have instructions right in the domain settings section of our admin UI for every site that has a custom domain. You can find them (redirects, pre-filled in for your site) by clicking the small chevron shown in my screenshot from the top configuration card on that page:
1 Like
Yeah I added that into my redirects fie but it doesn’t appear to be working?
EDIT: hold the phone I didn’t notice the ! which i bet makes all the difference! lol
EDIT2: Yeah the ! was key to that rule working
1 Like
awesome! thanks for continuing to work on this!
Hey @fool,
That link is not present anymore in that section. Can you share it here? I can’t find it.
Thanks!
Hi, @patricio. Are you asking about the link in the screenshot? I don’t think that link exist anywhere anymore.
The rules themselves (in _redirects format) look like this:
http://example.netlify.app/* https://www.example.com/:splat 301!
https://example.netlify.app/* https://www.example.com/:splat 301!
There are two redirect rules (one for HTTP and one for HTTPS). They redirect any traffic for the site subdomain under netlify.app which directs all traffic to the custom domain (www.example.com in this example). This rule makes navigating the site under the netlify.app subdomain impossible (as only 301 redirects for the custom domain will be returned for it with this rule).
If this doesn’t work or if there are any questions, please let us know.
Thanks for the quick reply, @luke! That’s exactly what I was looking for.
BTW, I’m a bit shocked/confused that by default netlify doesn’t automatically redirect the .netlify.app subdomain to your main domain, nor prevents search engines from indexing your branch deploys. I’d expect a warning at least. Despite that, I love netlify 
Our browseable deploys (specific to a point in time, or HEAD of a branch like staging) are one of our best features in my opinion. Auto-redirecting would break those features. As our deploys are immutable - frozen at the point the build completes, and any build can be published at your production URL, doing so would break many folks’ production workflows as well.
This is what we have in place as far as automatically preventing SEO trouble:
Yeah, I agree. I didn’t mean autoredirect. That would decrease netlify’s utility.
That link you shared is very useful, thanks. I think the same should be done by default for branch deployments. The risk of hurting your SEO is about the same.
For branch deployments, you can conditionally apply any protections you like. You could do it for deploy previews as well using this method if you don’t like the canonical link way we’ve implemented.
Thanks for your answer! You and luke have been very helpful. Appreciate it.