If I turn on branch deploy subdomains (say for something like test.mysite.com), will this be crawled and indexed for search? If so, what is the preferred method for ensuring this doesn’t happen. I’ve not done much work with SEO, but I know I don’t want a subdomain used for testing to accidentally be showing up in searches. I’m using Gatsby to build my site.
Hi @muffintheman! I’d suggest using a pattern like the one described here:
…which shows how to do “something conditional per branch during build”. You could put a robots.txt in place ONLY on branch deploys, or set a password as mentioned there - both will be effective for SEO. You could also just not link those branches anywhere google would find them
We do AUTOMATICALLY do this at deploy preview URL’s (how we build your PR’s) for you - we send an HTTP response header
X-robots-tag: noindex to prevent THOSE from being indexed directly (these have URL’s like https://somelonghash–yoursitename.netlify.com).
How can I prevent my website on netlify subdomain from being indexed on google? To clarify, my website is being indexed with its Custom URL which is what i would want, but some pages on the netlify subdomain are also showing up in search results…something i definitely DO NOT want to happen
Can you advise please
The fix for that is a bit different, but we have instructions right in the domain settings section of our admin UI for every site that has a custom domain. You can find them (redirects, pre-filled in for your site) by clicking the small chevron shown in my screenshot from the top configuration card on that page:
Yeah I added that into my redirects fie but it doesn’t appear to be working?
EDIT: hold the phone I didn’t notice the ! which i bet makes all the difference! lol
EDIT2: Yeah the ! was key to that rule working
awesome! thanks for continuing to work on this!