Inquiry: Why is my custom domain going up and down with periodic "page not found" error?

Question

Why is my custom domain showing page not found?

Weird Behavior

Sometimes I can see my custom domain up on my browser. However, when I try to access the my custom domain url via my phone, it is not showing it is up. Weirder yet, when it shows up on my browser, it is down for friends checking on my behalf states away. Other times they can see it, but I can’t.

What puzzles me is that it is up for my https://heathyates.netlify.app/ but only sometimes maybe working for www.heathyates.com

Background

I recently wrote a blogsite using blogdown and deployed from git to netlify. I purchased my custom domain with namecheap and set up custom DNS server names as specified with the netlify. The domain https://heathyates.netlify.app/ is working. At first, my custom domain www.heathyates.com was working, but now is giving me “Page Not Found Error” depending on when I check it. Here is the error:

I have verified that www.heathyates.com is a custom domain with netlify. I am getting the nice green icons telling me things look good there. In addition to the above, I have verified that the proper custom DNS addresses are configured with namecheap. The HTTPS is enabled and things look OK there.

Does anyone have any ideas on what could be causing this behaviour? Update: This issue has not abated after 24 hours.

Update: Potential Similar Issue and Dig Utility

This may be a similar issue, but I do not know how to troubleshoot in my case. See: New Namecheap Domain Not Working - #4 by kmanderson12

However, I am at a loss at how to troubleshoot this still? As far as I know, everything seems to be configured on my side, but I cannot get it routed correctly.

I ran the dig utility and it is showing a status of no error. It shows this even when I cannot manually navigate to the page. I have exhausted all ideas and would appreciate a response from netlify folks in the know.

Update: Last Attempt

I have deleted the website with netlify and redid everything: new website, configure with git using default settings (relying on netlify.toml for environmental variables on my repo), configured the custom domain with netlify. If that doesn’t work, I may try to go back to namecheap for DNS as well.

1 Like

hey there, i just tried to pull up your site and i can’t see it - i am not sure where you are in your debugging steps, but we definitely want to help you get your site live!

can you give us an update of where you are along this winding road?

Interesting, it works on my side of the world (New Zealand):

Heya @heathyates! I see a very interesting tale spread across our CDN of 16 different deploys currently being served. I guess you had some number of sites you created, applied hostname to, deployed, and removed after deploying - all at that hostname, which is a potential cause of such problems, though you win as far as “worst effects of this ever seen”. Could you give us a bit more detail about what series of steps you’ve taken with your site? Did you recreate it a couple times? 16 times? did you deploy at all between deletion/recreation? This information will help me file a better bug since this is clearly

I have asked our team to help clear this up since once a site is removed, I (and you) lose the ability to try to purge it from our cache directly (in your case with a new deploy; in my case with some commands issued via our API). They’ll be back online tomorrow and will do the needful then and I will follow up here to confirm.

@fool @perry First, thank you for your responses. Let me update you on things I have done.

Brief Recap

  • AFAIK, I only created the website 2 times. I deleted the first website. I did not attempt 16 times. I am not sure why the logs show 16 attempts?

  • I have used the command dig @DSN3.P03.NSONE.NET and I often get different IP addresses. Is this a sign of load balancer activity? My colleague in Kevin also has confirmed this.

  • I can always access the website heathyates.netlify.app or Heath Yates - Data Science Muses. However, the issue is always accessing www.heathyates.com

  • What did seem to not work very well both times as the the part of getting HTTPS set up. I am not sure if this is a red herring, but this part kept failing:

This is the part that was acting very strange during my deploy.

  • I think another thing to look at is whatever happened with establishing the Netlify DNS. That was acting totally wonky as well. It’s hard for me to debug what occurred there, but I went through the necessary steps both times. I attempted to get the Netlify DNS up only a third time, but I cancelled. Oddly enough, things went through on my HTTPS and the Netlify DNS during that attempt. This could be a red herring, but not sure.

  • My logs looked good both times I tried to upload from git, however, you can find my netlify.toml here.

  • Last but not least here are some screenshots @fool that I think collaborate what you are seeing on your side by my colleague Kevin who has jumped in to see what IP are on his side when it is up or down:

Status Code: 404
IP Address: 206.189.73.52:443

However, you can see it up here:

Status Code: 200
IP Address: 104.248.78.23:443

  • I made a few changes to my git repo before I deleted the website and started over. I wonder if the deployment from the git repo caused the recreate the 16 times? I doubt this and maybe a red herring, but just offer it in the remote chance.

  • Since I deleted the website. I set up the custom DNS nameservers with namecheap twice.

Conclusion

I did several things, but it boils down to that I tried to get it working 2 times by deleting once and redoing everything. There may be some rabbit holes and false leads, but I tried to mention everything I did and tried during the process.

I am sorry if the above is not useful to you, but I cannot see any details beyond the log when I tried to upload my github. There were no errors observed.

Please let me know what the next steps you suggest are. Thanks!

Thanks, this isn’t about DNS - just about how many times you deployed your site (I was also interested in how many times you created it; thanks for confirming it wasn’t 16 :)). I guess you must have deployed at least 15 times on the since-deleted site since I see all those deploy ID’s and only one deploy on the new site - can you confirm that seems in line with your commit history?

Also of interest: did you happen to use any of these features on the old site? I can see none are configured on your current site, but maybe were on the old one…As you may guess from these questions, deleting the site deletes all our records about it, so I have to ask:

  1. split testing
  2. proxy redirects or rewrites
  3. functions

? Thanks in advance for your help in investigating!

Our team is still working to repair, will hopefully have it cleared up in a couple hours (the cache invalidation process takes some time as we have to proceed through the CDN sequentially rather than in parallel to avoid high load on our backing store).

@fool

Thanks for the clarification. I created the website two times. As for deploying, I interpret this to mean how many times I made changes to my website in git and those persisted to netlify through continuous deployment? Here is my git commit history here. On Sunday, I committed 3 times and on Saturday I committed 1 times.

You asked me about split testing, proxy redirect/rewrites, or functions. I did not do any of those things. Sorry.

The deployment thing got me thinking. On Saturday, after I created my website I noticed it was not working correctly. So I created a netlify.toml file here. I then commited the change to github on Saturday. I did not have the baseurl configured either at first either if I recall right. What this means is hugo -b $DEPLOY_PRIME_URL --buildFuture was not correctly configured. That is, DEPLOY_PRIME_URL was null. On Sunday I fixed this by configuring the baseurl in the first git commit, but the website was still messed up so I eventually tried my infamous 2nd website rebuilt attempt.

Just to make sure. After the repair, I should be good to make website changes in git and have those persisted through in deployment? Split testing is cool and I put in some code in my netlify.toml on Saturday to put myself in the position to try it eventually, but thankfully was not brave enough to try it on top of everything else that was going on. Ha ha ha. :slight_smile:

I am thinking if the netlify.toml issues were at play, then I apologize for my share of the blame for the issue. Thanks for your efforts.

Update
If the improperly configured baseurl caused all this, is there any way I could force refresh/deploy so that all servers get a version with a good base url?

Got that cleared up for you - please let me know if you still see misbehavior!

Thank you for confirming that you didn’t use these features - that information makes things seem very odd, but our team is smart and will be able to work with that data.

You definitely do not need to change anything about git history, to ensure proper operation going forward!

Since all deploys are atomic, you should only need to have the latest deploy published to remove any traces of the old one. This was a bug that let old deploys “stay live” beyond their lifespan and is extremely rare; it’s quite unlikely you’ll run into it again, but please let me know if they do!

PS: Nothing you did was wrong! All bug, no user error!

Hey, sorry for jumping on the thread but we’re facing the exact same issue - I was very happy when I found this thread whilst searching on the matter :sweat_smile:

I’ve been using Netlify for a while now with custom domains, but for some reason on a new project I’m constantly seeing the “Page not found” screen. I can then open it in incognito (or force clear cache + hard reload) and it’ll work, but other times it doesn’t.

Then when I complain to my colleague and ask him to check, he says it works, I re-check, and the site is there… But it is going down randomly.

The main Netlify domain for this project always works but we have the custom domain posted anywhere we’ve mentioned the project, so that’s no use to us.

Any ideas?

Hey @breadadams,

Same as above – we’ve cleared it up; can you confirm whether this is till occurring? Thanks!

1 Like

Yep, I haven’t seen it go down since - thanks a bunch! :raised_hands:

1 Like

Sorry for bumping this thread but I’m facing the exact same issue. My Netlify domain works wonders, but my custom domain often goes down for no reason. Any way to fix this?

Hi, @icarofr, that will depend on what the root cause of the issue is. To determine that, we’ll need more information to research this.

I would encourage you do collect this data locally as we might not see what you are seeing, even if we use a VPN to simulate our request from the same geographic region.

First, it is a DNS issue or a HTTP issue?

If the issue is DNS

If the issue is DNS, the following details would be helpful:

  • what is the custom domain name being used?
  • what DNS resolver is being queried?
  • what are the results of the DNS query?

If the issue is HTTP

For this, the x-nf-request-id HTTP response header (or the information it replaces) will be most helpful.

There more information about this header here:

If that header isn’t available for any reason, please send the information it replaces (or as many of these details as possible). Those details are:

  • the complete URL requested
  • the IP address for the system making the request
  • the IP address for the CDN node that responded
  • the day of the request
  • the time of the request
  • the timezone the time is in

To summarize, we will be happy to research this issue but we require more information in order to do so.

Is it possible to send these informations par private message or mail?

Hey @icarofr,

I’ve searched and I can see that your custom domain was issued at 5:28PM BST yesterday. Therefore, I am very confident that the issues you are exhibiting are due to DNS propagation after a very recent setup.

If the issues persist in 24-48 hours then feel free to create a topic in the Admin section of Community.