Redirect running despite shadowed file existing in deploying

Support
,
1

I have a problem that even though the file exists on disk. Netlify is still routing to my serverless function

url: https://zenunu.netlify.com/

netlify.toml

# More info on configuring this file: https://www.netlify.com/docs/netlify-toml-reference/
# Global configuration
[build]
  publish = "dist"
  command = "yarn now-build"
  functions = "serverless-dist/functions"
  ignore = "git diff --quiet HEAD^ HEAD ./"

# Development environment
[build.dev]
  port = 4200 # Port that the dev server will be listening on
  publish = "dist" # Folder with the static content for _redirect file
  command = "yarn start" # Command to start your dev server (npm scripts)

# Post processing files
[build.processing]
  skip_processing = false
[build.processing.js]
  bundle = false # Breaks Auth0
  minify = true
[build.processing.css]
  bundle = true
  minify = true
[build.processing.html]
  pretty_urls = true
[build.processing.images]
  compress = true

## Uncomment to use this redirect for APIs to netlify functions.
## Not needed for static site generators.
[[redirects]]
  status = 200
  force = true
  from = "/api/*"
  to = "/.netlify/functions/api/:splat"

## Uncomment for redirect to fastboot
[[redirects]]
  status = 200
  force = false
  from = "/*"
  to = "/.netlify/functions/fastboot/:splat"

## Uncomment to use this redirect for Single Page Applications like create-react-app.
## Not needed for static site generators.
[[redirects]]
  status = 200
  force = false
  from = "/*"
  to = "/index.html"

## Cache Control Headers
[[headers]]
  for = "/api/*"
  [headers.values]
    Cache-Control = "public, max-age=300, s-maxage=600, stale-while-revalidate"

[[headers]]
  for = "/*"
  [headers.values]
    Cache-Control = "public, max-age=86400, s-maxage=86400, stale-while-revalidate"
2

Hi, @devotox. We’re happy to troubleshoot why this is happening. To do this, we’ll need to identify an HTTP request which was routed to the Function when the file was present.

The more direct way to do this is to send us the x-nf-request-id header which we include with every HTTP response.

There more information about this header here:

If that header isn’t available for any reason, please send the information it replaces (or as many of these details as possible). Those details are:

  • the complete URL requested
  • the IP address for the system making the request
  • the IP address for the CDN node that responded
  • the day of the request
  • the time of the request
  • the timezone the time is in

Would you please send us this header or the other details and we’ll be happy to research this further?

3

@luke Here is a request dump as you can see It is attempting to get a file from the assets directory through the function while this should be available on disk

{
    "level": "info",
    "time": 1581686334283,
    "pid": 8,
    "hostname": "169.254.0.81",
    "message": "Fastboot Visit",
    "hashedKey": "e7fad4f83ae6d9b9cdfda9e76201847825a54bdd",
    "cacheHit": false,
    "request": {
        "url": "/assets/zenunu-c8eb44e8c48167ea6d4f84a8e55c9c7f.js",
        "host": "zenunu.netlify.com",
        "port": 80,
        "path": "/assets/zenunu-c8eb44e8c48167ea6d4f84a8e55c9c7f.js",
        "method": "GET",
        "protocol": "https",
        "location": "https://zenunu.netlify.com/assets/zenunu-c8eb44e8c48167ea6d4f84a8e55c9c7f.js",
        "headers": {
            "accept": "*/*",
            "accept-encoding": "br, gzip",
            "accept-language": "en-US,en;q=0.9",
            "client-ip": "90.255.3.162",
            "connection": "keep-alive",
            "cookie": "ember_simple_auth-session=%7B%22authenticated%22%3A%7B%7D%7D; ember_simple_auth-session-expiration_time=86400",
            "referer": "https://zenunu.netlify.com/",
            "sec-fetch-mode": "no-cors",
            "sec-fetch-site": "same-origin",
            "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36",
            "via": "https/2 Netlify[e10e78c7-0174-4ddc-ada3-8bae7f8a7f83] (ApacheTrafficServer/7.1.8)",
            "x-bb-ab": "0.226852",
            "x-bb-client-request-uuid": "e10e78c7-0174-4ddc-ada3-8bae7f8a7f83-6676795",
            "x-bb-ip": "90.255.3.162",
            "x-bb-loop": "1",
            "x-cdn-domain": "www.bitballoon.com",
            "x-country": "GB",
            "x-datadog-parent-id": "5781150176488674814",
            "x-datadog-sampling-priority": "0",
            "x-datadog-trace-id": "8327508323675804903",
            "x-forwarded-for": "90.255.3.162",
            "x-forwarded-proto": "https",
            "x-language": "en,en;q=0.9",
            "x-nf-client-connection-ip": "90.255.3.162",
            "x-apigateway-event": "%7B%22path%22%3A%22%2Fassets%2Fzenunu-c8eb44e8c48167ea6d4f84a8e55c9c7f.js%22%2C%22httpMethod%22%3A%22GET%22%2C%22headers%22%3A%7B%22accept%22%3A%22*%2F*%22%2C%22accept-encoding%22%3A%22br%2C%20gzip%22%2C%22accept-language%22%3A%22en-US%2Cen%3Bq%3D0.9%22%2C%22client-ip%22%3A%2290.255.3.162%22%2C%22connection%22%3A%22keep-alive%22%2C%22cookie%22%3A%22ember_simple_auth-session%3D%257B%2522authenticated%2522%253A%257B%257D%257D%3B%20ember_simple_auth-session-expiration_time%3D86400%22%2C%22referer%22%3A%22https%3A%2F%2Fzenunu.netlify.com%2F%22%2C%22sec-fetch-mode%22%3A%22no-cors%22%2C%22sec-fetch-site%22%3A%22same-origin%22%2C%22user-agent%22%3A%22Mozilla%2F5.0%20(Macintosh%3B%20Intel%20Mac%20OS%20X%2010_15_0)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F79.0.3945.130%20Safari%2F537.36%22%2C%22via%22%3A%22https%2F2%20Netlify%5Be10e78c7-0174-4ddc-ada3-8bae7f8a7f83%5D%20(ApacheTrafficServer%2F7.1.8)%22%2C%22x-bb-ab%22%3A%220.226852%22%2C%22x-bb-client-request-uuid%22%3A%22e10e78c7-0174-4ddc-ada3-8bae7f8a7f83-6676795%22%2C%22x-bb-ip%22%3A%2290.255.3.162%22%2C%22x-bb-loop%22%3A%221%22%2C%22x-cdn-domain%22%3A%22www.bitballoon.com%22%2C%22x-country%22%3A%22GB%22%2C%22x-datadog-parent-id%22%3A%225781150176488674814%22%2C%22x-datadog-sampling-priority%22%3A%220%22%2C%22x-datadog-trace-id%22%3A%228327508323675804903%22%2C%22x-forwarded-for%22%3A%2290.255.3.162%22%2C%22x-forwarded-proto%22%3A%22https%22%2C%22x-language%22%3A%22en%2Cen%3Bq%3D0.9%22%2C%22x-nf-client-connection-ip%22%3A%2290.255.3.162%22%7D%2C%22queryStringParameters%22%3A%7B%7D%2C%22isBase64Encoded%22%3Atrue%7D",
            "x-apigateway-context": "%7B%22callbackWaitsForEmptyEventLoop%22%3Afalse%2C%22functionVersion%22%3A%22%24LATEST%22%2C%22functionName%22%3A%22df3c3b2f5bde88b6c40ceec083166a7ceb164e7ee08dd304aebcef292665419b%22%2C%22memoryLimitInMB%22%3A%221024%22%2C%22logGroupName%22%3A%22%2Faws%2Flambda%2Fdf3c3b2f5bde88b6c40ceec083166a7ceb164e7ee08dd304aebcef292665419b%22%2C%22logStreamName%22%3A%222020%2F02%2F14%2F%5B%24LATEST%5D0cff87e952054e5f9795fa19387195fc%22%2C%22clientContext%22%3A%7B%22custom%22%3A%7B%22netlify%22%3A%22eyJzaXRlX3VybCI6Imh0dHBzOi8vemVudW51Lm5ldGxpZnkuY29tIn0%3D%22%7D%7D%2C%22invokedFunctionArn%22%3A%22arn%3Aaws%3Alambda%3Aus-east-1%3A622137718745%3Afunction%3Adf3c3b2f5bde88b6c40ceec083166a7ceb164e7ee08dd304aebcef292665419b%22%2C%22awsRequestId%22%3A%22d60af3c4-0511-41c8-93d9-f5b85692b0de%22%7D",
            "host": "zenunu.netlify.com",
            "content-length": "0",
            "x-forwarded-host": "zenunu.netlify.com",
            "location": "https://zenunu.netlify.com/assets/zenunu-c8eb44e8c48167ea6d4f84a8e55c9c7f.js"
        }
    },
    "response": {
        "_headers": {}
    },
    "v": 1,
    "environment": "production",
    "label": "zenunu",
    "timestamp": "2020-02-14T13:18:54.283Z"
}
1 Like
4

Hello devotox!
I have seen you post against the process to run leverge browser cache for static assets, browser cookie and http headers.
I am working on a website, that has built up before 4 years in bootstrap 3 free responsive HTML template.

As of now trend has been changed, bootstrap 4 I in current trend and that too be using serverless, cdn for running, minifying, compressing and inlining css, jQuery and js files offered by maxbootstrapcdn, stackpath, angular, ajax, clouldflare etc…

I wanted to know how can I migrate my website from bootstrap3 to bootstrap 4 using above mentioned server less cdn urls. I am not a web developer, I am a beginner, who can upload the website files. Editing and changing, text, images and some other onpage seo related work.

And also let me know the way to use the functions for assigning cache , cookies get and response headers.

If you can help me, than please let me know the way to do it without paying. My website I http://www.akshitapatel.biz and and http://www.anushkasingh.biz. you can email me also with the positive response at akshitapatel4u@gmail.com

Thanks
With Regards
Akshita Patel

6

@devotox, it seem like this might be a bug. We’ll file an issue on this. We’ll update here if and when we have a fix out for it.

@Akshita_Patel, since you post is unrelated to this thread, please create a separate post by clicking the ‘new topic’ button on the main page: https://answers.netlify.com/.

7

Hi @devotox, I’ve made a change to how your site’s redirects work. Can you check if your redirect rules work now?

8

Was this made in the settings somewhere that I can see or in the netlify.toml? I ask because now that I have been able to look at this my site as automatically deployed at least 3 times (wooo robots ay)

9

At the moment with the latest deploy I still have the same problem but like I said many deploys have happened since your message

10

Can you tell me which urls are still getting redirected? Since your build seems to change all your filenames, I recommend also taking a look at the post about making the most of our CDN cache

11

Anything in the /assets/ directory

12

Filenames are changed depending on md5 hashes. This will slow down when development slows down

15

Hi, we have an experimental fix for the issue that you describe and have enabled it for your site. Could you re-deploy your site and see if that helps?

Also, I just wanted to point out that you have a redirect rule conflict here:

[[redirects]]
  status = 200
  force = false
  from = "/*"
  to = "/.netlify/functions/fastboot/:splat"

[[redirects]]
  status = 200
  force = false
  from = "/*"
  to = "/index.html"

Basically, that second rule will never get used.

16

Thank you. Deploying now to test.

Yes i know the second will never run and that is intentional. I have them both still in there because i was commenting out the functions redirect every time i needed the site to actually work

17

Please let us know how the testing goes, @devotox. We’re happy to answer if there are other questions as well.

18

I am still getting a 502 on requests that should hit the filesystem after my latest deploy.

6:20:18 PM: {"level":"info","time":1582914018317,"pid":8,"hostname":"169.254.172.189","reqId":4,"req":{"method":"GET","url":"/engines-dist/home/assets/engine.js"},"v":1,"message":"incoming request","environment":"production","label":"zenunu","timestamp":"2020-02-28T18:20:18.317Z"}
19

Hi, @devotox, that IP address isn’t routable (it is a local-link address) and therefore I cannot search our logs for it.

We still need the x-nf-request-id or the information it replaces to trace the requests returning the 502s. Would you please send us that information?

20

@luke

{
    "level": "info",
    "time": 1583163637524,
    "pid": 7,
    "hostname": "169.254.56.253",
    "message": "Fastboot Visit",
    "hashedKey": "82e8be118c5db25d4d824978014156ca7cbf8b05",
    "cacheHit": false,
    "request": {
        "url": "/assets/images/icons/icon-16-3cf27b93565c6c1e5359b2ea256f0416.png",
        "host": "zenunu.netlify.com",
        "port": 80,
        "path": "/assets/images/icons/icon-16-3cf27b93565c6c1e5359b2ea256f0416.png",
        "method": "GET",
        "protocol": "https",
        "location": "https://zenunu.netlify.com/assets/images/icons/icon-16-3cf27b93565c6c1e5359b2ea256f0416.png",
        "headers": {
            "accept": "image/webp,image/apng,image/*,*/*;q=0.8",
            "accept-encoding": "br, gzip",
            "accept-language": "en-GB,en-US;q=0.9,en;q=0.8",
            "client-ip": "86.53.244.42",
            "cookie": "ember_simple_auth-session=%7B%22authenticated%22%3A%7B%7D%7D; ember_simple_auth-session-expiration_time=86400",
            "referer": "https://zenunu.netlify.com/",
            "sec-fetch-mode": "no-cors",
            "sec-fetch-site": "same-origin",
            "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/79.0.3945.130 Safari/537.36",
            "via": "https/2 Netlify[b5fe4bec-7a3d-48f7-af89-114c527bb1a1] (ApacheTrafficServer/7.1.9)",
            "x-bb-ab": "0.246849",
            "x-bb-client-request-uuid": "b5fe4bec-7a3d-48f7-af89-114c527bb1a1-3663412",
            "x-bb-ip": "86.53.244.42",
            "x-bb-loop": "1",
            "x-country": "GB",
            "x-forwarded-for": "86.53.244.42",
            "x-forwarded-proto": "https",
            "x-language": "en,en,en;q=0.8",
            "x-nf-client-connection-ip": "86.53.244.42",
            "x-apigateway-event": "%7B%22path%22%3A%22%2Fassets%2Fimages%2Ficons%2Ficon-16-3cf27b93565c6c1e5359b2ea256f0416.png%22%2C%22httpMethod%22%3A%22GET%22%2C%22headers%22%3A%7B%22accept%22%3A%22image%2Fwebp%2Cimage%2Fapng%2Cimage%2F*%2C*%2F*%3Bq%3D0.8%22%2C%22accept-encoding%22%3A%22br%2C%20gzip%22%2C%22accept-language%22%3A%22en-GB%2Cen-US%3Bq%3D0.9%2Cen%3Bq%3D0.8%22%2C%22client-ip%22%3A%2286.53.244.42%22%2C%22cookie%22%3A%22ember_simple_auth-session%3D%257B%2522authenticated%2522%253A%257B%257D%257D%3B%20ember_simple_auth-session-expiration_time%3D86400%22%2C%22referer%22%3A%22https%3A%2F%2Fzenunu.netlify.com%2F%22%2C%22sec-fetch-mode%22%3A%22no-cors%22%2C%22sec-fetch-site%22%3A%22same-origin%22%2C%22user-agent%22%3A%22Mozilla%2F5.0%20(Macintosh%3B%20Intel%20Mac%20OS%20X%2010_15_0)%20AppleWebKit%2F537.36%20(KHTML%2C%20like%20Gecko)%20Chrome%2F79.0.3945.130%20Safari%2F537.36%22%2C%22via%22%3A%22https%2F2%20Netlify%5Bb5fe4bec-7a3d-48f7-af89-114c527bb1a1%5D%20(ApacheTrafficServer%2F7.1.9)%22%2C%22x-bb-ab%22%3A%220.246849%22%2C%22x-bb-client-request-uuid%22%3A%22b5fe4bec-7a3d-48f7-af89-114c527bb1a1-3663412%22%2C%22x-bb-ip%22%3A%2286.53.244.42%22%2C%22x-bb-loop%22%3A%221%22%2C%22x-country%22%3A%22GB%22%2C%22x-forwarded-for%22%3A%2286.53.244.42%22%2C%22x-forwarded-proto%22%3A%22https%22%2C%22x-language%22%3A%22en%2Cen%2Cen%3Bq%3D0.8%22%2C%22x-nf-client-connection-ip%22%3A%2286.53.244.42%22%7D%2C%22queryStringParameters%22%3A%7B%7D%2C%22isBase64Encoded%22%3Atrue%7D",
            "x-apigateway-context": "%7B%22callbackWaitsForEmptyEventLoop%22%3Afalse%2C%22functionVersion%22%3A%22%24LATEST%22%2C%22functionName%22%3A%228ab29a7c639d7e8e0eeecaf0a2b6d86cb758fd5d01f3df2444eb4ab2b031aef1%22%2C%22memoryLimitInMB%22%3A%221024%22%2C%22logGroupName%22%3A%22%2Faws%2Flambda%2F8ab29a7c639d7e8e0eeecaf0a2b6d86cb758fd5d01f3df2444eb4ab2b031aef1%22%2C%22logStreamName%22%3A%222020%2F03%2F02%2F%5B%24LATEST%5D483b81943be84320b7c025354f81d466%22%2C%22clientContext%22%3A%7B%22custom%22%3A%7B%22netlify%22%3A%22eyJzaXRlX3VybCI6Imh0dHBzOi8vemVudW51Lm5ldGxpZnkuY29tIn0%3D%22%7D%7D%2C%22invokedFunctionArn%22%3A%22arn%3Aaws%3Alambda%3Aus-east-1%3A673769985865%3Afunction%3A8ab29a7c639d7e8e0eeecaf0a2b6d86cb758fd5d01f3df2444eb4ab2b031aef1%22%2C%22awsRequestId%22%3A%221058669a-c566-4e35-b322-93a669ebb8e6%22%7D",
            "host": "zenunu.netlify.com",
            "content-length": "0",
            "x-forwarded-host": "zenunu.netlify.com",
            "location": "https://zenunu.netlify.com/assets/images/icons/icon-16-3cf27b93565c6c1e5359b2ea256f0416.png"
        }
    },
    "response": {
        "_headers": {}
    },
    "v": 1,
    "environment": "production",
    "label": "zenunu",
    "timestamp": "2020-03-02T15:40:37.524Z"
}
23

@luke

x-nf-request-id: dfe167ba-dfd9-4edb-81c3-557b6da929e4-9928409
24

Hi, @devotox. Thank you for sending those two examples.

For both the x-nf-request-ids (dfe167ba-dfd9-4edb-81c3-557b6da929e4-9928409 and b5fe4bec-7a3d-48f7-af89-114c527bb1a1-3663412), I can confirm that the file in question exists in that deploy but that the HTTP request is being proxied to the function instead.

This is incorrect behavior as the rule is not a force or 200! (with the !) so the URL should send the file in the deploy and not proxy the HTTP request.

We do have an open issue tracking this. I’m also researching a possible workaround in the meantime.

We’ll update this community topic again as soon as I know more about the workaround and if/when the issue itself is known to be resolved.

26

@luke Thank you guys for being on top of this and your quick replies. could you point me to the internal ticket to also watch that