Add support for "Range" header for Large Media files

This allows fetching only a small part of the file. Sometimes files are big and I don’t need to download the whole file just to get some small part of it. With Range header, you can specify the range of bytes you want to download.

See for request header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range

See for response header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range

4 Likes

Hi there,

I’ve added your voice to the open feature request on this topic so we can let you know if we implement it. I don’t anticipate that the functionality there will change anytime soon, though.

2 Likes

Would love this feature too, for hosting podcasts.

1 Like

This appears to be required by the latest Safari for playing files via audio. What’s annoying is sometimes requests for files like .mp3 return 206 Partial Content header and sometimes it just returns a 200 OK.

@fool Is the development team actively testing something in production? Is that why this behavior is so sporadic?

1 Like

@brettbuddin, as far as I know, no one is testing changes to this in production.

Would be willing to send us the x-nf-request-id headers for similar requests which return the two different (200 and 206) status codes for the same resource?

If this can be reproduced in a browser and can captured as HAR file, that would also be very helplful.

@fool and @luke, I would also like this feature to be implemented. Since to be able to play audios or videos on macOS Safari or any of the browsers on iOS, the range header needs to be supported.

Right now, the media servers only send 200 response codes for requests with range headers.

$ curl -v --range 0-1 https://[SITE_ID].netlify.com/0.mp4
TLS handshake
*   Trying 165.227.12.111:443...
* TCP_NODELAY set
* Connected to [SITE_ID].netlify.com (165.227.12.111) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: path/to/ca-file
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=ca; L=San Francisco; O=Netlify, Inc; CN=*.netlify.com
*  start date: Jul  3 00:00:00 2019 GMT
*  expire date: Jul  7 12:00:00 2020 GMT
*  subjectAltName: host "[SITE_ID].netlify.com" matched cert's "*.netlify.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5c9cc0)
> GET /0.mp4 HTTP/2
> Host: [SIDE_ID].netlify.com
> range: bytes=0-1
> user-agent: curl/7.68.0
> accept: */*
>
TLS handshake
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 150)!
< HTTP/2 200
< alt-svc: clear
< cache-control: public, max-age=0, must-revalidate
< content-length: 47418
< content-type: video/mp4
< date: Sun, 08 Mar 2020 22:44:37 GMT
< etag: 6680879abb0b95dd64aa69c55fc7c28304d2aa75ed26e4ecc1a911ff26de3bf6
< via: 1.1 google
< x-bb-gen-id:
< age: 0
< server: Netlify
< vary: Origin
< x-nf-request-id: af47b372-d061-421e-9e3e-261a9d78dc3c-2736084
<

Also, the servers don’t respond with 206 - Partial Content when using HTTP 1.1 protocol:

$ curl -v --http1.1 --range 0-1 https://[SITE_ID].netlify.com/0.mp4

Looks like this is only the problem with #netlify-large-media-nlm . When I test the range header request against Netlify servers, they respond back with 206 Partial Content.

$ curl -v --http1.1 --range 0-1 https://[SITE_ID].netlify.com/0.js
"TLS handshake
*   Trying 165.227.0.164:443...
* TCP_NODELAY set
* Connected to [SITE_ID].netlify.com (165.227.0.164) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: path/to/ca-file
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=ca; L=San Francisco; O=Netlify, Inc; CN=*.netlify.com
*  start date: Jul  3 00:00:00 2019 GMT
*  expire date: Jul  7 12:00:00 2020 GMT
*  subjectAltName: host "[SITE_ID].netlify.com" matched cert's "*.netlify.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
> GET /0.js HTTP/1.1
> Host: [SITE_ID].netlify.com
> Range: bytes=0-1
> User-Agent: curl/7.68.0
> Accept: */*
>
TLS handshake
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse
< HTTP/1.1 206 Partial Content
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0, must-revalidate
< Content-Length: 2
< Content-Range: bytes 0-1/81
< Content-Type: application/javascript
< Date: Sun, 08 Mar 2020 23:01:53 GMT
< Etag: "c0c19e732817fb19be8c271f3cee6263-ssl"
< Strict-Transport-Security: max-age=31536000
< Age: 0
< Connection: keep-alive
< Server: Netlify
< X-NF-Request-ID: 52676c1a-4a40-43ba-b2a0-a20a2b9dc050-6321477
<

Using #service-worker I was able to work around this issue. The service-worker.js would intercept the requests with range headers, fetches the whole content and crafts the bytes it needs to send as a partial response back to the main browser window instance. But this is still not fully bulletproof.

Hi, @saw-mon_and_natalie. I’ve added this as a +1 to this feature request. We’ll post an update here if/when this becomes supported.

Thank you for sharing your solution of using a service worker to workaround this limitation as well. That isn’t something I’ve seen before and it “wow-ed” me. :+1:

1 Like

Here’s an excellent article on that @luke https://philna.sh/blog/2018/10/23/service-workers-beware-safaris-range-request/

1 Like

This is now blocking Safari from playing videos.
Does anyone know of a way around this until it’s supported?

1 Like

Hi, @brook, and welcome to the Netlify community site. :smiley:

I have added you as a +1 to the open feature request to support these headers.

The only workaround I know of is the service worker method posted above.

Hello, I’m also being affected.

Edit: I implemented the service worker workaround. This works fine for Safari mobile, but certain in-app browsers (like Instagram’s) either don’t support or don’t allow service workers. A use-case I’m trying to support is playing videos from the Instagram in-app browser, so this may be a dealbreaker for me.

Edit 2: I am trying to re-implement the service worker solution as a Netlify function that does a similar proxy/caching behavior. Will update here when I make more progress.

1 Like

I hear you, @mpace965 - I have added your voice to the chorus on this.

If you find an additional workaround to accomplish what you need, please do let us know.

+1. I had the same problems and implemented this as follows.

Had to adjust a few lines from the following example to make it work with netlify CDN rewrites.
https://googlechrome.github.io/samples/service-worker/prefetch-video/index.html

I am sure there needs to be a better way for the rewrites. Curious to get some pointers on how to hook into that process correctly.

Hi, @thojansen, I wanted to chime in that our support team doesn’t troubleshoot third-party code so we won’t be answering this question. The question itself is welcome and others might answer you here on the community site but I wanted to set expectations appropriately about the scope of support for our support team.

What is considered a large media file? I have 8 mp4 videos of about 4 seconds each and on average 500kb. Do I need this?

probably not, Timothy!

By “Large Media” files, we mean only specific files from people who have specifically and intentionally configured our Large Media service, described here: https://docs.netlify.com/large-media/overview/ . You would have had to opt in in our UI, and install some things on your local computer, and change a bit about how your site is laid out in your git repository, if you were using that service, so it’s nothing you use accidentally. I wouldn’t use the service for files under 10MByte, myself, and even if I had one or a few of those, I would take some careful consideration before enabling due to limitations like the one described in this thread :slight_smile:

Range header should work for files served from our normal CDN - are you not seeing that, Timothy? If you could capture a HAR file of what you are seeing (https://toolbox.googleapps.com/apps/har_analyzer/), and share it here, we can probably help diagnose what is happening since I think it probably isn’t the topic that started this thread.

1 Like

Hey thanks for the info. Also you are right, the videos load and play perfectly from your cdn, also on safari. Turns out the videos don’t load with jekyll when testing local. A bit weird, but hey software right :slight_smile: Also the videos showed green pixelated but this was because somehow when converting to mp4 via VLC media player it also compressed files up to about 1/3 of the original (still not sure why this is). I dragged the timeline back and forth when playing these in VLC and they began to break which resulted in videos showing green pixels in safari only. Moral of the story I guess is start with good mp4 videos.

Thanks for the update to share what you discovered, @Timothy.

1 Like