Add support for "Range" header for Large Media files

lazoviccorp · June 8, 2020, 10:06am

This allows fetching only a small part of the file. Sometimes files are big and I don’t need to download the whole file just to get some small part of it. With Range header, you can specify the range of bytes you want to download.

See for request header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Range

See for response header: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range

fool · January 22, 2020, 9:14pm

Hi there,

I’ve added your voice to the open feature request on this topic so we can let you know if we implement it. I don’t anticipate that the functionality there will change anytime soon, though.

bryanpackman · January 13, 2020, 12:34am

Would love this feature too, for hosting podcasts.

brettbuddin · January 14, 2020, 11:39pm

This appears to be required by the latest Safari for playing files via audio. What’s annoying is sometimes requests for files like .mp3 return 206 Partial Content header and sometimes it just returns a 200 OK.

@fool Is the development team actively testing something in production? Is that why this behavior is so sporadic?

luke · January 20, 2020, 9:04am

@brettbuddin, as far as I know, no one is testing changes to this in production.

Would be willing to send us the x-nf-request-id headers for similar requests which return the two different (200 and 206) status codes for the same resource?

If this can be reproduced in a browser and can captured as HAR file, that would also be very helplful.

saw-mon_and_natalie · March 8, 2020, 11:12pm

@fool and @luke, I would also like this feature to be implemented. Since to be able to play audios or videos on macOS Safari or any of the browsers on iOS, the range header needs to be supported.

Right now, the media servers only send 200 response codes for requests with range headers.

$ curl -v --range 0-1 https://[SITE_ID].netlify.com/0.mp4

TLS handshake

*   Trying 165.227.12.111:443...
* TCP_NODELAY set
* Connected to [SITE_ID].netlify.com (165.227.12.111) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: path/to/ca-file
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=ca; L=San Francisco; O=Netlify, Inc; CN=*.netlify.com
*  start date: Jul  3 00:00:00 2019 GMT
*  expire date: Jul  7 12:00:00 2020 GMT
*  subjectAltName: host "[SITE_ID].netlify.com" matched cert's "*.netlify.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x5c9cc0)

> GET /0.mp4 HTTP/2
> Host: [SIDE_ID].netlify.com
> range: bytes=0-1
> user-agent: curl/7.68.0
> accept: */*
>

TLS handshake

* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 150)!

< HTTP/2 200
< alt-svc: clear
< cache-control: public, max-age=0, must-revalidate
< content-length: 47418
< content-type: video/mp4
< date: Sun, 08 Mar 2020 22:44:37 GMT
< etag: 6680879abb0b95dd64aa69c55fc7c28304d2aa75ed26e4ecc1a911ff26de3bf6
< via: 1.1 google
< x-bb-gen-id:
< age: 0
< server: Netlify
< vary: Origin
< x-nf-request-id: af47b372-d061-421e-9e3e-261a9d78dc3c-2736084
<

Also, the servers don’t respond with 206 - Partial Content when using HTTP 1.1 protocol:

$ curl -v --http1.1 --range 0-1 https://[SITE_ID].netlify.com/0.mp4

Looks like this is only the problem with #netlify-large-media-nlm . When I test the range header request against Netlify servers, they respond back with 206 Partial Content.

$ curl -v --http1.1 --range 0-1 https://[SITE_ID].netlify.com/0.js

"TLS handshake

*   Trying 165.227.0.164:443...
* TCP_NODELAY set
* Connected to [SITE_ID].netlify.com (165.227.0.164) port 443 (#0)
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: path/to/ca-file
  CApath: none
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=ca; L=San Francisco; O=Netlify, Inc; CN=*.netlify.com
*  start date: Jul  3 00:00:00 2019 GMT
*  expire date: Jul  7 12:00:00 2020 GMT
*  subjectAltName: host "[SITE_ID].netlify.com" matched cert's "*.netlify.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.

> GET /0.js HTTP/1.1
> Host: [SITE_ID].netlify.com
> Range: bytes=0-1
> User-Agent: curl/7.68.0
> Accept: */*
>

TLS handshake

* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Mark bundle as not supporting multiuse

< HTTP/1.1 206 Partial Content
< Accept-Ranges: bytes
< Cache-Control: public, max-age=0, must-revalidate
< Content-Length: 2
< Content-Range: bytes 0-1/81
< Content-Type: application/javascript
< Date: Sun, 08 Mar 2020 23:01:53 GMT
< Etag: "c0c19e732817fb19be8c271f3cee6263-ssl"
< Strict-Transport-Security: max-age=31536000
< Age: 0
< Connection: keep-alive
< Server: Netlify
< X-NF-Request-ID: 52676c1a-4a40-43ba-b2a0-a20a2b9dc050-6321477
<

Using #service-worker I was able to work around this issue. The service-worker.js would intercept the requests with range headers, fetches the whole content and crafts the bytes it needs to send as a partial response back to the main browser window instance. But this is still not fully bulletproof.

luke · March 9, 2020, 12:14am

Hi, @saw-mon_and_natalie. I’ve added this as a +1 to this feature request. We’ll post an update here if/when this becomes supported.

Thank you for sharing your solution of using a service worker to workaround this limitation as well. That isn’t something I’ve seen before and it “wow-ed” me.

tomrutgers · March 9, 2020, 12:59am

Here’s an excellent article on that @luke https://philna.sh/blog/2018/10/23/service-workers-beware-safaris-range-request/

brook · March 28, 2020, 6:01am

This is now blocking Safari from playing videos.
Does anyone know of a way around this until it’s supported?

luke · March 31, 2020, 8:50am

Hi, @brook, and welcome to the Netlify community site.

I have added you as a +1 to the open feature request to support these headers.

The only workaround I know of is the service worker method posted above.

mpace965 · April 19, 2020, 8:14pm

Hello, I’m also being affected.

Edit: I implemented the service worker workaround. This works fine for Safari mobile, but certain in-app browsers (like Instagram’s) either don’t support or don’t allow service workers. A use-case I’m trying to support is playing videos from the Instagram in-app browser, so this may be a dealbreaker for me.

Edit 2: I am trying to re-implement the service worker solution as a Netlify function that does a similar proxy/caching behavior. Will update here when I make more progress.

perry · April 20, 2020, 6:15pm

I hear you, @mpace965 - I have added your voice to the chorus on this.

If you find an additional workaround to accomplish what you need, please do let us know.

thojansen · June 5, 2020, 12:56pm

+1. I had the same problems and implemented this as follows.

gist.github.com

https://gist.github.com/thojansen/46feeb99faba21c710f5770441a41843#file-service-worker-js-L50-L56

index.html

<html lang="en">
<head>        
  <script type="text/javascript" src="./index.js"></script>
  <script type="text/javascript" src="./service-worker.js"></script>
  <meta charset="utf-8">
</head>
<body>

  <video autoplay muted loop playsinline class="video">
    <source src="./assets/video/animation.mp4" type="video/mp4">

This file has been truncated. show original

index.js

function waitUntilInstalled(registration) {
    return new Promise(function(resolve, reject) {
      if (registration.installing) {        
        registration.installing.addEventListener('statechange', function(e) {
          if (e.target.state === 'installed') {
            resolve();
          } else if (e.target.state === 'redundant') {
            reject();
          }
        });

This file has been truncated. show original

service-worker.js

var CACHE_VERSION = 1;
var CURRENT_CACHES = {
  prefetch: 'prefetch-cache-v' + CACHE_VERSION
};

var urlsToPrefetch = [
  '/assets/video/animation.mp4'    
];

self.addEventListener('install', function(event) {

This file has been truncated. show original

Had to adjust a few lines from the following example to make it work with netlify CDN rewrites.
https://googlechrome.github.io/samples/service-worker/prefetch-video/index.html

I am sure there needs to be a better way for the rewrites. Curious to get some pointers on how to hook into that process correctly.

luke · June 8, 2020, 10:06am

Hi, @thojansen, I wanted to chime in that our support team doesn’t troubleshoot third-party code so we won’t be answering this question. The question itself is welcome and others might answer you here on the community site but I wanted to set expectations appropriately about the scope of support for our support team.