Welcome to the thread, @msanchez!
I think both of you are showing the compressed size of your files. Our CDN internally transfers, and stores in cache, uncompressed files in cache; that is the size that matters for performance checking from our internal logs. I understand that your browser downloaded many fewer bytes, but in transferring the file, the actual size on disk matters. That is the number one factor in how fast we can serve uncached contents - how big the uncompressed asset is. What started this thread - a 23MByte font file - will never work well on our CDN as we have not designed it with that use case in mind.
That said, I also do not think that an average response time of 5s for interactive content is acceptable; however, since our CDN caches opportunistically, so if the asset is frequently used on a CDN node, it will be served in milliseconds, not seconds. If it is not in cache, it may take some time to transfer. We aim for that time to be as small as possible, but smaller assets are transferred faster.
It is an interesting assertion that json files are cached differently and while I would be surprised to hear it, nonetheless I have posed the question to the edge networking team, and will let you know what I find out.