Pretty URLs, URL Routing, and Trailing-Slashes

Greetings :wave:t2:

I feel like this post may be most pertinent for @fool since this topic’s been handled a number of times here on The Community (by him and others) but I’m looking to clarify implementation details and strongly urge some extended documentation and/or UI changes.

The “Pretty URLs” check-box has global routing side effects that are un-documented (as far as I can tell) and un-beholden to the “Disable Asset Optimization” flag, meaning that the “Pretty URLs” checkbox has side effects that are beyond the scope of the parent “disable” flag, even though the “Pretty URLs” checkbox itself becomes disabled when the parent flag is checked.

To walk this out, I made four sites cataloging the routing (and markup re-writing) behavior of when the Pretty URLs flag is checked or unchecked and when the “Disable Asset Optimization” flag is checked or unchecked:

The good news is that the advertised benefit of the “Pretty URLs” feature seems to work fine when the box is checked and asset optimization is turned on - the part where anchor tags in the HTML markup have their href= changed to remove the .html tail (at least when the file exists). This makes sense. That’s the control pattern I would expect - if AO is disabled overall, anchor-href-swapping shouldn’t occur. If AO is enabled overall but Pretty URLs is disabled, anchor-href-swapping shouldn’t occur. If AO is enabled overall and Pretty URLs is enabled, anchor-href-swapping should occur. This is the working behavior and that’s great.

What’s not so great and quite confusing is that the value of the Pretty URLs check-box, even when it’s disabled (e.g. “disable all asset optimization” is enabled) has meaningful routing changes for the site. Those four example sites document that routing fairly well, but there are some serious impacts there. Namely, SEO gets super knocked if you’re serving the same content on two paths (notably like ./blog/post no-slash and ./blog/post/ slashed) - which is controlled by the Pretty URLs routing side-effect.

In regards to the Pretty URLs feature, the docs currently state:

In addition to forwarding paths like /about to /about/ (a common practice in static sites and single-page apps), it will also rewrite paths like /about.html to /about/

But that’s it. I think the first part, "In addition to forwarding paths like /about to /about/" may be referencing routing changes? And the latter mentions the markup changes / anchor-href-swapping? But overall there could be a lot more documentation on what’s going on with that flag. Especially thoroughly documenting that, even when that flag is not editable, its checked-or-not-checked status will have significant routing impacts on the site.

Outside of adding more documentation, could this functionality be refactored and split? The concept of parsing my HTML and anchor-href-swapping sounds good, and I understand is related to the server’s routing to some degree, but I think they could be de-coupled and it would make things easier to understand. Ideally I’d love to see a separate panel inside the Build & Deploy screen that allows a site to switch between “Resolve regardless of slash trailing” and “Follow document / directory based routing for trailing slash” (and perhaps a warning that the former can hurt SEO? :stuck_out_tongue_winking_eye: ) – this toggle should be switchable and controllable regardless of what the Asset Optimization settings are, and that would allow the removal of the global routing side-effects tied to the Pretty URLs feature. Pretty URLs would truly just be anchor-href-swapping at that point (which is what we’d expect given the available controls).

I realize that would be a breaking change and would probably require some cross-over time for things to get hashed out, but the current functionality is difficult to work with and mostly undocumented.

For what it’s worth, I’m also not remiss to the fact that a lot of people are running PWAs on Netlify and are having client-side routing issues too, making this whole conversation more convoluted and difficult, but I think it would actually be more helpful to sort out PWA routing problems if the Netlify controls were more clear on exactly what the Edge is going to do regarding its own routing.

Thanks for the long read :slight_smile: Technically @Pie called for it :wink:

Jon :netliheart:

Edit: Gatsby community reference

1 Like

@jonsully Wow, this must have been a lot of work to figure out.

I just use canonical meta tags:

1 Like

Haha. Yeah Canonical tags can be super useful and are a great tool, but even they always work best when routing is clear and straightforward. They also can’t help with client-side routing, which is a whole bag of worms on its own :rofl:

Plus analytics platforms recognize the slashed and non-slashed URLs as different and that can break up your analytics data in bad ways too :grimacing: that’s actually how I first realized my site was having this issue and what lead me to dig ALL THE WAY DOWN the rabbit hole :laughing: - (note the /blog/ and /blog as separate records)

Hey @jonsully,

Love the repros. I’ll get this raised internally and we’ll take it from there. As always, thanks for your help and input!

1 Like

Hey Jon,

I’ve briefly spoken with our traffic team ahead of our meeting later this week. We’ve got a plan moving forward – we’re going to have a sit down, work out what happens (using your repros) for each use case and I’ll get a support guide published with our advised configs, the role that the SSG plays in trailing slashes and how we make use of canonical link headers. Stay tuned!

1 Like

Awesome. Thanks @Pie. Hopefully getting a good guide in place (especially covering the awkward controls in that UI panel) will clear things up for a lot of folks. :netliheart:

1 Like