There’s currently 2 main types of content: Pages and static files. Copying off a design pattern from Jekyll, i’d like to add a 3rd, which i’ll refer to assets.
Given an additional subdirectory in the project root, assets/, all files inside are copied to the output directory as-is, as with static files, but are passed through the template engine first.
Specifically to allow templating other assets which don’t fall into the standard page templating such as those in https://tools.ietf.org/html/rfc5785, or to generate more specific sitemaps / feeds for ingesting by specific services.
It’s a really nice byproduct of how Jekyll handles its content structure, which I think would be a really nice thing to add!
The current special-casing of such pages as robots.txt, 404.html and sitemap.xml is quite icky, and this proposal would fix it. The first two, especially, as their default templates are essentially useless (especially robots.txt). I bring sitemap.xml into it because I personally dislike the concept for most sites, and because it adds an interesting dimension to the consideration.
robots.txt and 404.html could be removed without harm, and if the user wants them to be generated with templates, they can be assets/{robots.txt,404.html}: then they’re not special-cased at all in Zola.
With sitemap.xml, you’d lose something by removing all traces of the built-in template, as anyone wanting it would need to replicate the template; and adding a config option to disable its generation still has it special-cased. One possibility, with a templated assets directory, would be to remove sitemap.xml generation from Zola altogether, but leave the built-in template; then assets/sitemap.xml can just contain {% extends "sitemap.xml" %}.
I’m not going to argue for removing sitemap.xml by default in this way; I imagine that would be an up-hill battle, and would distract from this thread. I have mentioned it because of what it demonstrates about the power and desirability of an assets directory.
For example for the sitemaps, someone opened an issue to split them into many files when they get too big: https://github.com/getzola/zola/issues/562 so you get a master sitemap and n other sitemaps containing the actual links.
I quite like @chris-morgan 's idea of keeping the existing templates. If assets are processed after builtins, then they could be easily customized, especially with {% extends %}. It would allow removing of special casing, or at least making them a lot more generic, although i’m not sure how configuration would go if people wanted to turn them off (alternatively, we could just always render the templates, they’re fairly cheap).
RE https://github.com/getzola/zola/issues/562, this could be solved easily by configuring zola not to output a sitemap, and then just copying the existing sitemap.xml template, and limitting where necessary. I think any more automated one (such as generating specific sitemaps for each section, much like Hugo does), then that’s probably unrelated to this. We could handle this by using templated directory names ($section/sitemap.xml), but that’s probably unnecessary.
Would all your needs be addressed by a proposal for custom output formats? From what i understand, the usecases seem similar so we would benefit thinking this through as a community and taking the best from both patterns/proposals for the future of Zola