2024-06-05 00:00

I found Zola site hot reloading to not work in certain circumstances on Linux and set out to fix it. This is the tale of the journey with some extra thoughts.

Zola

At the time of this writing, this site is built by the static site generator zola. Zola is simple (relative to other options such as Jekyll and Hugo) and minimal, two things I'm a big fan of.

Hot Reloading

The zola serve command spawns a local web server and serves up your site locally so that you can see changes you make to the site reflected on the development instance in near real time. I author most of my material on macOS, but one day I tried to do a bit of writing on Linux and hit this issue, noticing that changes to the main configuration file were not triggering site reloads. The issue itself matched my usecase, editing the config file with some flavor of vim. I enjoy using zola and so wanted to fix this. I also had a hunch of what the issue might be.

Long story short, I fixed the issue here. Going into detail, the issue is rooted in the way watching for files works on Linux compared to, for example, macOS. Linux uses inotify which watches individual file inodes and reports changes to the content an inode references. macOS uses FSEvents which monitors content on the directory level.

When editing files with Vim, Vim will by default stage changes in a swap file. Upon saving, the swap file takes the place of the original file being edited, for example by asking the OS to delete the original file and then renaming the swap file to the original file's name. When this happens, the inode of the file we're editing changes. FSEvents has no problem tracking changes made in such a manner. While I haven't looked into the implementation of FSEvents, I suspect the filename participates in some kind of event filtering, so even if the inode of a file changes, events related to the same filename can be captured with no perceived discontinuity. With inotify, the inode we're watching is freed up after our file is deleted, so although a "different" file with the same filename exists, we won't see events related to it.

The solution to the issue above is to watch for changes on a specific file by watching the owning parent directory non-recursively and filtering events raised on that directory for the filenames we're interested in, emulating behavior of FSEvents.

notify-rs

Alas, I could not simply land the change above. I hit a build issue in Windows CI caused by some other change on the next branch (the branch that prepares the next stable release) updating the underlying libc dependency version to one that was incompatible with the file system events library notify-rs version zola uses and the Windows runtime. So to unblock myself I needed to upgrade zola's notify-rs dependency from v4 to v6.

I fixed that issue here. This change took some time because the fix was not as localized as fixing the file-watching issue described above. Below I describe some takeaways from this experience.

Open Source Contributions

Firstly, should the author of zola, Vincent "Keats" Prouillet, read this, I would like to thank him for his guidance, patience, and professionalism while helping me as a new contributor to zola.

Now, unrelated to him or zola in particular, my time contributing to zola reminded me that open source contribution is cool and sexy when you are either getting paid to do it, for example when the company you work for sponsors your contributions, or when you otherwise derive some kind of productive benefit from it. Some do it for fun, and that's fine, but if you're just trying to get some work done, you need to pick your battles.

As an aside, I'll mention that I have interviewed numerous people with dense GitHub contribution histories, and I have found no correlation between history density (which includes things such as opening, closing, and commenting on issues or pull requests) and software engineering skill relative to some practical application.

Not every open source project has a roadmap or outline on preferred approaches or design philosophy related to solving outstanding problems. This means that while you could just show up and solve the issue in your special way, you're likely not going to land that solution without first having a discussion on the preferences of the maintainers and then maybe some refactoring or rework. This is often a one-time cost for new maintainers--after someone is assimilated, they just "get it" and can move changes through more quickly, taking into account the local development culture. Still, this onboarding process can be trying if all you wanted to do was a drive-by commit--write some code and land it.

My last note is just expanding on return on investment, in particular what you get in exchange for your time. Clout is not very interesting or meaningful to me. So if not money to pay the bills, I at least need to be solving a problem I'm hitting in my own work. The nice thing about distributed open-source projects is that my patch can fix the same issue for everyone else. Then, an additional benefit is that your work is recorded on a public ledger of sorts as a showcase of how you operate in such an often high-ambiguity, proceed-by-rough-consensus environment and what you're willing to put your name on implementation-wise. In my patches above, I spent most of my time on refining my implementation to strike what I felt was a good balance between integrating into the existing codebase and something that solved the problem in a way that added as little code as possible while being verbose and flexible enough to be easily maintained.