From 913b8fcdfa57dba41b22e8a98de67819f6714f49 Mon Sep 17 00:00:00 2001 From: Greg Hellings Date: Tue, 30 Apr 2024 23:17:02 -0500 Subject: [PATCH] Add Nix build post --- content/posts/better-image-building.md | 229 ++++++++++++++++++++++ content/posts/multiple-version-testing.md | 2 + content/posts/nixos-on-raspberry-pi-4.md | 3 + 3 files changed, 234 insertions(+) create mode 100644 content/posts/better-image-building.md diff --git a/content/posts/better-image-building.md b/content/posts/better-image-building.md new file mode 100644 index 0000000..ff0a1b0 --- /dev/null +++ b/content/posts/better-image-building.md @@ -0,0 +1,229 @@ +--- +title: "Better Image Building" +date: 2024-04-30T22:18:35-05:00 +draft: false +tags: + - Nix + - NixOS + - Flakes + - CI +--- + +I've managed to recently upgrade my image building process for this +website, using some of what I have been learning about +[Nix](https://nixos.org/). For a while +while I was first learning to use Nix I thoguht of the language, the +packages, and the OS as being one thing. And this did a disservice to +both the breadth of what Nix offers as well as the capabilities it +offers. + +## Old Paradigm + +So previously with this site I used a fairly straightforward and manual +process. I installed [Hugo](https://gohugo.io/) on my local machine with +dnf, ran the dev server, and +edited my pages. When I pushed the commits into GitHub, I had a +`Containerfile` in the repository that would get built with Docker or +[Podman](https://podman.io/), pushing the resulting image to my image +repository. I used a +multi-stage file, first pulling the Hugo image to build the files and +then grabbing the resulting build produts to place them into the +[Nginx](https://nginx.org/en/) image. + +When I first learned how to use NixOS I spun up [Gitea](https://nginx.org/en/), +added a [flake.nix](https://src.thehellings.com/greg/homepage/-/blob/main/flake.nix) +file that would offer me Hugo locally through direnv, and wrote a shell +script that would test building the Containerfile locally to give myself +a sanity test. I felt that this was a pretty snappy way to handle it. +Now, anywhere I had Nix, I could just check out my code and run the +development bits of it. + +## More Nix + +Although I have since moved my self-hosting to Gitlab instead of Gitea, +I was operating under the same basic flow. However, recently, I beefed up +my use of Nix in a few nifty ways that stray far from just leveraging the +pacakages and the OS. Now, the whole cycle of the build process is +managed by Nix. + +In my flake.nix file, I have added a few enhancements. First, I added +[devshell](https://github.com/numtide/devshell). This is not to be +confused with the built-in devShells from flakes themselves. While there +is nothing in devshell that you cannot do with the built-in tool, there +is just so much boiler-plate built in. For one, it gives me a nice +listing of all the tools I want to highlight in this development. +Secondly, it produces a very nice motd by default to help alert the +developer that they are entering the shell (this is helpful for me, +as I use direnv to automate activating my shell environments). + +The second enhancement I added was using a flake to wrap +[process-compose](https://community.flake.parts/process-compose-flake). +This eliminates the need for me to write a shell script to start up +Hugo or remember the exact command to use. Plus, process-compose has +a very nice Curses tui to show the status of the applications you +are running. While starting up the hugo server is not exactly +rocket science, process-compose is capable of far more than just +starting one service in a tui. You can start up databases, caching +services, and more with a single command and see them all nicely +displayed in the tui. The available options are quite extensive. +While they might be slight overkill for just this site, the power +that can be shown for more complex development work is impressive. +Spinning up whole fleets of VMs or other servies, allowing for +dependencies and readiness checks. The ability to spin up a reproducible +environment for development work with such a tool is very impressive. + +## Image Building with Nix + +I was reading someone else's blog post a few weeks ago about building +OCI container images with Nix and how the images you build are even +better than the ones you can build with a Dockerfile or Containerfile. +I had my doubts, at first, but as I looked into the subject, it turns +out this is actually pretty true. On top of the reproducible ability +for packages that are identified with a `flake.lock` file which +you do not get when using a different base for your container, +a layered image from Nix creates one layer per package you add to +the image. This means that every image I build and deploy with +my Nix tools can share the same set of base packages. Yes, I hear +you say, if you build a dozen different images from the same +base image, all of those resulting images will share that base +layer. And that is true. But when you start to install other +packages on top of it, unless you install all of the pacakges +in the same way and the same order in every separate Containerfile, +you will not have the benefit of sharing the layers above that +base image. With a Nix built container, this part is already handled +by the tooling directly. + +So how do I build this container? + +### Build the site + +First, I create a derivation that builds and packages my site's static +version: + +```nix +site = pkgs.stdenv.mkDerivation { + name = "Greg's homepage"; + nativeBuildInputs = [ pkgs.hugo ]; + buildPhase = '' + hugo -v + ''; + installPhase = '' + mkdir -p $out + cp -R public/* public/.* $out + ''; + src = ./.; +}; +``` + +This derivation is simple. It uses `pkgs.hugo` as a native build +input, executes hugo against the repository, and copies the +resulting files to the `$out` target. It isn't important now, but +I build this as a package in the flake outputs. I could easily +have put it as just local variable, or such. But this example +shows easily how to build the local package directly in the +included flake. + +### Create Nginx Config + +I still plan to create a static site that is hosted by Nginx +in my container. To accomplish this, Nginx will need a very +minimal configuration. At least for now. So I add another +package to the flake outputs that just creates the Nginx file: + +```nix +nginx-config = pkgs.writeText "nginx.conf" '' + user root root; + worker_processes 10; + pid /tmp/nginx.pid; + daemon off; + + error_log stderr; + + events { + } + + http { + access_log stdout; + include ${pkgs.nginx}/conf/mime.types; + server { + listen 80; + root ${self'.packages.site}; + autoindex off; + } + } +''; +``` + +This derivation is even simpler than the previous one. I +leverage the `pkgs.writeText` function to create the +configuration file for Nginx. Unlike in the official version +of the Nginx container, there is no need here to copy my +resulting site files to some path. I can directly pass the +path of my site to this package. You can see this happen +at the line `root ${self'.packages.site}`. In this case, the +value of `self'` is just what I use to reference the flake +itself. Since `site` is the derivation that includes the built +site products, I can just reference this directly. + +### Create the Image + +Again, this is just a static site, so the resulting image is +fairly mundane and straightforward. However, it is telling +for how easy it is to build up the resulting packages and +derivations into a simple image. I choose to make the container +image the default package for this particular flake repo. And +I am sure to build a layered image so that I can make the most +of caching and reusing shared packages with any other containers +I might run. Here is how I build the image: + +```nix +default = pkgs.dockerTools.buildLayeredImage { + name = "gregs-homepage"; + tag = "latest"; + contents = [ + self'.packages.site + (pkgs.buildEnv { + name = "image"; + paths = with pkgs; [ + dockerTools.fakeNss + nginx + ]; + pathsToLink = [ "/bin" "/etc" "/var" ]; + }) + ]; + fakeRootCommands = '' + mkdir tmp + chmod 0777 tmp + ''; + config = { + Cmd = [ (lib.getExe pkgs.nginx) "-c" self'.packages.nginx-config "-e" "stderr" ]; + Port = [ 80 ]; + }; +}; +``` + +Here we can see a simple image with Nginx to serve up the +base image that is needed. Now, with this added into the flake +package output, I can run `nix build .` and Nix will grab the +exact version of Hugo that I use, build my site, and package +it into an OCI container image. By default, nix build drops +a symlink to the final derivation in the current directory +with the name `result`. Using Podman or Docker, I can import +my image with `podman load -i result` and my image `gregs-homepage:latest` +will be loaded into my local images. I can then tag and push +it anywhere I want. + +## Conclusion + +I know there is nothing Earth shattering here. There is not +anything that cannot be done manually or with some amount of +effort using other tools. What is so nice about Nix is that +it wraps up all that manual and boiler plate bits that I +would probably never have spent the time to write all for myself +and gives it to me in very nice, neat packages. + +What do you use to simplify your development, building, and +deployment process? So far, Nix is great for development +and building, and I use NixOS on all my servers. But I'm +still hoping to get even deeper into the Nix world and find +or create some good deployment tooling. diff --git a/content/posts/multiple-version-testing.md b/content/posts/multiple-version-testing.md index ebd6a56..9e104dd 100644 --- a/content/posts/multiple-version-testing.md +++ b/content/posts/multiple-version-testing.md @@ -2,6 +2,8 @@ title: "Multiple Version Testing" date: 2023-05-02T11:07:07-05:00 draft: false +tags: + - Nix --- If you are interested in running multiple versions of the same development language using Nix and Nix Flakes, [ride along](https://src.thehellings.com/greg/multi-python-demo) diff --git a/content/posts/nixos-on-raspberry-pi-4.md b/content/posts/nixos-on-raspberry-pi-4.md index e822c29..fae9ec3 100644 --- a/content/posts/nixos-on-raspberry-pi-4.md +++ b/content/posts/nixos-on-raspberry-pi-4.md @@ -2,6 +2,9 @@ title: "Nixos on Raspberry Pi 4" date: 2022-04-07T12:20:02-05:00 draft: false +tags: + - Nix + - NixOS --- So I've been playing around with NixOS on a Raspberry Pi 4 that I have at the house. NixOS seems like a great idea, but getting it onto the Raspberry Pi 4 is something of a beast of a process. So I wanted to document here how I did