245 lines
9.6 KiB
Markdown
245 lines
9.6 KiB
Markdown
---
|
|
title: "Multiple Version Testing"
|
|
date: 2023-05-02T11:07:07-05:00
|
|
draft: false
|
|
---
|
|
If you are interested in running multiple versions of the same development language using
|
|
Nix and Nix Flakes, [ride along](https://src.thehellings.com/greg/multi-python-demo)
|
|
while I setup a development environment that includes
|
|
Python 2.7, 3.5-3.12 along with necessary development libraries. I will not be covering
|
|
installing Nix on your system or the basics of the Nix package system or language. At the
|
|
link above, you can see and fetch the flake.nix file. Each commit in the repository
|
|
corresponds to a version of the file in this post.
|
|
|
|
If you only want to see the full flake file and get to using it for yourself, check down
|
|
[at the bottom of the post](#full-array).
|
|
|
|
**Caveat: This post assumes you have basic knowledge of Nix, its installation, and its
|
|
file syntax.**
|
|
|
|
# Initial Flake Setup
|
|
|
|
Create a new folder and create your `flake.nix` file with the basic inputs. Tell it to
|
|
install the Python 3 package and a shell to use.
|
|
```nix
|
|
{
|
|
description = "A Python development environment";
|
|
inputs = {
|
|
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
|
flake-utils.url = "github:numtide/flake-utils";
|
|
};
|
|
|
|
outputs = {
|
|
self,
|
|
flake-utils,
|
|
nixpkgs
|
|
}: flake-utils.lib.eachDefaultSystem (system: let
|
|
pkgs = nixpkgs.legacyPackages.${system};
|
|
in {
|
|
devShell = pkgs.mkShell {
|
|
nativeBuildInputs = [ pkgs.bashInteractive ];
|
|
buildInputs = [
|
|
pkgs.python3
|
|
];
|
|
};
|
|
});
|
|
}
|
|
```
|
|
Enter the development environment with the command `nix develop`. This will put you into
|
|
a shell with the Nix packages defined in the `flake.nix` file. Of course, if you add
|
|
flake.nix and the generated flake.lock files to your repository, you can be sure that
|
|
everyone else who works on a project is able to use the exact same set of packages across
|
|
their systems.
|
|
|
|
Personally, I use direnv tied into my shell to automatically activate my devshells,
|
|
but for the purposes of this exercise I am going to use the `nix develop` command to
|
|
build and test the environments. Setting up direnv and distributing Nix flakes is beyond
|
|
the scope of this post
|
|
|
|
With the current version of Nix unstable, I get the following output:
|
|
```terminal
|
|
[greg@jude:~/tmp]$ python3 --version
|
|
Python 3.10.11
|
|
```
|
|
|
|
# Add Python Packages
|
|
|
|
When developing with Python, it is common to use a number of Python packages. Nixpkgs comes with
|
|
a huge number of them already packaged and it is easy to package more for it. For the purpose of
|
|
demonstration, I will only add one packge to my installation, but you can add any Python packages
|
|
that you need. To do this, we are going to call the `withPackages` function on the Python
|
|
package and give it a function that returns the packages we want added. That function will look
|
|
like this
|
|
```nix
|
|
packages = python-packages: with python-packages; [
|
|
requests
|
|
];
|
|
```
|
|
Similarly we invoke the function by changing
|
|
```diff
|
|
-pkgs.python3
|
|
+(pkgs.python3.withPackages packages)
|
|
```
|
|
|
|
When that is added to the Flake file, the whole file looks like this
|
|
```nix
|
|
{
|
|
description = "A Python development environment";
|
|
inputs = {
|
|
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
|
flake-utils.url = "github:numtide/flake-utils";
|
|
};
|
|
|
|
outputs = {
|
|
self,
|
|
flake-utils,
|
|
nixpkgs
|
|
}: flake-utils.lib.eachDefaultSystem (system: let
|
|
pkgs = nixpkgs.legacyPackages.${system};
|
|
packages = python-packages: with python-packages; [
|
|
requests
|
|
];
|
|
in {
|
|
devShell = pkgs.mkShell {
|
|
nativeBuildInputs = [ pkgs.bashInteractive ];
|
|
buildInputs = [
|
|
(pkgs.python3.withPackages packages)
|
|
];
|
|
};
|
|
});
|
|
}
|
|
```
|
|
|
|
Now I can run `nix develop` again and get a shell where I can import the `requests` library
|
|
in my Python interpreter.
|
|
|
|
# Add more versions
|
|
|
|
Having one version of Python is good, but the point is to enable working with different versions.
|
|
To do this, we will create a Flake input for each version from Python 3.5 through 3.11 that we
|
|
have available. Creating a separate input, even when several of them will point to the same nixpkgs git
|
|
branch, is necessary because each version will leave support at a different time. We want to be
|
|
able to update the flake in the future and easily leave unsupported versions of Python on their
|
|
latest Nix branch while also not holding back supported versions from their future patch releases.
|
|
We also add a pin for Python 2.7, which is still in use in some legacy servers.
|
|
|
|
So let's add a bunch of inputs like so:
|
|
```nix
|
|
nixpkgs311.url = "github:NixOS/nixpkgs/nixos-unstable";
|
|
nixpkgs310.url = "github:NixOS/nixpkgs/nixos-22.11";
|
|
nixpkgs39.url = "github:NixOS/nixpkgs/nixos-22.11";
|
|
nixpkgs38.url = "github:NixOS/nixpkgs/nixos-22.11";
|
|
nixpkgs37.url = "github:NixOS/nixpkgs/nixos-22.05";
|
|
nixpkgs36.url = "github:NixOS/nixpkgs/nixos-21.05";
|
|
nixpkgs35.url = "github:NixOS/nixpkgs/nixos-20.03";
|
|
nixpkgs27.url = "github:NixOS/nixpkgs/nixos-20.09";
|
|
|
|
```
|
|
|
|
At the time of writing NixOS 22.11 is the latest release and Python 3.8-3.11 are all still in
|
|
support. Therefore, all of them are available in the 22.11 branch for the time being. At the same
|
|
time we do this, we add the new inputs to the output function, update the identification of
|
|
system support, and add the package to the available list.
|
|
|
|
# Full Array {#full-array}
|
|
|
|
Now our flake looks like this:
|
|
```nix
|
|
{
|
|
description = "A Python development environment";
|
|
inputs = {
|
|
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
|
|
flake-utils.url = "github:numtide/flake-utils";
|
|
nixpkgs311.url = "github:NixOS/nixpkgs/nixos-unstable";
|
|
nixpkgs310.url = "github:NixOS/nixpkgs/nixos-22.11";
|
|
nixpkgs39.url = "github:NixOS/nixpkgs/nixos-22.11";
|
|
nixpkgs38.url = "github:NixOS/nixpkgs/nixos-22.11";
|
|
nixpkgs37.url = "github:NixOS/nixpkgs/nixos-22.05";
|
|
nixpkgs36.url = "github:NixOS/nixpkgs/nixos-21.05";
|
|
nixpkgs35.url = "github:NixOS/nixpkgs/nixos-20.03";
|
|
nixpkgs27.url = "github:NixOS/nixpkgs/nixos-20.09";
|
|
};
|
|
|
|
outputs = {
|
|
self,
|
|
flake-utils,
|
|
nixpkgs,
|
|
nixpkgs311,
|
|
nixpkgs310,
|
|
nixpkgs39,
|
|
nixpkgs38,
|
|
nixpkgs37,
|
|
nixpkgs36,
|
|
nixpkgs35,
|
|
nixpkgs27,
|
|
}: flake-utils.lib.eachDefaultSystem (system: let
|
|
pkgs = nixpkgs.legacyPackages.${system};
|
|
pkgs311 = nixpkgs311.legacyPackages.${system};
|
|
pkgs310 = nixpkgs310.legacyPackages.${system};
|
|
pkgs39 = nixpkgs39.legacyPackages.${system};
|
|
pkgs38 = nixpkgs38.legacyPackages.${system};
|
|
pkgs37 = nixpkgs37.legacyPackages.${system};
|
|
pkgs36 = nixpkgs36.legacyPackages.${system};
|
|
pkgs35 = nixpkgs35.legacyPackages.${system};
|
|
pkgs27 = nixpkgs27.legacyPackages.${system};
|
|
packages = python-packages: with python-packages; [
|
|
requests
|
|
];
|
|
in {
|
|
devShell = pkgs.mkShell {
|
|
nativeBuildInputs = [ pkgs.bashInteractive ];
|
|
buildInputs = [
|
|
(pkgs.python3.withPackages packages)
|
|
(pkgs311.python311.withPackages packages)
|
|
(pkgs310.python310.withPackages packages)
|
|
(pkgs39.python39.withPackages packages)
|
|
(pkgs38.python38.withPackages packages)
|
|
(pkgs37.python37.withPackages packages)
|
|
(pkgs36.python36.withPackages packages)
|
|
(pkgs35.python35.withPackages packages)
|
|
(pkgs27.python27.withPackages packages)
|
|
];
|
|
};
|
|
});
|
|
}
|
|
```
|
|
|
|
We can test that they are all present by doing a `nix develop`. This build might take a long time the
|
|
first time you run it on your machine, because older packages are not available in the binary
|
|
cache for NixOS. Fortunately, all Nix packages include the entire history of how to build them from
|
|
source, so you CAN get it done. It just might take a while. Let's run a tiny Bash script to check
|
|
that we have all the Python versions and that they all have the Requests library available:
|
|
|
|
```bash
|
|
for ver in $(seq 5 11); do
|
|
python3.${ver} -c 'import sys, requests; print(sys.version_info)'
|
|
done
|
|
```
|
|
|
|
Sample output:
|
|
```terminal
|
|
$ for ver in $(seq 5 11); do python3.${ver} -c 'import requests, sys; print(sys.version_info)'; done
|
|
sys.version_info(major=3, minor=5, micro=9, releaselevel='final', serial=0)
|
|
sys.version_info(major=3, minor=6, micro=14, releaselevel='final', serial=0)
|
|
sys.version_info(major=3, minor=7, micro=16, releaselevel='final', serial=0)
|
|
sys.version_info(major=3, minor=8, micro=16, releaselevel='final', serial=0)
|
|
sys.version_info(major=3, minor=9, micro=16, releaselevel='final', serial=0)
|
|
sys.version_info(major=3, minor=10, micro=11, releaselevel='final', serial=0)
|
|
sys.version_info(major=3, minor=11, micro=3, releaselevel='final', serial=0)
|
|
```
|
|
|
|
Likewise we can validate that Python 2.7 is available:
|
|
```terminal
|
|
$ python2.7 -c 'import requests, sys; print(sys.version_info)'
|
|
sys.version_info(major=2, minor=7, micro=18, releaselevel='final', serial=0)
|
|
```
|
|
|
|
# Python 3.0 to 3.4
|
|
|
|
These versions of Python were released before Nix was its current, modern self. There is not support
|
|
in those Nixpkgs releases for Flakes which makes adding them to the environment a pain. I tried testing
|
|
importing the tarball directly, but the actual builds of the old versions were broken. Versions 3.2-3.4 are in
|
|
the repository but they do not build without major changes which would be beyond the scope of this
|
|
post. Meanwhile Python 3.0 and 3.1 were released before Nix packages even existed and were never
|
|
packaged for the main repository.
|