* Do not commit the .venv file

* Fix packer formatting issues

* I am, once again, asking to fix the cache path

* Fix URI generation for tests

* Prevent prompting during Hyper-V tests

* Add more waits to hopefully build on GHA

* Hyper-V ISO expansion

Expand coverage of Hyper-V ISO to all distros except Debian
Simplify a few steps in the update/upgrade process for VMs
Edit the test script for Windows support

* Fix plugin cache path on Mac

* Point to the correct mirror URL
This commit is contained in:
Greg Hellings
2023-12-07 12:15:48 -06:00
committed by GitHub
parent 634aae8d75
commit ebbc8a7371
34 changed files with 145 additions and 72 deletions
+3 -3
View File
@@ -83,9 +83,9 @@ jobs:
uses: actions/cache/restore@v3
id: cache-restore
with:
path: ~/.config/packer/plugins
path: ${{ runner.os == 'Linux' && '~/.config/packer/plugins' || '/usr/local/bin/github.com' }}
key: ${{ runner.os }}-${{ hashFiles('sources/build.pkr.hcl') }}
- name: Install Packer plugins
shell: bash
run: packer init sources
@@ -94,7 +94,7 @@ jobs:
uses: actions/cache/save@v3
if: always()
with:
path: ~/.config/packer/plugins
path: ${{ runner.os == 'Linux' && '~/.config/packer/plugins' || '/usr/local/bin/github.com' }}
key: ${{ steps.cache-restore.outputs.cache-primary-key }}
- name: Build
+3
View File
@@ -16,3 +16,6 @@ distros/centos-stream/**/*.hcl
__pycache__
*.pyc
*.pyo
packer_cache
.venv
venv
+19 -18
View File
@@ -1,23 +1,24 @@
# Building on Windows
It's probably a good idea to have Windows Terminal installed, if you
don't already. While it's not strictly necessary, it does provide a
much better interface than the standard command prompt interface.
1. Be sure that Hyper-V is installed and enabled on your system
1. Download and install packer.exe to your system, and add the folder it is in to your user's PATH environment variable
1. Download the [Windows ADK](https://learn.microsoft.com/en-us/windows-hardware/get-started/adk-install) being sure to select
at the least "Deployment Tools" from the feature set.
1. To see more details, including a walkthrough, check [this web page](https://www.makeuseof.com/windows-iso-folder-command-prompt/)
1. Be sure to add the install path for the Oscdimg tool to your Windows PATH.
1. On my system, that path is "C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Deployment Tools\amd64\Oscdimg"
1. Open PowerShell (on my system I had to run as administrator), and ensure that you have access to python
1. If you type "python" at the prompt and do not have it installed, it should prompt you to install from the Store
1. Do not install the version downloaded from the Internet unless you know what you're doing
* Get ahead of the curve and create an external switch called `packer-amd64` that connects directly to the external network
1. Install [Chocolatey](https://chocolatey.org/install)
* Use Chocolatey to install packer `choco install packer`
* Use Chooolatey to install the Windows Assessment and Development Kit's CD tooling `choco install windows-adk-oscdimg`
1. Install Python
* From a PowerShell prompt type `python`. If it is not already installed on your system, then your Windows Store will prompt you to install it. This might require you to be in an Administrator terminal
1. Create a venv `python -m venv .venv`
1. Activate it `.\.venv\scripts\Activate.ps1`
1. Install the packages in requirements-windows.txt
1. Install xonsh and PyYAML `pip install xonsh PyYAML`
1. Try a build
1. You can execute directly with the `packer` executable, if you want
1. If you want to wrap up standard options, or build multiple versions/distros at once, use the `build.xsh` file
1. `xonsh build.xsh -d <distro> -b hyperv-iso.amd64 -v`
1. You can substitute out any of the options with `-a` to build "all" targets that match
* You can execute directly with the `packer` executable, if you want
* If you want to wrap up standard options, or build multiple versions/distros at once, use the `build.xsh` file
* `xonsh build.xsh -d <distro> -b hyperv-iso.amd64 -v`
* You can substitute out any of the options with `-a` to build "all" targets that match
### Troubleshooting
@@ -26,7 +27,7 @@ need to download Shell files from the Packer http server (NixOS, Archlinux, and
HTTP server on the host. To rectify this:
1. From the Windows Start/Search type "Firewall"
1. Click "Allow and App Through The Firewall"
1. Click "Allow an App Through The Firewall"
1. Click "Change Settings"
1. Scroll all the way down to where you find the entries for the packer-hyperv plugin
1. Ensure that all such plugin versions have the box on the left and BOTH boxes on the right checked
@@ -40,8 +41,8 @@ to reach it.
1. Open Hyper-V Manager
1. In the right hand side, click "Virtual Switch Manager..."
1. Select the `packer-x86_64` switch
1. Select the `packer-amd64` switch
1. Set connection type to 'External'
1. In your DHCP server, assign a static IP address to MAC address "00:00:de:ad:be:ef"
1. Personally, I use dnsmasq and my subnet is 10.42.2.1/16 for DHCP, so the appropriate config line is `dhcp-host=00:00:de:ad:be:ef,10.42.2.254`
1. The actual IP address you assign is unimportant, and if you need to spoof a different Mac address, edit the file `sources/hyperv-iso.pkr.hcl`
1. Personally, I use dnsmasq and my subnet is 10.42.2.1/16 for DHCP, so the appropriate config line is `dhcp-host=00:00:de:ad:be:ef,10.42.2.254`
1. The actual IP address you assign is unimportant, and if you need to spoof a different Mac address, edit the file `sources/hyperv-iso.pkr.hcl`
+5
View File
@@ -30,6 +30,11 @@ Vagrant.configure("2") do |config|
# # Customize the amount of memory on the VM:
# vb.memory = "1024"
# end
config.vm.provider "hyperv" do |hv|
config.vm.network "private_network", bridge: "packer-amd64"
end
#
# View the documentation for the provider you are using for more
# information on available options.
+2 -2
View File
@@ -6,7 +6,7 @@ set -ex
name="${0%.sh}"
checksums="$(mktemp)"
until curl -f -L -o "${checksums}" "http://mirror.centos.org/centos/8-stream/isos/x86_64/CHECKSUM"; do
until curl -f -L "https://mirrors.edge.kernel.org/centos/8-stream/isos/x86_64/CHECKSUM" > "${checksums}"; do
sleep 1
done
sha="$(grep -e latest-boot "${checksums}" | grep SHA256 | cut -d' ' -f 4)"
@@ -15,7 +15,7 @@ cat << HERPDERP > "${name}"
distro = "centos-stream"
version = "8"
iso = {
url = "http://mirror.centos.org/centos/8-stream/isos/x86_64/CentOS-Stream-8-x86_64-latest-boot.iso"
url = "https://mirrors.edge.kernel.org/centos/8-stream/isos/x86_64/CentOS-Stream-8-x86_64-latest-boot.iso"
checksum = "${sha}"
}
boot_command = [
+10 -1
View File
@@ -61,7 +61,9 @@ open-vm-tools
%addon com_redhat_kdump --disable
%end
%post
%post --erroronfail
dnf update -y
# configure swap to a file
fallocate -l 2G /swapfile
chmod 600 /swapfile
@@ -138,6 +140,13 @@ restorecon -f - <<EOF
/etc/dracut.conf.d/nofloppy.conf
EOF
virt="$(systemd-detect-virt)"
if [ "${virt}" == "microsoft" ]; then
systemctl enable hpervfcopyd
systemctl enable hypervkvpd
systemctl enable hypervvssd
fi
# Rerun dracut for the installed kernel (not the running kernel):
KERNEL_VERSION=$(rpm -q kernel --qf '%{version}-%{release}.%{arch}\n')
dracut -f /boot/initramfs-${KERNEL_VERSION}.img ${KERNEL_VERSION}
-2
View File
@@ -1,6 +1,4 @@
#!/usr/bin/env bash
set -ex
dnf -y update
dnf clean all
+2 -2
View File
@@ -7,7 +7,7 @@ name="${0%.sh}"
baseurl="https://mirror.stream.centos.org/9-stream/BaseOS/x86_64/iso/CentOS-Stream-9-latest-x86_64-boot.iso"
checksums="$(mktemp)"
until curl -f -L -o "${checksums}" "${baseurl}.SHA256SUM"; do
until curl -f -L "${baseurl}.SHA256SUM" > "${checksums}"; do
sleep 1
done
sha="$(awk '/SHA256/ {print $4}' "${checksums}")"
@@ -26,4 +26,4 @@ boot_command = [
"<enter><wait>"
]
cd_files = [ "distros/centos-stream/9/disc/*" ]
HERPDERP
HERPDERP
+10 -1
View File
@@ -70,7 +70,9 @@ open-vm-tools
%addon com_redhat_kdump --disable
%end
%post
%post --erroronfail
dnf update -y
# configure swap to a file
fallocate -l 2G /swapfile
chmod 600 /swapfile
@@ -150,6 +152,13 @@ restorecon -f - <<EOF
/etc/dracut.conf.d/nofloppy.conf
EOF
virt="$(systemd-detect-virt)"
if [ "${virt}" == "microsoft" ]; then
systemctl enable hpervfcopyd
systemctl enable hypervkvpd
systemctl enable hypervvssd
fi
# Rerun dracut for the installed kernel (not the running kernel):
KERNEL_VERSION=$(rpm -q kernel --qf '%{version}-%{release}.%{arch}\n')
dracut -f /boot/initramfs-${KERNEL_VERSION}.img ${KERNEL_VERSION}
+1 -3
View File
@@ -1,6 +1,4 @@
#!/usr/bin/env bash
set -ex
dnf -y update
dnf clean all
dnf clean all
+2
View File
@@ -5,4 +5,6 @@
provider: virtualbox-iso
- arch: amd64
provider: vmware-iso
- arch: amd64
provider: hyperv-iso
9: *supp
+1 -1
View File
@@ -6,7 +6,7 @@ iso = {
}
boot_command = [
"<up><tab><wait>",
"<bs><bs><bs><bs><bs> inst.ks=cdrom:/ks.cfg console=tty0",
"<bs><bs><bs><bs><bs><wait> inst.ks<wait>=cdrom:/ks.cfg<wait> console<wait>=tty0<wait>",
"<enter>"
]
cd_files = ["distros/centos/7/disc/*"]
+9
View File
@@ -34,6 +34,15 @@ libselinux-python
-rdma
%end
%pre
# In Hyper-V this needs a re-do for some reason
virt="$(systemd-detect-virt)"
if [ "${virt}" == "microsoft" ]; then
nmcli c down eth0
nmcli c up eth0
fi
%end
%post --erroronfail
yum -y update
+1 -2
View File
@@ -5,5 +5,4 @@
provider: virtualbox-iso
- arch: amd64
provider: vmware-iso
- arch: amd64
provider: hyperv-iso
# Hyper-V, I have been thus far unable to have it bring up its network automatically
+8 -2
View File
@@ -1,9 +1,7 @@
#!/usr/bin/env bash
set -ex
apt update
apt autoremove -y
apt upgrade -y
# Configure sshd to be faster
sed -i -E -e 's,^.*(UseDNS).*$,\1 No,' /etc/ssh/sshd_config
@@ -11,3 +9,11 @@ echo "PubkeyAcceptedKeyTypes +ssh-rsa" >> /etc/ssh/sshd_config
# Clean up after ourselves
apt-get clean
virt="$(systemd-detect-virt)"
if [ "${virt}" == "microsoft" ]; then
yum install -y hyperv-daemons
systemctl enable hpervfcopyd
systemctl enable hypervkvpd
systemctl enable hypervvssd
fi
+4 -2
View File
@@ -41,7 +41,7 @@ d-i apt-setup/non-free boolean true
d-i apt-setup/contrib boolean true
tasksel tasksel/first multiselect
d-i pkgsel/include string curl openssh-server sudo python3
d-i pkgsel/include string curl openssh-server sudo python3 hyperv-daemons
d-i grub-installer/bootdev string default
@@ -55,4 +55,6 @@ d-i preseed/early_command string
d-i preseed/late_command string \
echo 'Defaults:vagrant !requiretty' > /target/etc/sudoers.d/vagrant; \
echo 'vagrant ALL=(ALL) NOPASSWD: ALL' >> /target/etc/sudoers.d/vagrant; \
chmod 440 /target/etc/sudoers.d/vagrant
chmod 440 /target/etc/sudoers.d/vagrant; \
in-target apt-get update; \
in-target apt-get upgrade -y
+4 -4
View File
@@ -7,10 +7,10 @@ iso = {
boot_command = [
"<esc><wait>",
"auto <wait>",
"DEBIAN_FRONTEND=text <wait>",
"preseed/url<wait>=http://{{.HTTPIP}}<wait>:{{ .HTTPPort }}/preseed.cfg <wait>",
"console=tty0 <wait>",
"--- <wait>net.ifnames=0 <wait>biosdevnames=0 <wait>",
"DEBIAN<wait>_FRONTEND<wait>=text <wait>",
"preseed<wait>/url<wait>=http://{{.HTTPIP}}<wait>:{{ .HTTPPort }}<wait>/preseed<wait>.cfg <wait>",
"console<wait>=tty0 <wait>",
"--- <wait>net<wait>.ifnames<wait>=0 <wait>bios<wait>dev<wait>names=0 <wait>",
"<enter>"
]
http_directory = "distros/debian/11/http/"
+8 -2
View File
@@ -1,9 +1,7 @@
#!/usr/bin/env bash
set -ex
apt update
apt autoremove -y
apt upgrade -y
# Configure sshd to be faster
cat << EOF > /etc/ssh/sshd_config.d/00-vagrant_accept_pubkey.conf
@@ -13,3 +11,11 @@ EOF
# Clean up after ourselves
apt-get clean
virt="$(systemd-detect-virt)"
if [ "${virt}" == "microsoft" ]; then
yum install -y hyperv-daemons
systemctl enable hpervfcopyd
systemctl enable hypervkvpd
systemctl enable hypervvssd
fi
+4 -2
View File
@@ -55,8 +55,10 @@ d-i preseed/early_command string
d-i preseed/late_command string \
echo 'Defaults:vagrant !requiretty' > /target/etc/sudoers.d/vagrant; \
echo 'vagrant ALL=(ALL) NOPASSWD: ALL' >> /target/etc/sudoers.d/vagrant; \
chmod 440 /target/etc/sudoers.d/vagrant; \
sed -i -e 's/ens3/ens5/g' /target/etc/network/interfaces
chmod 440 /target/etc/sudoers.d/vagrant; \
sed -i -e 's/ens3/ens5/g' /target/etc/network/interfaces; \
in-target apt-get update; \
in-target apt-get upgrade -y
#echo '[Match]' >> /target/etc/systemd/network/dhcp.network; \
#echo 'Name=en*' >> /target/etc/systemd/network/dhcp.network; \
+4 -4
View File
@@ -7,10 +7,10 @@ iso = {
boot_command = [
"<esc><wait>",
"auto <wait>",
"DEBIAN_FRONTEND=text <wait>",
"preseed/url<wait>=http://{{.HTTPIP}}<wait>:{{ .HTTPPort }}/preseed.cfg <wait>",
"console=tty0 <wait>",
"--- <wait>net.ifnames=0 <wait>biosdevnames=0 ",
"DEBIAN<wait>_FRONTEND<wait>=text <wait>",
"preseed<wait>/url<wait>=http://{{.HTTPIP}}<wait>:{{ .HTTPPort }}<wait>/preseed.cfg <wait>",
"console<wait>=tty0 <wait>",
"--- <wait>net<wait>.ifnames<wait>=0 <wait>bios<wait>dev<wait>names=0 <wait>",
"<enter>"
]
http_directory = "distros/debian/12/http/"
+8 -2
View File
@@ -1,9 +1,7 @@
#!/usr/bin/env bash
set -ex
apt update
apt autoremove -y
apt upgrade -y
# Configure sshd to be faster
cat << EOF > /etc/ssh/sshd_config.d/00-vagrant_accept_pubkey.conf
@@ -13,3 +11,11 @@ EOF
# Clean up after ourselves
apt-get clean
virt="$(systemd-detect-virt)"
if [ "${virt}" == "microsoft" ]; then
yum install -y hyperv-daemons
systemctl enable hpervfcopyd
systemctl enable hypervkvpd
systemctl enable hypervvssd
fi
+3 -1
View File
@@ -56,4 +56,6 @@ d-i preseed/early_command string
d-i preseed/late_command string \
echo 'Defaults:vagrant !requiretty' > /target/etc/sudoers.d/vagrant; \
echo 'vagrant ALL=(ALL) NOPASSWD: ALL' >> /target/etc/sudoers.d/vagrant; \
chmod 440 /target/etc/sudoers.d/vagrant
chmod 440 /target/etc/sudoers.d/vagrant; \
in-target apt-get update; \
in-target apt-get upgarde -y
+1 -1
View File
@@ -6,7 +6,7 @@ iso = {
}
boot_command = [
"<up>e<down><down><end> ",
"net.ifnames=0 biosdevnames=0 inst.ks=cdrom:/x86_64.ks console=tty0<f10>"
"net.if<wait>names=0<wait> biosdev<wait>names=0<wait> inst.ks=<wait>cdrom:/x86_64.ks<wait> console<wait>=tty0<f10>"
]
cd_files = [
"distros/fedora/39/*.ks"
+1 -1
View File
@@ -35,7 +35,7 @@ iso = {
}
boot_command = [
"<up>e<down><down><end><bs><bs><bs><bs><bs>",
"net.ifnames=0 biosdevnames=0 inst.ks=cdrom:/x86_64.ks console=tty0 console=ttyS0<f10>"
"net.ifnames=0<wait> biosdevnames=0<wait> inst.ks=cdrom:/x86_64.ks<wait> console=tty0<wait> console=<wait>ttyS0<f10>"
]
cd_files = [
"distros/fedora/rawhide/*.ks"
+4
View File
@@ -5,8 +5,12 @@
provider: virtualbox-iso
- arch: amd64
provider: vmware-iso
- arch: amd64
provider: hyperv-iso
38: *supp
39: *supp
rawhide:
- arch: amd64
provider: qemu
- arch: amd64
provider: hyperv-iso
+2 -2
View File
@@ -31,9 +31,9 @@ iso = {
boot_command = [
"<tab><wait> net.ifnames=0<wait> <wait>bi<wait>osdevnames=0<enter>",
"<wait60>", # Wait for system to come up?
"curl -o /tmp/install.sh http://{{ .HTTPIP }}:{{ .HTTPPort }}/install.sh<enter>",
"curl<wait> -o<wait> /tmp/install.sh<wait> http://<wait>{{ .HTTPIP }}<wait>:{{ .HTTPPort }}<wait>/install.sh<enter>",
"<wait>",
"sudo bash /tmp/install.sh {{ .HTTPPort }} {{ .HTTPIP }}<enter>"
"sudo<wait> bash<wait> /tmp/install.sh<wait> {{ .HTTPPort }}<wait> {{ .HTTPIP }}<enter>"
]
http_directory = "distros/nixos/23.05/http/"
boot_wait = "5s"
+3 -3
View File
@@ -30,10 +30,10 @@ iso = {
}
boot_command = [
"<tab><wait> net.ifnames=0<wait> <wait>bi<wait>osdevnames=0<enter>",
"<wait60>", # Wait for system to come up?
"curl -o /tmp/install.sh http://{{ .HTTPIP }}:{{ .HTTPPort }}/install.sh<enter>",
"<wait120>", # Wait for system to come up?
"curl<wait> -o<wait> /tmp/install.sh<wait> http://<wait>{{ .HTTPIP }}<wait>:{{ .HTTPPort }}/<wait>install.sh<enter>",
"<wait>",
"sudo bash /tmp/install.sh {{ .HTTPPort }} {{ .HTTPIP }}<enter>"
"sudo<wait> bash<wait> /tmp/install.sh<wait> {{ .HTTPPort }}<wait> {{ .HTTPIP }}<wait><enter>"
]
http_directory = "distros/nixos/23.05/http/"
boot_wait = "5s"
+2
View File
@@ -5,4 +5,6 @@
provider: virtualbox-iso
- arch: amd64
provider: vmware-iso
- arch: amd64
provider: hyperv-iso
23.11: *supp
+3 -3
View File
@@ -7,10 +7,10 @@ iso = {
boot_command = [
"e<down><down><down><end><bs><bs><bs>",
"console=tty0 <wait>",
"fsck.mode=skip <wait>",
"locale=en_US <wait>",
"fsck.mode<wait>=skip <wait>",
"locale=<wait>en_US <wait>",
"auto=true <wait>",
"priority=critical <wait>",
"priority<wait>=critical <wait>",
"autoinstall<wait>",
"<F10>",
]
+2
View File
@@ -5,6 +5,8 @@
provider: virtualbox-iso
- arch: amd64
provider: vmware-iso
- arch: amd64
provider: hyperv-iso
22.04: *supp
"23.10": *supp
#18.04: *supp # EOL
+2
View File
@@ -26,3 +26,5 @@ if [ -n "${swap_device}" ] && [ -e "${swap_device}" ]; then
sudo dd if=/dev/zero of="$swap_device" bs=1M || :
sudo /sbin/mkswap ${swap_device_label:+-L "$swap_device_label"} ${swap_device_uuid:+-U "$swap_device_uuid"} "$swap_device"
fi
sleep 15 # Hyper-v seems to be super fast during boot
+1 -1
View File
@@ -17,7 +17,7 @@ packer {
source = "github.com/hashicorp/vmware"
}
hyperv = {
version = "= 1.1.1"
version = "= 1.1.3"
source = "github.com/hashicorp/hyperv"
}
virtualbox = {
+8 -5
View File
@@ -11,12 +11,15 @@ source "hyperv-iso" "amd64" {
ssh_password = local.ssh.password
ssh_handshake_attempts = local.ssh.handshake_attempts
ssh_private_key_file = local.ssh.private_key_file
ssh_host = "10.42.2.254"
vm_name = "packer-${local.name}"
disk_size = "${local.disk_size}000"
cpus = local.cpus
memory = local.memory
headless = var.headless
vm_name = "packer-${local.name}"
disk_size = "${local.disk_size}000"
cpus = local.cpus
memory = local.memory
headless = var.headless
switch_name = "packer-amd64"
mac_address = "0000deadbeef"
http_content = var.http_files
http_directory = var.http_directory
+5 -2
View File
@@ -5,6 +5,7 @@ import pathlib
import re
import sys
import time
from shutil import rmtree
def get_vagrant_provider(provider):
@@ -62,7 +63,7 @@ def main():
p = pathlib.Path(box)
with open("Vagrantfile.in", "r") as fr:
content = fr.read()
content = content.replace("@@BOX@@", f"file:///home/greg/src/vms/vms/{box}")
content = content.replace("@@BOX@@", p.resolve().as_uri())
content = content.replace("@@BOX_NAME@@", p.name)
with open("Vagrantfile", "w") as fw:
fw.write(content)
@@ -73,7 +74,9 @@ def main():
print(f"Now whatcha wanna go and do that for? {box}")
continue
vagrant destroy -f
rm -rf .vagrant
vag_folder = pathlib.Path(".vagrant")
if vag_folder.exists():
rmtree(vag_folder)
if __name__ == '__main__':