Guix vs QubesOS Xen Build

¶ Configure

¶ Guix

#:configure-flags
#~(list "--enable-rpath"
        "--disable-qemu-traditional" ; tries to "git clone"
        "--disable-rombios"       ; tries to "git clone" via etherboot
        ;; TODO: Re-enable stubdom (it's "more secure" to use it).
        "--disable-stubdom"    ; tries to "git clone" old patched newlib
        (string-append "--with-initddir=" #$output "/etc/init.d")
        (string-append "--with-system-qemu="
                       (search-input-file %build-inputs
                                          "bin/qemu-system-i386"))
        (string-append "--with-system-seabios="
                       (search-input-file %build-inputs
                                          "share/firmware/bios.bin"))
        (string-append "--with-system-ovmf="
                       (search-input-file %build-inputs
                                          "share/firmware/ovmf_ia32.bin")))

¶ QubesOS

%define build_qemutrad %{?_with_qemutrad: 1} %{?!_with_qemutrad: 0}
%define build_ovmf %{?_without_ovmf: 0} %{?!_without_ovmf: 1}
...
%if %build_qemutrad
CONFIG_EXTRA="--enable-qemu-traditional"
%else
CONFIG_EXTRA=""
%endif
%if %build_ovmf
CONFIG_EXTRA="$CONFIG_EXTRA --with-system-ovmf=%{_libexecdir}/%{name}/boot/ovmf.bin"
%endif
%ifnarch armv7hl aarch64
CONFIG_EXTRA="$CONFIG_EXTRA --with-system-ipxe=/usr/share/ipxe/10ec8139.rom"
%endif
%if %(test -f /usr/share/seabios/bios-256k.bin && echo 1|| echo 0)
CONFIG_EXTRA="$CONFIG_EXTRA --with-system-seabios=/usr/share/seabios/bios-256k.bin"
%else
CONFIG_EXTRA="$CONFIG_EXTRA --disable-seabios"
%endif
./configure --prefix=%{_prefix} --libdir=%{_libdir} --libexecdir=%{_libexecdir} --with-system-qemu=/usr/bin/qemu-system-i386 --with-linux-backend-modules="xen-evtchn xen-gntdev xen-gntalloc xen-blkback xen-netback xen-pciback xen-scsiback xen-acpi-processor" $CONFIG_EXTRA
...
# BEGIN QUBES SPECIFIC PART
%ifnarch armv7hl aarch64
#CONFIG_EXTRA="$CONFIG_EXTRA --with-system-ipxe=/usr/share/ipxe"
CONFIG_EXTRA="$CONFIG_EXTRA --disable-ipxe --disable-rombios"
CONFIG_EXTRA="$CONFIG_EXTRA --disable-pvshim"
%endif
CONFIG_EXTRA="$CONFIG_EXTRA --with-system-qemu=/usr/bin/qemu-system-x86_64"
export PATH="/usr/bin:$PATH"
autoreconf -i
# END QUBES SPECIFIC PART

¶ Discussion

The Qubes-specific part modifies CONFIG_EXTRA after the ./configure line. I am not familiar with RPM or autotools, so I assume that this is correct for some reason. Perhaps autoreconf picks up on it.

• Guix-Only Flags

  • --enable-rpath: Per Xen's INSTALL file, this fixes problems that occur when using a non-standard prefix. Should be used.

  • --disable-stubdom: Guix has a note to re-enable this. Unclear why it is disabled. IIUC, driver domains in QubesOS are stubdom. This should not cause problems for a normal AppVM running Guix. Should be used temporarily.

  • --with-initddir: Guix sets this to the correct store location. Unclear if other packages add to this directory, and if so how that should be handled. Should be used.

• Qubes-Only Flags

  • --with-linux-backend-modules: Looks like kernel modules. Probably not the best way to handle this in Guix (should be part of an <operating-system-fragment>). Should be used temporarily.

  • --disable-pvshim: This disables a mitigation for meltdown. QubesOS opts to disable hyperthreading. Probably gives better performance for PV VMs, though I'm not sure what realistic use-case still exists for those (driver domains are HVM). Should be used.

  • --disable-ipxe: Based on SUPPORT.md in the Xen repository, it seems likely that this is for guest VMs. It might be for the host hardware. In either case it makes sense to disable: for the guest there is no need and it increases the attack surface; for the host, it seems unwise to assume that every network that the host will connect to is trusted. Should be used.

• Conflicting Flags

  • --with-sytem-qemu: Guix sets this to a binary with the suffix i386, Qubes to x86_64. From Xen wiki, the architecture doesn't actually matter for Xen's usage of QEMU, and i386 is the only one they test. The commit that adds x86_64 in the QubesOS package (1e25a131f1fa12ee3589143ae725313deafc4428) says that the intent is to avoid building QEMU because it's not used in dom0. So it sounds like there's no particular reason to prefer x86_64 on QubesOS, and i386 is the most sensible default. Use i386.
  • --with-system-seabios: Guix sets this to bios.bin, Qubes to bios-256k.bin. Unclear if this is a meaningful difference.
  • --with-system-ovmf: Guix sets this to share/firmware/ovmf_ia32.bin for some unspecified build input, Qubes sets this to /usr/libexec/xen/boot/ovmf.bin, but only if build_ovmf is enabled. The recent build log (see below) does include it. QubesOS does not officially support UEFI on VMs, although it does "just work" for at least some simple configurations. Keep the Guix value.

• Shared Flags

  • --disable-rombios: Guix disables because it tries to git clone. Unclear why Qubes disables. INSTALL states that this is only used for old QEMU, which I don't think I care about - it is only needed for VMs created on old QEMU, and upstreamed QEMU has been default since Xen 4.3.

  • --disable-qemu-traditional: Guix disables because it tries to git clone. This flag does not appear in xen.spec.in, but it is present in the recent build log (see below).

¶ Compile (package definitions)

¶ Guix

#:make-flags
#~(list "XEN_BUILD_DATE=Thu Jan  1 01:00:01 CET 1970"
        "XEN_BUILD_TIME=01:00:01"
        "XEN_BUILD_HOST="
        "ETHERBOOT_NICS="
        "SMBIOS_REL_DATE=01/01/1970"
        "VGABIOS_REL_DATE=01 Jan 1970"
        ;; QEMU_TRADITIONAL_LOC
        ;; QEMU_UPSTREAM_LOC
        "SYSCONFIG_DIR=/tmp/etc/default"
        (string-append "BASH_COMPLETION_DIR=" #$output
                       "/etc/bash_completion.d")
        (string-append "BOOT_DIR=" #$output "/boot")
        (string-append "DEBUG_DIR=" #$output "/lib/debug")
        (string-append "EFI_DIR=" #$output "/lib/efi")
        "MINIOS_UPSTREAM_URL=")
...
#:phases
#~(modify-phases %standard-phases
    (add-after 'unpack-mini-os 'patch
      (lambda* (#:key inputs #:allow-other-keys)
        (substitute* "tools/firmware/Rules.mk"
          (("override XEN_TARGET_ARCH = x86_32" match)
           (string-append match "\noverride CC = "
                          (search-input-file inputs
                                             "bin/i686-linux-gnu-gcc")))
          (("^CFLAGS =$" match)
           (string-append match " -I" (assoc-ref inputs "cross-libc")
                          "/include\n")))
        (substitute* "config/x86_32.mk"
          (("(CFLAGS += )-m32 -march=i686" _ match)
           (string-append match "-march=i686 -I"
                          (assoc-ref inputs "cross-libc") "/include")))
        ;; /var is not in /gnu/store, so don't try to create it.
        (substitute* '("tools/Makefile"
                       "tools/xenstore/Makefile"
                       "tools/xenpaging/Makefile")
          (("\\$\\(INSTALL_DIR\\) .*XEN_(DUMP|LOG|RUN|LIB|PAGING)_DIR.*")
           "\n")
          (("\\$\\(INSTALL_DIR\\) .*XEN_(RUN|LIB)_STORED.*") "\n"))
        ;; Prevent xen from creating /etc.
        (substitute* "tools/examples/Makefile"
          ((" install-(configs|readmes)") ""))))
    (replace 'build
      (lambda* (#:key make-flags parallel-build? #:allow-other-keys)
        (apply invoke "make" "world"
               "-j" (number->string
                     (if parallel-build? (parallel-job-count) 1))
               make-flags)))

¶ Qubes

EXTRA_CFLAGS_XEN_TOOLS="$RPM_OPT_FLAGS $LDFLAGS"
%if 0%{?fedora} >= 37
EXTRA_CFLAGS_XEN_TOOLS="$EXTRA_CFLAGS_XEN_TOOLS -Wno-error=use-after-free"
%endif
export EXTRA_CFLAGS_XEN_TOOLS
# END QUBES SPECIFIC PART
export EXTRA_CFLAGS_QEMU_TRADITIONAL="$RPM_OPT_FLAGS"
export EXTRA_CFLAGS_QEMU_XEN="$RPM_OPT_FLAGS"
export PYTHON="%{__python3}"
export LDFLAGS_SAVE=`echo $LDFLAGS | sed -e 's/-Wl,//g' -e 's/,/ /g' -e 's? -specs=[-a-z/0-9]*??g'`
...
%if %build_hyp
# QUBES SPECIFIC LINE
export CFLAGS=`echo $CFLAGS | sed -e 's/-specs=\/usr\/lib\/rpm\/redhat\/redhat-annobin-cc1//g'`
%if %build_crosshyp
export CFLAGS=`echo $CFLAGS | sed -e 's/-m32//g' -e 's/-march=i686//g' 's/-specs=\/usr\/lib\/rpm\/redhat\/redhat-annobin-cc1//g'`
XEN_TARGET_ARCH=x86_64 %make_build %{?efi_flags} prefix=/usr xen CC="/usr/bin/x86_64-linux-gnu-gcc"
%else
%ifarch armv7hl
export CFLAGS=`echo $CFLAGS | sed -e 's/-mfloat-abi=hard//g' -e 's/-march=armv7-a//g'`
%endif
# armv7hl aarch64 or x86_64
%make_build %{?efi_flags} prefix=/usr xen
%endif
%endif

¶ Discussion

The guix package builds the "world" target.

The QubesOS package builds:

• if %build_hyp

  • xen
• tools

• if %build_docs

  • docs

• if %build_stubdom && ifnarch armv7hl aarch64

  • mini-os-dir
  • -C stubdom build
  • install-stubdom

• ifarch x86_64

  • -C stubdom pv-grub-if-enabled

I'm not concerned about the #:make-flags. The Qubes configuration seems mostly concerned with warnings, architecture, and adding in $RPM_OPT_FLAGS. I'm not concerned about the first and the second is already a TODO item. The third sounds non-urgent because optimizations are something to worry about after the thing is working well, but some optimization flags are actually hardening flags (I guess you could say it's optimizing for security, instead of speed or size).

My confidence in reading these is low. In part because I am not familiar with RPM and autotools, in part because at least some of the compiler flags are set by the Xen configuration.

¶ Compile (build logs)

¶ Guix

From a build log of the qubes-xen package generated locally.

gcc
    ; Misc
    -m64
    -pthread
    -std=gnu99
    -c /tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/libacpi/build.c
    -o build.o

    ; Uninvestigated
    -MMD
    -MP
    -MF .build.o.d

    ; Definitions
    -DBUILD_ID
    -DCONFIG_PCI_SUPP_LEGACY_IRQ
    -D_LARGEFILE_SOURCE
    -D_LARGEFILE64_SOURCE
    -DLIBACPI_STDUTILS=\"/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/libxl_x86_acpi.h\"
    -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__
    -D__XEN_TOOLS__

    ; Includes
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/include
    -I/gnu/store/3lfhjyimi6fwnjzk198rc37s6pwngiim-libnl-3.5.0/include/libnl3
    -include
    /tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../../tools/config.h
    -I.

    ; Optimizations
    -O2
    -ffile-prefix-map=/tmp/guix-build-qubes-xen-4.17.3.drv-0/source/tools/libs/light/../../..=.
    -fomit-frame-pointer
    -fno-strict-aliasing

    ; Warnings
    -Wall
    -Werror

    -Wformat-nonliteral
    -Wmissing-declarations
    -Wmissing-prototypes
    -Wno-format-zero-length
    -Wno-unused-but-set-variable
    -Wno-unused-local-typedefs
    -Wshadow
    -Wstrict-prototypes

¶ QubesOS

From log_2024-01-23_04-41-50 line 10304.

gcc
    ; Misc
    -fPIC
    -m64
    -mtune=generic
    -pthread
    -std=gnu99
    -c /builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libacpi/build.c
    -o build.o

    ; Uninvestigated
    -MMD
    -MP
    -MF .build.o.d
    -specs=/usr/lib/rpm/redhat/redhat-hardened-cc1
    -specs=/usr/lib/rpm/redhat/redhat-annobin-cc1

    ; Debugging
    -g
    -grecord-gcc-switches
    -pipe

    ; Definitions
    -DBUILD_ID
    -DLIBACPI_STDUTILS=\"/builddir/build/BUILD/xen-4.14.6/tools/libxl/libxl_x86_acpi.h\"
    -D_LARGEFILE_SOURCE
    -D_LARGEFILE64_SOURCE
    -D__XEN_INTERFACE_VERSION__=__XEN_LATEST_INTERFACE_VERSION__
    -D__XEN_TOOLS__

    -Wp,-D_FORTIFY_SOURCE=2
    -Wp,-D_GLIBCXX_ASSERTIONS

    ; Optimization
    -O2

    -fasynchronous-unwind-tables
    -fcf-protection
    -ffile-prefix-map=/builddir/build/BUILD/xen-4.14.6/tools/libxl/../..=.
    -fexceptions
    -fomit-frame-pointer
    -fstack-clash-protection
    -fstack-protector-strong
    -fno-strict-aliasing

    ; Includes
    -I.
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/toollog/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/toolcore/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/evtchn/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libxc/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/toollog/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/foreignmemory/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/devicemodel/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libxc/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/evtchn/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/foreignmemory/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/libs/hypfs/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/xenstore/include
    -I/builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/include
    -I/usr/include/libnl3
    -include /builddir/build/BUILD/xen-4.14.6/tools/libxl/../../tools/config.h

    ; Warnings
    -Wall
    -Werror
    -Werror=format-security

    -Wdeclaration-after-statement
    -Wformat-nonliteral
    -Wmissing-declarations
    -Wshadow
    -Wstrict-prototypes
    -Wno-error=declaration-after-statement
    -Wno-format-zero-length
    -Wno-unused-but-set-variable
    -Wno-unused-local-typedefs
    -Wno-declaration-after-statement

¶ Discussion

Putting warning and include flags aside, there are 5 flags which QubesOS uses and Guix does not. There are no flags that Guix uses and QubesOS does not. I do not intend to add any of the missing flags at this time. They are: fPIC, mtune=generic, nostdinc, specs=/usr/lib/rpm/redhat/redhat-hardened-cc1, and specs=/usr/lib/rpm/redhat/redhat-annobin-cc1.

fPIC can be added if needed, but Guix does not use it by default. There are currently no packages that depend on Xen, so this might be omitted just because it is not needed. If so, it can be added later.

mtune=generic is the default, no need to re-specify it.

nostdinc deals with search paths for include files. Guix uses a peculiar directory structure and sets up builds to follow it correctly. Adding this flag in Guix seems risky and it's ot clear that it is needed, so leave it out unless a specific reason to add it emerges.

The hardened spec adds -fPIE if there is no conflicting flag (such as -fPIC). This flag does not appear anywhere in the QubesOS build log. It seems likely that the reason QubesOS specifies -fPIC and Guix does not is to work aruond this spec.

The annobin spec adds a GCC plugin which annotates binaries. Comparing the output of readelf --notes --wide xen-access on the Guix build and the QubesOS build show that they both have .note.gnu.property and .note.ABI-tag, and the contents of these sections looks similar. The QubesOS build has 2 additional properties, .note.gnu.build-id and .gnu.build.attributes. The build id is the store path in guix, so that doesn't seem useful. The only build attribute included on my machine is the annobin version which... also does not seem useful in this case.

¶ Output Files

Legend:

┌───┬─────────────────────────────┐
│ + │ Same file in same location  │
├───┼─────────────────────────────┤
│ - │ Only exists in one project  │
├───┼─────────────────────────────┤
│ = │ Same file in other location │
└───┴─────────────────────────────┘

¶ Guix

.
├── bin
│   ├── + vchan-socket-proxy
│   ├── + xenalyze
│   ├── + xencons
│   ├── + xencov_split
│   ├── + xen-cpuid
│   ├── + xen-detect
│   ├── + xenstore
│   ├── + xenstore-chmod
│   ├── + xenstore-control
│   ├── + xenstore-exists
│   ├── + xenstore-list
│   ├── + xenstore-ls
│   ├── + xenstore-read
│   ├── + xenstore-rm
│   ├── + xenstore-watch
│   ├── + xenstore-write
│   └── + xentrace_format
├── boot
│   ├── + xen-4.17.3.config
│   ├── + xen-4.17.3.gz
│   ├── xen-4.17.gz -> xen-4.17.3.gz
│   ├── xen-4.gz -> xen-4.17.3.gz
│   └── xen.gz -> xen-4.17.3.gz
├── etc
│   ├── bash_completion.d
│   │   └── + xl
│   ├── init.d # not in qubes because systemd?
│   │   ├── - xencommons # contains significant content; contrast to qubes in /etc/sysconfig
│   │   ├── - xendomains
│   │   ├── - xendriverdomain
│   │   └── - xen-watchdog
│   ├── - ld.so.cache # probably don't care that much
│   └── xen
│       └── scripts
│           ├── + block
│           ├── + block-common.sh
│           ├── + block-drbd-probe
│           ├── + block-dummy
│           ├── + block-enbd
│           ├── + block-iscsi
│           ├── + block-nbd
│           ├── + block-tap
│           ├── + colo-proxy-setup
│           ├── + external-device-migrate
│           ├── + hotplugpath.sh
│           ├── + launch-xenstore
│           ├── + locking.sh
│           ├── + logging.sh
│           ├── + remus-netbuf-setup
│           ├── + vif-bridge
│           ├── + vif-common.sh
│           ├── + vif-nat
│           ├── + vif-openvswitch
│           ├── + vif-route
│           ├── + vif-setup
│           ├── + vscsi
│           ├── + xen-hotplug-common.sh
│           ├── + xen-network-common.sh
│           └── + xen-script-common.sh
├── - include # headers probably in a separate package on qubes, not too concerned
├── = lib
│   ├── debug
│   │   ├── + xen-syms-4.17.3
│   │   └── + xen-syms-4.17.3.map
│   ├── lib* # see note in qubes /usr/lib64
│   ├── - pkgconfig
│   ├── python3.10
│   │   └── site-packages
│   │       ├── xen
│   │       │   ├── + __init__.py
│   │       │   ├── lowlevel
│   │       │   │   ├── + __init__.py
│   │       │   │   ├── __pycache__
│   │       │   │   │   └── + __init__.cpython-310.pyc
│   │       │   │   ├── + xc.cpython-310-x86_64-linux-gnu.so
│   │       │   │   └── + xs.cpython-310-x86_64-linux-gnu.so
│   │       │   ├── migration
│   │       │   │   ├── + __init__.py
│   │       │   │   ├── + legacy.py
│   │       │   │   ├── + libxc.py
│   │       │   │   ├── + libxl.py
│   │       │   │   ├── + public.py
│   │       │   │   ├── __pycache__
│   │       │   │   │   ├── + __init__.cpython-310.pyc
│   │       │   │   │   ├── + legacy.cpython-310.pyc
│   │       │   │   │   ├── + libxc.cpython-310.pyc
│   │       │   │   │   ├── + libxl.cpython-310.pyc
│   │       │   │   │   ├── + public.cpython-310.pyc
│   │       │   │   │   ├── + tests.cpython-310.pyc
│   │       │   │   │   ├── + verify.cpython-310.pyc
│   │       │   │   │   └── + xl.cpython-310.pyc
│   │       │   │   ├── + tests.py
│   │       │   │   ├── + verify.py
│   │       │   │   └── + xl.py
│   │       │   ├── __pycache__
│   │       │   │   ├── + __init__.cpython-310.pyc
│   │       │   │   └── + util.cpython-310.pyc
│   │       │   └── + util.py
│   │       └── xen-3.0-py3.10.egg-info
│   │           ├── + dependency_links.txt
│   │           ├── + PKG-INFO
│   │           ├── + SOURCES.txt
│   │           └── + top_level.txt
│   ├── xen # technically in libexec on Qubes, but lib/xen is symlinked so idc
│   │   ├── bin
│   │   │   ├── + convert-legacy-stream
│   │   │   ├── + depriv-fd-checker
│   │   │   ├── + init-xenstore-domain
│   │   │   ├── + libxl-save-helper
│   │   │   ├── + lsevtchn
│   │   │   ├── + readnotes
│   │   │   ├── + test-cpu-policy
│   │   │   ├── + test-paging-mempool
│   │   │   ├── + test-resource
│   │   │   ├── + test-tsx
│   │   │   ├── + test-xenstore
│   │   │   ├── + verify-stream-v2
│   │   │   ├── + xenconsole
│   │   │   ├── + xenctx
│   │   │   ├── + xendomains
│   │   │   ├── + xen-init-dom0
│   │   │   ├── + xenpaging
│   │   │   └── + xenpvnetboot
│   │   └── boot
│   │       └── + hvmloader
│   └── xenfsimage
│       ├── ext2fs
│       │   └── + fsimage.so
│       ├── fat
│       │   └── + fsimage.so
│       ├── iso9660
│       │   └── + fsimage.so
│       ├── reiserfs
│       │   └── + fsimage.so
│       ├── ufs
│       │   └── + fsimage.so
│       ├── xfs
│       │   └── + fsimage.so
│       └── zfs
│           └── + fsimage.so
├── sbin
│   ├── - flask-get-bool
│   ├── - flask-getenforce
│   ├── - flask-label-pci
│   ├── - flask-loadpolicy
│   ├── - flask-set-bool
│   ├── - flask-setenforce
│   ├── + gdbsx
│   ├── + xen-access
│   ├── + xenbaked
│   ├── + xenconsoled
│   ├── + xencov
│   ├── + xen-diag
│   ├── + xen-hptool
│   ├── + xen-hvmcrash
│   ├── + xen-hvmctx
│   ├── + xenhypfs
│   ├── + xen-kdd
│   ├── + xen-livepatch
│   ├── + xenlockprof
│   ├── + xen-lowmemd
│   ├── + xen-mceinj
│   ├── + xen-memshare
│   ├── + xen-mfndump
│   ├── + xenmon
│   ├── + xenperf
│   ├── + xenpm
│   ├── + xenpmd
│   ├── + xenstored
│   ├── + xentop
│   ├── + xentrace
│   ├── + xentrace_setmask
│   ├── + xentrace_setsize
│   ├── + xen-ucode
│   ├── + xen-vmtrace
│   ├── + xenwatchdogd
│   └── + xl  # qubesos places in bin

NOTE: Removed share/ from above output, it only contains documentation

¶ QubesOS

$ dnf repoquery -l $(dnf list --installed | grep xen | cut -f1 -d' ')
- /boot/efi/EFI/qubes/xen-4.17.2.efi
+ /boot/xen-4.17.2.config
+ /boot/xen-4.17.2.gz
+ /etc/bash_completion.d/xl
- /etc/logrotate.d/xen
# in dom0, just a skeleton file; contrast to guix in /etc/init.d
- /etc/sysconfig/xencommons
- /etc/xen/cpupool
+ /etc/xen/scripts/block
+ /etc/xen/scripts/block-common.sh
+ /etc/xen/scripts/block-drbd-probe
+ /etc/xen/scripts/block-dummy
+ /etc/xen/scripts/block-enbd
+ /etc/xen/scripts/block-iscsi
+ /etc/xen/scripts/block-nbd
+ /etc/xen/scripts/block-tap
+ /etc/xen/scripts/colo-proxy-setup
+ /etc/xen/scripts/external-device-migrate
+ /etc/xen/scripts/hotplugpath.sh
+ /etc/xen/scripts/launch-xenstore
+ /etc/xen/scripts/locking.sh
+ /etc/xen/scripts/logging.sh
+ /etc/xen/scripts/remus-netbuf-setup
+ /etc/xen/scripts/vif-bridge
+ /etc/xen/scripts/vif-common.sh
+ /etc/xen/scripts/vif-nat
+ /etc/xen/scripts/vif-openvswitch
+ /etc/xen/scripts/vif-route
+ /etc/xen/scripts/vif-setup
+ /etc/xen/scripts/vscsi
+ /etc/xen/scripts/xen-hotplug-common.sh
+ /etc/xen/scripts/xen-network-common.sh
+ /etc/xen/scripts/xen-script-common.sh
- /etc/xen/xl.conf
- /etc/xen/xlexample.hvm
- /etc/xen/xlexample.pvhlinux
- /etc/xen/xlexample.pvlinux
+ /usr/bin/vchan-socket-proxy
+ /usr/bin/xenalyze
+ /usr/bin/xencons
+ /usr/bin/xencov_split
+ /usr/bin/xen-cpuid
+ /usr/bin/xen-detect
+ /usr/bin/xenstore
+ /usr/bin/xenstore-chmod
+ /usr/bin/xenstore-control
+ /usr/bin/xenstore-exists
+ /usr/bin/xenstore-list
+ /usr/bin/xenstore-ls
+ /usr/bin/xenstore-read
+ /usr/bin/xenstore-rm
+ /usr/bin/xenstore-watch
+ /usr/bin/xenstore-write
+ /usr/bin/xentrace_format
= /usr/bin/xl # symlink to /sbin/xl, which guix does provide
= /usr/lib64/lib* # They provide the same set of libraries by name. Guix has more symlinks
                  # and static archives, I don't care either way atm.
- /usr/lib64/python3.11/site-packages/grub/ExtLinuxConf.py
- /usr/lib64/python3.11/site-packages/grub/GrubConf.py
- /usr/lib64/python3.11/site-packages/grub/__init__.py
- /usr/lib64/python3.11/site-packages/grub/LiloConf.py
- /usr/lib64/python3.11/site-packages/grub/__pycache__
- /usr/lib64/python3.11/site-packages/grub/__pycache__/ExtLinuxConf.cpython-311.opt-1.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/ExtLinuxConf.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/GrubConf.cpython-311.opt-1.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/GrubConf.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/__init__.cpython-311.opt-1.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/__init__.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/LiloConf.cpython-311.opt-1.pyc
- /usr/lib64/python3.11/site-packages/grub/__pycache__/LiloConf.cpython-311.pyc
+ /usr/lib64/python3.11/site-packages/xen-3.0-py3.11.egg-info/dependency_links.txt
+ /usr/lib64/python3.11/site-packages/xen-3.0-py3.11.egg-info/PKG-INFO
+ /usr/lib64/python3.11/site-packages/xen-3.0-py3.11.egg-info/SOURCES.txt
+ /usr/lib64/python3.11/site-packages/xen-3.0-py3.11.egg-info/top_level.txt
- /usr/lib64/python3.11/site-packages/xenfsimage.cpython-311-x86_64-linux-gnu.so
+ /usr/lib64/python3.11/site-packages/xen/__init__.py
+ /usr/lib64/python3.11/site-packages/xen/lowlevel/__init__.py
- /usr/lib64/python3.11/site-packages/xen/lowlevel/__pycache__/__init__.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/lowlevel/__pycache__/__init__.cpython-311.pyc
+ /usr/lib64/python3.11/site-packages/xen/lowlevel/xc.cpython-311-x86_64-linux-gnu.so
+ /usr/lib64/python3.11/site-packages/xen/lowlevel/xs.cpython-311-x86_64-linux-gnu.so
+ /usr/lib64/python3.11/site-packages/xen/migration/__init__.py
+ /usr/lib64/python3.11/site-packages/xen/migration/legacy.py
+ /usr/lib64/python3.11/site-packages/xen/migration/libxc.py
+ /usr/lib64/python3.11/site-packages/xen/migration/libxl.py
+ /usr/lib64/python3.11/site-packages/xen/migration/public.py
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/__init__.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/__init__.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/legacy.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/legacy.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/libxc.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/libxc.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/libxl.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/libxl.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/public.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/public.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/tests.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/tests.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/verify.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/verify.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/xl.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/__pycache__/xl.cpython-311.pyc
+ /usr/lib64/python3.11/site-packages/xen/migration/tests.py
+ /usr/lib64/python3.11/site-packages/xen/migration/verify.py
+ /usr/lib64/python3.11/site-packages/xen/migration/xl.py
- /usr/lib64/python3.11/site-packages/xen/__pycache__/__init__.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/__pycache__/__init__.cpython-311.pyc
- /usr/lib64/python3.11/site-packages/xen/__pycache__/util.cpython-311.opt-1.pyc
+ /usr/lib64/python3.11/site-packages/xen/__pycache__/util.cpython-311.pyc
+ /usr/lib64/python3.11/site-packages/xen/util.py
+ /usr/lib64/xenfsimage/ext2fs-lib/fsimage.so
+ /usr/lib64/xenfsimage/fat/fsimage.so
+ /usr/lib64/xenfsimage/iso9660/fsimage.so
+ /usr/lib64/xenfsimage/reiserfs/fsimage.so
+ /usr/lib64/xenfsimage/ufs/fsimage.so
+ /usr/lib64/xenfsimage/xfs/fsimage.so
+ /usr/lib64/xenfsimage/zfs/fsimage.so
- /usr/lib/debug/usr/libexec/xen/boot/xen-shim-syms
- /usr/lib/debug/xen-4.17.2.efi.map
+ /usr/lib/debug/xen-syms-4.17.2
+ /usr/lib/debug/xen-syms-4.17.2.map
+ /usr/libexec/xen/bin/convert-legacy-stream
+ /usr/libexec/xen/bin/depriv-fd-checker
+ /usr/libexec/xen/bin/init-xenstore-domain
+ /usr/libexec/xen/bin/libxl-save-helper
+ /usr/libexec/xen/bin/lsevtchn
- /usr/libexec/xen/bin/pygrub # Qubes removes this after building
+ /usr/libexec/xen/bin/readnotes
+ /usr/libexec/xen/bin/test-cpu-policy
+ /usr/libexec/xen/bin/test-paging-mempool
+ /usr/libexec/xen/bin/test-resource
+ /usr/libexec/xen/bin/test-tsx
+ /usr/libexec/xen/bin/test-xenstore
+ /usr/libexec/xen/bin/verify-stream-v2
+ /usr/libexec/xen/bin/xenconsole
+ /usr/libexec/xen/bin/xenctx
+ /usr/libexec/xen/bin/xendomains
+ /usr/libexec/xen/bin/xen-init-dom0
+ /usr/libexec/xen/bin/xenpaging
+ /usr/libexec/xen/bin/xenpvnetboot
+ /usr/libexec/xen/boot/hvmloader
- /usr/libexec/xen/boot/ovmf.bin # guix sets ovmf from a build input
- /usr/libexec/xen/boot/xen-shim
/usr/lib/modules-load.d/xen.conf
# Presumably, the systemd services implement the same functionality as the init.d scripts
# in guix. There is some overlap in the names but it's not a 1:1 match.
- /usr/lib/systemd/system/proc-xen.mount
- /usr/lib/systemd/system/xenconsoled.service
- /usr/lib/systemd/system/xen-init-dom0.service
- /usr/lib/systemd/system/xenstored.service
- /usr/lib/systemd/system/xen-watchdog.service
- /usr/lib/xen
+ /usr/sbin/gdbsx
+ /usr/sbin/xen-access
+ /usr/sbin/xenbaked
+ /usr/sbin/xenconsoled
+ /usr/sbin/xencov
+ /usr/sbin/xen-diag
+ /usr/sbin/xen-hptool
+ /usr/sbin/xen-hvmcrash
+ /usr/sbin/xen-hvmctx
+ /usr/sbin/xenhypfs
+ /usr/sbin/xen-kdd
+ /usr/sbin/xen-livepatch
+ /usr/sbin/xenlockprof
+ /usr/sbin/xen-lowmemd
+ /usr/sbin/xen-mceinj
+ /usr/sbin/xen-memshare
+ /usr/sbin/xen-mfndump
+ /usr/sbin/xenmon
+ /usr/sbin/xenperf
+ /usr/sbin/xenpm
+ /usr/sbin/xenpmd
+ /usr/sbin/xenstored
+ /usr/sbin/xentop
+ /usr/sbin/xentrace
+ /usr/sbin/xentrace_setmask
+ /usr/sbin/xentrace_setsize
+ /usr/sbin/xen-ucode
+ /usr/sbin/xen-vmtrace
+ /usr/sbin/xenwatchdogd
+ /usr/sbin/xl
- /var/lib/xen/dump
- /var/lib/xen/images
- /var/log/xen/console
- /var/run/xenstored

NOTE: Removed /usr/share/ from above output, it only contains documentation
NOTE: Removed listings that end with a directory name
NOTE: Removed .build-id entries

¶ Discussion

I haven't spent too much time digging into these yet, but nothing jumps out to me as an obvious red flag. I don't expect that there are significant differences between the systemd service files in QubesOS and the init.d scripts in Guix, this should just be presenting the same thing differently depending on the environment. But it should be double-checked.

The QubesOS python caches have extra files with .opt, I assume that this stands for "optimization" and can be safely ignored for the time being. The difference might be due to different python versions. Either way I'll look at it when I actually focus on the python bindings, which will be after Guix is booting as PVH.

I'm generally not too concerned about files provided by Guix but not QubesOS. Extra stuff generally causes less problems than missing stuff.