The squashfs images of the TP-Link RE200 v1 and RE210 v1 differ between
builds; the uncompressed kernel of the RE200 one was 271 bytes larger,
from its device tree on:
-00001040: 6d00 0080 00cb 1a9e 0000 0000 0000 006f
+00001040: 6d00 0080 00da 1b9e 0000 0000 0000 006f
The RE210 device tree is exactly 271 bytes larger than the RE200 one, so
the RE200 image carried the kernel of the RE210.
tplink-v1-okli-image puts kernel and rootfs together in
$(IMAGE_ROOTFS).factory or .sysupgrade, a name shared by all devices with
the same rootfs. Both devices are built at the same time, so one can
overwrite the file of the other between cp, cat and mktplinkfw, and an
image can end up with the kernel and device tree of the other device.
Link: https://github.com/openwrt/openwrt/pull/25816
Signed-off-by: Paul Spooren <mail@aparcar.org>
The firmware images of Layerscape boards booting from QSPI, e.g. the
LS1046A-FRWY, differ between builds in 4 bytes at the end of the PBL:
-00009e80: 019f 1e98 0000 0000 0000 0000 0000 0000
+00009e80: e10f e4a8 0000 0000 0000 0000 0000 0000
byte_swap pads the PBL to a multiple of 8 bytes from a 4 byte variable,
so fwrite() reads up to 4 bytes of the stack past it, and an aligned
PBL always gets 8 bytes appended, 4 of them random.
Add a patch that pads with an 8 byte buffer of zeros. The output keeps
its length and layout: for an aligned input only the 4 former random
bytes become zeros, an unaligned input gives the same output as before.
Link: https://github.com/openwrt/openwrt/pull/25720
Signed-off-by: Paul Spooren <mail@aparcar.org>
libopenssl3 differs between builds depending on the environment of the
build host, e.g. in the images of the Turris Omnia:
-compiler: arm-openwrt-linux-muslgnueabi-gcc -fPIC -pthread -Os ...
+compiler: arm-openwrt-linux-muslgnueabi-gcc -fPIC -pthread -Wa,--noexecstack -Os ...
- GNU_STACK ... RWE 0x10
+ GNU_STACK ... RW 0x10
Configure probes whether the assembler supports --noexecstack by running
$CROSS_COMPILE$CC, with CC taken from the environment. A host which sets
CC to select its compiler, e.g. CC=gcc-13, makes it run
arm-openwrt-linux-muslgnueabi-gcc-13, which does not exist, so the flag
is silently dropped and libcrypto ends up with an executable stack.
Pass CC=gcc to Configure, so that it always probes the cross compiler.
With CC=gcc-13 in the environment, the configuration now gets
-Wa,--noexecstack like a build without CC set.
Link: https://github.com/openwrt/openwrt/pull/25719
Signed-off-by: Paul Spooren <mail@aparcar.org>
The compressed LoongArch kernel (vmlinuz.efi) differs between builds in
different directories. The code and data of vmlinux are identical, but
its GNU build ID is not: the debug information of vmlinux holds a
single absolute path, the include directory of the compiler, e.g.
/builder/shared-workdir/build/staging_dir/toolchain-loongarch64_generic_gcc-14.4.0_musl/lib/gcc/loongarch64-openwrt-linux-musl/14.4.0/include
asm/loongarch.h includes the compiler's larchintrin.h, so core files
like arch/loongarch/kernel/cpu-probe.c refer to it. The prefix maps of
the kernel cover the build and the staging directory, but not the
toolchain directory. The build ID hashes all of vmlinux, so it depends
on the build directory, and with it the compressed kernel.
Map the toolchain directory as well, like the staging directory.
On loongarch64/generic, vmlinux built without the map holds this one
absolute path, like the published one; built with it, no absolute path
is left, while .text, .rodata and .data stay byte-identical.
Link: https://github.com/openwrt/openwrt/pull/25648
Signed-off-by: Paul Spooren <mail@aparcar.org>
With CONFIG_TARGET_IMAGES_GZIP enabled, which is the default, ext4
images are gzipped when they are copied to $(BIN_DIR), unless their
name ends with .gz or .dtb. This causes several problems:
- The JSON info contains the uncompressed name and hash, so
profiles.json lists images which don't exist on the download server:
profiles.json: openwrt-at91-sam9x-atmel_at91sam9263ek-ext4-root.ubi
sha256sums: openwrt-at91-sam9x-atmel_at91sam9263ek-ext4-root.ubi.gz
- The image is gzipped as a whole, including the metadata and signature
added by append-metadata. fwtool looks for them at the end of the
file, so sysupgrade rejects these images. Explicitly gzipped images
run append-metadata after gzip instead.
- A rule for the gzipped name is defined for every image, which clashes
with images that are built both plain and gzipped, as it was the case
for malta.
The previous commits removed the ext4 images which can't be used and
gzip the remaining ones explicitly, so drop the implicit gzip. The file
name in $(BIN_DIR) is now always the one from IMAGES. Targets wanting
gzipped images define them explicitly, e.g. sdcard.img.gz, or select
them via CONFIG_TARGET_IMAGES_GZIP like x86, armsr, loongarch64 and
malta. Limit the option to these targets.
The ext4 kernel.bin and factory.bin of ixp4xx, which only contain the
kernel, and the optional ext4 VM images of x86 and armsr are now stored
uncompressed, like their squashfs counterparts.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
Image/Gzip lost its last user with commit df9ffedb75 ("archs38: drop
target") and Image/gzip-ext4-padded-squashfs with commit 8dfa38b82c
("malta: convert to Device macro image building"). The latter also
calls Image/Build/gzip, which isn't defined anywhere.
Remove both, along with Image/pad-root-squashfs, which was only used by
Image/gzip-ext4-padded-squashfs.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
Currently malta defines both `rootfs.img` and `rootfs.img.gz` as IMAGES.
With CONFIG_TARGET_IMAGES_GZIP enabled (the default), the ext4 rootfs.img
is gzipped to the same name, resulting in two recipes for the same file.
Make merges the prerequisites and the `cp` fails, which is hidden by
`.IGNORE`:
cp: target '...squashfs-rootfs.img.gz': No such file or directory
make[4]: [Makefile:32: ...squashfs-rootfs.img.gz] Error 1 (ignored)
Since the JSON info is created from $(KDIR)/tmp, the profiles.json lists
images which don't exist on the download server.
Fix this by selecting either the gzipped or plain image based on
CONFIG_TARGET_IMAGES_GZIP, like it's done for armsr and loongarch64.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
ext4 was added in commit e824864ede ("ixp4xx: Add ext4 and rootfs-part
to features") for machines mounting root from an external hard drive.
The ext4 rootfs.bin is only published gzipped, because the build system
implicitly gzips ext4 images.
In preparation for dropping the implicit gzip, build rootfs.bin for
squashfs only and add rootfs.bin.gz for ext4, which is gzipped
explicitly. The published file names stay the same.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
ext4 was added for the SD card images in commit 1b368f1641 ("at91: Add
ext4 filesystem"), but root.ubi and factory.bin are built for ext4 as
well. These put the ext4 file system into a UBI volume, which can only
be used read-only through ubiblock, so the images are of no use.
Build root.ubi and factory.bin for squashfs and ubifs only, while the
SD card images are still built for ext4.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
Every image is built for all filesystems in FILESYSTEMS. A device that
needs an image only for some of them, e.g. a flash image for squashfs
and a disk image for ext4, can only restrict FILESYSTEMS for the whole
device, which drops its other images as well.
Add FILESYSTEMS/<image> to build an image only for some filesystems of
the device, similar to IMAGE/<image>/<fs> for per filesystem recipes.
Like the recipes, it is not reset between devices, so it should be set
next to the recipe of the image.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
gemini enables ext4 for the hard disk image of the D-Link DNS-313, so
the flash images of the DIR-685 are built for ext4 as well. The ext4
root file system is padded to CONFIG_TARGET_ROOTFS_PARTSIZE, which
makes these images about 108 MiB, while the DIR-685 has 32 MiB of
flash.
The ext4 sysupgrade image is broken anyway: the ext4 images are gzipped
as a whole, which moves the image metadata into the gzip stream, so
sysupgrade rejects it with "Image metadata not present".
Build the DIR-685 images for squashfs only, like the StorLink reference
designs.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
cortexa72 enables ext4 for its eMMC and SD card devices, so the UBI
images of the MikroTik RB5009 are built for ext4 as well. An ext4 file
system in a UBI volume can only be used read-only through ubiblock, so
these images are of no use.
They are broken anyway: the ext4 images are gzipped as a whole, which
moves the metadata of the sysupgrade image into the gzip stream. fwtool
reads the metadata from the end of the file, so sysupgrade rejects the
image with "Image metadata not present".
Build the UBI images for squashfs only.
Link: https://github.com/openwrt/openwrt/pull/25658
Signed-off-by: Paul Spooren <mail@aparcar.org>
UBIFS root file systems differ between builds, e.g. the one of the
Microchip SAMA7G5-EK: the superblock gets a random UUID, the inodes get
the ctime of the build host, which can't be reset with touch, and the
entries of each directory are added in readdir() order, which depends on
the host file system and decides inode numbers and layout.
Add a patch that, when SOURCE_DATE_EPOCH is set, uses a time-based
(version 1) UUID with that time and clamps all inode times to it. The
node ID of the UUID is taken from the new --uuid-node option, which the
image build sets from IMG_PART_DISKGUID, itself derived from
SOURCE_DATE_EPOCH and the kernel vermagic.
Directory entries are added sorted by name. Two trees with the same
contents but different ctimes and readdir() order now give identical
images; without SOURCE_DATE_EPOCH the UUID stays random.
Link: https://github.com/openwrt/openwrt/pull/25430
Signed-off-by: Paul Spooren <mail@aparcar.org>
The boot partition of the DNS-313 factory image differs between builds:
-Linux rev 1.0 ext2 filesystem data, UUID=9f7d7721-0bf7-4483-bd56-b539d195f181
+Linux rev 1.0 ext2 filesystem data, UUID=d3ca4870-378b-48dd-9b05-760812dfeb81
-drwxr-xr-x ... 2026-09-23 07:37:23.000000 ./.boot/
+drwxr-xr-x ... 2026-09-23 09:38:09.000000 ./.boot/
genext2fs copies the mtimes of the staged files from the host and adds
them in readdir() order, which also decides their inode numbers and
blocks. It leaves the UUID empty, so e2fsck fills in a random one, and
e2fsck stamps the last check with the current time.
Pass genext2fs a name sorted tarball with mtimes clamped to
SOURCE_DATE_EPOCH and group/other write bits cleared, derive the UUID
from IMG_PART_DISKGUID and run e2fsck with E2FSCK_TIME set to
SOURCE_DATE_EPOCH.
Link: https://github.com/openwrt/openwrt/pull/25358
Signed-off-by: Paul Spooren <mail@aparcar.org>
The sysupgrade image of the Ctera C200 V1 differs between builds in the
user and group names of the tar members:
--rw-r--r-- 0 buildbot (1000) buildbot (1000) 175 ... header
+-rw-r--r-- 0 rebuilder (1000) rebuilder (1000) 175 ... header
ctera-firmware sets the mtime of the members, but not their owner, so
the name of the build user ends up in the image. mvebu fixed the same
recipe in commit 0d42a1258f ("mvebu: improve reproducibility of ctera
firmware"); store the members as root here too.
Link: https://github.com/openwrt/openwrt/pull/25525
Signed-off-by: Paul Spooren <mail@aparcar.org>
The powerpc kernel of the WD My Book Live differs between builds. The
difference sits in the vDSO, which the kernel embeds with its debug
information as vdso32.so.dbg:
-GNU C11 14.4.0 ... -mstack-protector-guard=tls -mstack-protector-guard-reg=r2 -mstack-protector-guard-offset=872 -m32 -g -O2 ...
+GNU C11 14.4.0 ... -m32 -g -O2 ...
stack_protector_prepare appends these flags to KBUILD_CFLAGS with $(eval)
from its recipe, while vdso_prepare builds the vDSO from a sibling
prerequisite of prepare. Nothing orders the two, so whether the flags
end up in DW_AT_producer depends on the order in which make starts them.
Add a pending patch that makes vdso_prepare depend on
stack_protector_prepare.
Link: https://github.com/openwrt/openwrt/pull/25429
Signed-off-by: Paul Spooren <mail@aparcar.org>
The ext2 boot partition of the WD My Book Live images differs between
builds:
-Linux rev 1.0 ext2 filesystem data, UUID=8474430c-7c37-4000-95a5-1d8f4f0607d1
+Linux rev 1.0 ext2 filesystem data, UUID=4b8e89bb-aee0-4fde-971c-893df8d8fcd1
-drwxr-xr-x 0 1000 1000 0 2026-09-23 08:27:40.000000 ./boot/
+drwxr-xr-x 0 1000 1000 0 2026-09-23 10:17:28.000000 ./boot/
genext2fs copies owner, mode and mtime of the staged files from the host
and adds them in readdir() order, which also decides their inode numbers
and blocks. It leaves the UUID empty, so e2fsck fills in a random one,
and e2fsck stamps the last check with the current time.
Pass genext2fs a name sorted tarball with root ownership, group and
other write permission dropped and mtimes clamped to SOURCE_DATE_EPOCH,
derive the UUID from the file system contents and run e2fsck with
E2FSCK_TIME set to SOURCE_DATE_EPOCH. Remove the staging directory and
the tarball afterwards.
Link: https://github.com/openwrt/openwrt/pull/25357
Signed-off-by: Paul Spooren <mail@aparcar.org>
The factory images of the ZyXEL NWA11xx devices differ between builds
in the tar.bz2 they wrap:
-drwxr-xr-x 0 buildbot (1000) buildbot (1000) 0 2026-09-24 21:39:18 ./
-rw-r--r-- 0 buildbot (1000) buildbot (1000) 3387 2026-09-24 21:39:18 ./vmlinux_mi124_f1e.lzma.uImage
-rw-r--r-- 0 buildbot (1000) buildbot (1000) 8388608 2026-09-24 21:39:18 ./mi124_f1e-jffs2
+drwxr-xr-x 0 rebuilder (1000) rebuilder (1000) 0 2026-09-25 01:03:01 ./
+-rw-r--r-- 0 rebuilder (1000) rebuilder (1000) 8388608 2026-09-25 01:03:01 ./mi124_f1e-jffs2
+-rw-r--r-- 0 rebuilder (1000) rebuilder (1000) 3387 2026-09-25 01:03:01 ./vmlinux_mi124_f1e.lzma.uImage
zyxel-tar-bz2 archives the staging directory as it is, so owner, mtime
and the order of the members come from the build host.
Store the members as root with the mtime set to SOURCE_DATE_EPOCH and
without group or other write permission. List them explicitly, kernel
first, which is the order of the images built so far, instead of
sorting them by name.
Link: https://github.com/openwrt/openwrt/pull/25389
Signed-off-by: Paul Spooren <mail@aparcar.org>
Kernel modules from the packages feed differ between builds on 6.18,
e.g. kmod-v4l2loopback, kmod-jool-netfilter, kmod-pf-ring and
kmod-usb-serial-dmx_usb_module:
-/builder/shared-workdir/build/build_dir/.../linux-6.18.52/include/linux/ucopysize.h
+/tmp/rebuilderd-openwrt.Xebb8P/src/build_dir/.../linux-6.18.52/include/linux/ucopysize.h
These packages call make -C $(LINUX_DIR) M=... without
KERNEL_MAKE_FLAGS, so they get none of the OpenWrt prefix maps. With
6.12 that did not matter, since srctree stayed "." and the kernel
headers were included by a relative path. Since 6.13 srctree holds the
absolute path of the kernel tree for external modules, while kbuild
only maps srcroot, the module directory.
Add a pending patch that maps the kernel tree for external modules too.
It is added before the map of srcroot and before KCFLAGS, and GCC uses
the last matching map, so modules built through KERNEL_MAKE_FLAGS keep
their current output.
Link: https://github.com/openwrt/openwrt/pull/25356
Signed-off-by: Paul Spooren <mail@aparcar.org>
CompileBPF calls clang with -g but without any prefix map. Stripping debug info
keeps .BTF and .BTF.ext, since libbpf needs them, and their line info carries
the absolute path of each source file. This makes BPF objects like qosify-bpf.o
depend on the build directory and breaks reproducibility:
-/builder/shared-workdir/build/build_dir/.../qosify-bpf.c
+/tmp/rebuilderd-openwrt.juCsuQ/src/build_dir/.../qosify-bpf.c
IREMAP_CFLAGS is not enough here: without CONFIG_REPRODUCIBLE_DEBUG_INFO it
only sets -fmacro-prefix-map, which leaves the line info alone, and it is empty
for external toolchains, which have no say over the host clang. Unlike regular
debug info, BTF ships with the package, so always pass -ffile-prefix-map for
the build and staging directories.
Add an fremap helper next to iremap for that, and let IREMAP_STAGING_DIR and
CompileBPF share the map of the staging directory, so they cannot drift apart.
Link: https://github.com/openwrt/openwrt/pull/25359
Signed-off-by: Paul Spooren <mail@aparcar.org>
The APEX boot loader embeds the time of the build in its version
banner, which makes the Iomega NAS100D factory image differ between
builds:
-compiled for OpenWrt NAS100D/BE on 2026.Sep.23-04:17:21
+compiled for OpenWrt NAS100D/BE on 2026.Sep.23-11:06:03
APEX sets BUILDDATE with $(shell date) in its top level Makefile. Pass
BUILDDATE on the command line, formatted from SOURCE_DATE_EPOCH in UTC
and the C locale, which overrides it for all sub-makes.
Link: https://github.com/openwrt/openwrt/pull/25354
Signed-off-by: Paul Spooren <mail@aparcar.org>
The factory images of the Linksys LGS3xxC switches differ between
builds in the Belkin header:
-00000000: 0780 0001 7917 f807 6ab3 0fe3 0058 2000 ....y...j....X .
+00000000: 0780 0001 2a5a 3f79 6ab3 d0c7 0058 2000 ....*Z?yj....X .
belkin-header.py stores time.time() as the image time, which also
changes the header checksum. Use SOURCE_DATE_EPOCH when it is set, like
linksys-image.sh does for the tarball inside.
Link: https://github.com/openwrt/openwrt/pull/25355
Signed-off-by: Paul Spooren <mail@aparcar.org>
This reverts commit 9292f32fd9.
Downstream patches may change the output. If the GCC version doesn't
incooperate the patches somehow (hash, git commit, ...), ccache may not
invalidate existing cache.
Revert this until a cleaner solution is found.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The GL.iNet specific metadata contains a `date`-field which takes the current
timestamp instead of using SOURCE_DATE_EPOCH. This commits make use of the
variable, if available.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Storing the ever changing revision in the GCC binary may result in leakage in
package builds. If two equal GCC versions build the same package version with
different OpenWrt revisions, the package becomes unreproducible.
Solve this by dropping the revision.
Below is an example of the `cni` package:
│ ├── readelf --wide --decompress --string-dump=.comment {}
│ │ @@ -1,4 +1,4 @@
│ │
│ │ String dump of section '.comment':
│ │ - [ 0] GCC: (OpenWrt GCC 14.4.0 r35470-1b2aeb4f8a) 14.4.0
│ │ + [ 0] GCC: (OpenWrt GCC 14.4.0 r35485-0f256a0a7a) 14.4.0
Signed-off-by: Paul Spooren <mail@aparcar.org>
Firmware image signing in append-metadata/append-gl-metadata was conditional
only by the presence of the build key. Add an explicit SIGN_FIRMWARE option
(default enabled except with BUILDBOT) so appending the fwtool signature can be
toggled on its own.
It's disable on buildbot since we use a decentralized building with no
individual keys on builders. Instead, a fake key is currently used, which adds
an insecure signature.
The future of signing firmware and key distribution across the build
infrastructure should be discussed separately.
Link: https://github.com/openwrt/openwrt/pull/24291
Signed-off-by: Paul Spooren <mail@aparcar.org>
Currently assembly code of the Kernel may contain the build path, resulting in
unreproducible firmware images (and Kernels). Fix this by adding a iremap
KAFLAG.
Before:
strings "/path/to/vmlinux" | grep linux-6.18.38
/path/to/openwrt/build_dir/target-x86_64_musl/linux-x86_64/linux-6.18.38
/path/to/openwrt/build_dir/target-x86_64_musl/linux-x86_64/linux-6.18.38
After
strings "/path/to/vmlinux" | grep linux-6.18.38
target-x86_64_musl/linux-x86_64/linux-6.18.38
target-x86_64_musl/linux-x86_64/linux-6.18.38
Link: https://github.com/openwrt/openwrt/pull/24271
Signed-off-by: Paul Spooren <mail@aparcar.org>
This commit statically sets SOURCE_DATE_EPOCH to 0, which causes `apk mkpkg` to
drop the date from generated packages. This improves reproducibility while
leaving package specific SOURCE_DATE_EPOCH untouched. The latter can be
interesting for i.e. packages like busybox, which prominently show that date on
the login shell.
Without this commit, packages that live inside feed repositories (openwrt.git,
packages.git etc) like `mtd` or `attendedsysupgrade-common` would have random
timestamps since no "full" git log was available.
Below an example with different source dates (0, 1):
│ 0755 root:wheel - - /
│ 0755 root:wheel - - lib/
│ 0755 root:wheel - - lib/apk/
│ 0755 root:wheel - - lib/apk/packages/
│ -0644 root:wheel 1970-01-01T00:00:01Z 10 lib/apk/packages/mtd.list
│ +0644 root:wheel - 10 lib/apk/packages/mtd.list
│ 0755 root:wheel - - sbin/
│ -0755 root:wheel 1970-01-01T00:00:01Z 24595 sbin/mtd
│ +0755 root:wheel - 24595 sbin/mtd
Link: https://github.com/openwrt/openwrt/pull/24005
Signed-off-by: Paul Spooren <mail@aparcar.org>
This commit fixes "286f377389a button-hotplug: add KEY_SETUP and KEY_VENDOR
handling" which changed the code without bumping the PKG_RELEASE, resulting in
different binaries under the same version.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Link: https://github.com/openwrt/openwrt/pull/23826
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This commit fixes "a5107ad58c6 mtd: fix buffer leak and fd leak in mtd_dump()"
which changed the code but did not increase the release. This causes two
packages with the same version to have different content and thereby hashes.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Link: https://github.com/openwrt/openwrt/pull/23827
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Published sources archives may contain a mixture of MTIMEs, so taking the time
of the first file found varies based on filesystem order. To fix this, sort all
MTIMEs and take the latest.
Since piping into a file directly creates that file, we also need to
specifically tell `find` to ignore `version.date`.
Furthermore, move this prior to the ./src copy step, since for some packages,
the OpenWrt build system ships extra ./src files, which would introduce the
wrong timestamps again.
Fix: e36c2946b7 build: derive PKG_SOURCE_DATE_EPOCH from the unpacked source tree
Link: https://github.com/openwrt/openwrt/pull/23756
Signed-off-by: Paul Spooren <mail@aparcar.org>
APK compresses it's helptext using LUA and require `zlib`, which isn't
available on the Buildbots. It thens falls back to `gzip`, which embeds the
MTIME, making the binary itself unreproducible.
This commits adds a downstream patch to run `gzip` with `-n`, setting the time
to 0.
Link: https://github.com/openwrt/openwrt/pull/23736
Signed-off-by: Paul Spooren <mail@aparcar.org>
Previously PKG_SOURCE_DATE_EPOCH came from a git log on the OpenWrt
package directory. That fails in shallow feed clones and in the SDK
(no .git), collapsing to the script's mtime and breaking .apk
reproducibility across SDK rebuilds and between buildbot and SDK.
With this comment, PKG_UNPACK generates a version.date file, later used to
determine a reproducible SOURCE_DATE_EPOCH. Since unpack happens after
download, the evaluation of SOURCE_DATE_EPOCH is now lazy, invoking the
`get_source_date_epoch.sh` script on every use.
While at it, drop export of PKG_SOURCE_DATE_EPOCH and clean it from the
ipkg-build script.
Link: https://github.com/openwrt/openwrt/issues/21579
Link: https://github.com/openwrt/openwrt/pull/21587
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Link: https://github.com/openwrt/openwrt/pull/23576
Signed-off-by: Paul Spooren <mail@aparcar.org>
qemustart is a handy script to quickly test OpenWrt firmware using
qemu. Bringing up networking currently requires a bridge-helper
setup with privileged IP and bridge assignment. To simplify
testing scenarios like the package manager, which need both shell
access and outbound internet, add a user-mode networking option
backed by SLIRP that requires no privileges.
To stay backward compatible, the defaults don't change. The new
flag --user-network attaches two NICs (LAN + WAN) and forwards
three host ports to the guest LAN interface (192.168.1.1):
2222 -> 22 (ssh), 8080 -> 80 (http) and 8443 -> 443 (https). The
host-side ports can be overridden with --ssh-port, --http-port
and --https-port.
Link: https://github.com/openwrt/openwrt/pull/23424
Signed-off-by: Paul Spooren <mail@aparcar.org>
OpenWrt's 99-default_network assigns eth0 to lan and eth1 to wan
when no target-specific 02_network is present, which is the case
for malta. The qemustart bridge block however placed the wan
-device before the lan -device, so the guest's eth0 (lan) ended
up attached to $BR_WAN and eth1 (wan) to $BR_LAN.
Swap the order to match the guest's actual role assignment.
Link: https://github.com/openwrt/openwrt/pull/23424
Signed-off-by: Paul Spooren <mail@aparcar.org>
The ImageBuilder creates by default all filesystems enabled during it's own
build, which are typically squashfs and sometimes ext4.
This commit allows to set ROOTFS_FILESYSTEM to specify which specific
filesystem should be build (instead of all).
Motivation is to reduce the load on sysupgrade servers but also fix corner
cases where a squashfs filesystem results in a working image while the ext4
image fails, resulting in a ImageBuilder failure.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Right now we only have the special getver.sh output (i.e. r32802-f505120278)
instead of the actual, full git hash. Offer the full hash for downstream
tooling, specifically the KernelCI.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Switch http:// (and redundant ftp://) PKG_SOURCE_URL entries to https://
across tools/ and package/. PKG_HASH alone does not protect against an
attacker tampering with insecure downloads when a maintainer regenerates
the hash via `make ... FIXUP=1`: HTTPS authenticates the upstream so the
captured hash reflects real upstream content.
In-place http -> https (HTTPS reachability verified per host):
- tools/elftosb, tools/lzop, tools/liblzo, tools/mpfr, tools/dosfstools,
tools/libressl, tools/xz
- package/libs/mpfr, package/libs/libmnl, package/libs/libnfnetlink
Replaced with @OPENWRT (HTTPS-only mirror) where the upstream HTTPS host
is dead or has a broken certificate:
- package/libs/popt (ftp.rpm.org cert mismatch)
- package/firmware/ixp4xx-microcode (was http://downloads.openwrt.org)
- package/boot/imx-bootlets (trabant.uid0.hu cert mismatch)
- package/boot/kobs-ng (freescale.com URL is dead, redirects to nxp.com root)
Dropped redundant ftp://ftp.denx.de fallback (https://ftp.denx.de is
already listed):
- package/boot/uboot-tools, tools/mkimage
Signed-off-by: Paul Spooren <mail@aparcar.org>
* e3f6a41 main: exit 1 when showing the usage
* b17c31f main: exit 1 on getopt() errors
* e086664 lexer: fix a minor memleak in jp_get_token()/match_token()
* e5a07f4 main: defer processing until options are processed
* afe72ad main: usage spell fixes
Signed-off-by: Paul Spooren <mail@aparcar.org>
Update the default kernel path in start_qemu_malta() to match the new
image naming scheme after the malta target was converted to the Device
macro system with device name 'generic'.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Convert the malta target from the legacy Image/BuildKernel and
Image/Build pattern to the modern Device macro system. This is the
last target still using the legacy pattern.
The Device macro system automatically generates per-image JSON
metadata files which get aggregated into profiles.json, enabling
firmware selector and other tooling support for all malta subtargets
(be, le, be64, le64).
The kernel ELF is produced via KERNEL_NAME := vmlinux.elf (matching
octeon), uImage artifacts are built using the standard Build/lzma,
Build/gzip and Build/uImage commands with the existing load address
0x80100000, and rootfs images use append-rootfs with optional gzip
compression.
The device is named 'generic' following the convention used by other
virtual/emulated targets (x86, armsr, octeon).
Signed-off-by: Paul Spooren <mail@aparcar.org>
Run this only when a PR is created, so either author or anyone who can
edit the PR message (all maintainers) can trigger such build.
Instead of maintaining it in the main repository, move it to the
external repository to bloat this repository less with further CI
commits.
Signed-off-by: Paul Spooren <mail@aparcar.org>
APK, unlike OPKG, can sign individual packages and not just indexes.
Since OpenWrt uses a distributed build infrastructure and only the build
master owns the private keys, signing of individual buildworkers doesn't
work. Right now, each buildworker creates a temporary build key to sign
packages, then transmits the package index to the buildmaster for a
signature.
As a result, all individual packages contain a nonsensical signature,
making them harder to reproduce. This commit removes the individual
package signing.
Since APK requires signatures per default, explicitly allow installation
of unsigned packages during the build process.
The config option here is for historical reasons misleading,
SIGNED_PACKAGES refers to the package index, not the individual
packages.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The package bump to version rc6 incorporates two recent changes, first, the
stored `mtime` honors SOURCE_DATE_EPOCH, making a prior "touch" obsolete.
Secondly the order of files added to created packages is now sorted, improving
reproducibility, too.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Right now the sha256sums are only created for the targets/ folder (i.e.
firmware images) and only the buildbot generates those sha256sums. Instead, let
the build system create the sha256sums directly.
Signed-off-by: Paul Spooren <mail@aparcar.org>
APK kindly stores the mtime of each containing file in created packages,
breaking reproducibility. As a fix, touch all files of the package with the
timestamp of PKGSOURCE_DATE_EPOCH, which contains the timestamp based on the
last package modification.
Over at OPKG, something similar is done by setting mtime in the tar command,
see the `ipkg-build` script.
To tackle this in APK directly, some changes are suggested. However until this
is merged, we should fix it downstream.
https://gitlab.alpinelinux.org/alpine/apk-tools/-/merge_requests/348
Signed-off-by: Paul Spooren <mail@aparcar.org>
This commit fixes "aff2f096235 include: make APK .list files reproducible"
since it would create the .list file while `find` still runs. This causes the
.list file to be part of itself. As an alternative, write the file to a
temporary folder first and then move it.
Fix: aff2f09623 include: make APK .list files reproducible
Signed-off-by: Paul Spooren <mail@aparcar.org>
While this worked during testing, the repository of a organization requires
extra permissions. Add write access to pull requests, like we do over at
packages.git.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This has been requested many times, so let's add this to speed up reviews. When
a member of the "reviewers" group comments the magic word written below, that
specific firmware is created and attached by a bot.
build <target>/<subtarget>/<profile>
Members of the "reviewers" group have no extra privileges, they can not commit
to the repository nor perform any action outside the `build-on-comment` action.
Motivation is to speedup reviews and have a better source for sharing compiled
firmware.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The three packages base-files, libc and kernel are special, the former
can't be upgraded in place since it's content are modified on startup,
the latter two are virtual packages only used as constraints for the
package manager.
Historically base-files was "locked" via a special OPKG function, the
latter two were hidden from the package index and thereby never picked
as possible upgrade.
Time moved forward and we now have APK and tools like OWUT. The latter
compares available packages with installed packages and generates user
readable output, requiring versions for libc and kernel, too. At the
same time, APK uses a different looking mechanism, which is set during
installation instead of part of the package metadata.
In short, this patch adds version constraints to the three packages,
allowing them to be part of the package index.
Fixes: #17774Fixes: #17775Fixes: efahl/owut#31
Signed-off-by: Paul Spooren <mail@aparcar.org>
This new value points to where firmware can be downloaded. It's not
about a single release but all available firmware releases.
In the next step, this URL should be exposed via `ubus call system
board` as an entry of the `distribution` field. With that value, the
running firmware can check for newer releases.
We already have VERSION_REPO however that's different and only meant for
package managers to download their fitting package indexes/packages.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Link: https://github.com/openwrt/openwrt/pull/17780
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This commit comes after a long period of hard work, starting back in
early 2021 as a proof of concept.
Thanks to the Alpine Linux project for creating such a nice package
manager. Thanks to everyone involved; this is going to be great!
Signed-off-by: Paul Spooren <mail@aparcar.org>
In 451e2ce006 the code would only move packages separated by dashes,
however OPKG uses underscores to split between package name and version.
Result was the lack of such packages in the packed ImageBuilder.
Use dashes or underscores depending on the select package manger.
Fixes: 451e2ce006 "imagebuilder: fix package inclusion and APK index"
Signed-off-by: Paul Spooren <mail@aparcar.org>
The `--initdb` command creates basic folders required by APK,
previoiusly it would only run a single time when package_index is
actually called. Since the function isn't called if nothing changes,
`--initdb` doesn't initialize the rootfs again.
This commit moves it to package_reload, which runs every time.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Previously three different ways to include packages in an ImageBuilder
existed:
* buildbot: include libc, kernel (and base-files) in $(IB_LDIR)
* not buildbot, standalone: include all packages in ./packages/
* not buildbot, not standalone: include libc, kernel (and base-files) in
./packages/
First of, the separation between *buildbot* and *not buildbot, not
standalone* is not required, we can just always copy packages to
./packages instead of ever using the special place $(IB_LDIR).
Doing so drops the need to handle the extra case and also allows to
clean up the OPKG package installation, which no longer requries the
`firstword` logic, things are now always at ./packages.
Signed-off-by: Paul Spooren <mail@aparcar.org>
We need to exclude packages from the APK index which must not be
upgraded. To do so, the packages `libc`, `kernel` and `base-files` are
excluded to APK never suggestes them as upgradable.
The previous logic would however match packages like `libcomerr0`,
causing build failures. Make the copying and excluding logic more
precise by adding a single dash.
Signed-off-by: Paul Spooren <mail@aparcar.org>
This commit solves multiple issues. First of just install the three
special packages base-files, libc and kernel directly from the index. In
upstream indexes, those will never appear to prevent accidental upgrades
may breaking the system.
Next, enable caching for the ImageBuilder, which speeds up consecutive
builds from ~33 seconds to ~5 seconds. Using cache however makes APK
create the folder `/var/cache/apk/` which conflicts with the base-files
installation, which ships a symlink from `/var` to `/tmp`, so specify
`--no-cache` for the rootfs initialization.
Lastly, drop the use of `apk update` since APK automatically does that.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Both packages `ombnia-mcu-firmware` and `omnia-mcutool` would depend on
a specific device. The buildbots however build all devices and therefore
the package isn't build at all, due to unmet dependencies.
While this didn't cause issues with OPKG, APK fails actively due to the
missing packages. Drop the specific dependency, however wants to install
unrelated firmware on any device can do that anyway.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Those packages were not copied due to OPKG using an underscore while APK
uses dashes. Remove that char to copy kernel/libc for either APK/OPKG.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Previously APK would complain as it wasn't sure which package to
install by default when multiple packages would provide the same name.
Now, give the package a higher provider priority to make APK
automatically select the "default" package.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The same that is done in `ipkg-make-index.sh` should happen with APK.
If the pseudo packages, only added to add dependency constraints, are
added to the index, APK happily "upgrades" them and installs updated
kmods, too. However, the Kernel itself is never installed via a regular
package.
Fixes: https://github.com/openwrt/openwrt/issues/16808
Signed-off-by: Paul Spooren <mail@aparcar.org>
Prior to this commit keys would only be generated if `make` is called
alone, but not for something like `make package/busybox/compile`.
The exact reasons are in the depth of make magic, so this is sheer luck!
Signed-off-by: Paul Spooren <mail@aparcar.org>
Adding a version to the provides causes it to conflict with other
packages that provides the same package, further details are available
here: https://gitlab.alpinelinux.org/alpine/apk-tools/-/blob/master/doc/apk-package.5.scd#L199
This was intitally done, if I remember correctly, to support depending
on the specific kernel modules. Due to patches to APK, versions
containing hashes work now, too, so this is no longer required.
Only add the version to packages that define an ABI version since other
packages depend against the package name plus ABI version.
While at it, format the now rather complex call.
Fixes: https://github.com/openwrt/openwrt/issues/16795
Signed-off-by: Paul Spooren <mail@aparcar.org>
The version was a mix of strings, hex numbers and semantic numbers.
Switch the PKG_VERSION to something digestible by APK and introduce
PKG_SOURCE_VERSION to handle the actual filename.
While at it, drop the redundant PKG_B_NAME which was the same as
PKG_NAME anyway.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The keys are created differently compared to the old OPKG keys. Instead
of being part of base-files/configure, they are created as a Makefile
requirement of `package/compile`, which is a cleaner solution.
This requirement would only be added to non SDK environments, however
APK always requires keys to be available. Add an `else` case for the SDK
and create keys.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Initially APK would sign packages and package index and verify
signatures individually. With the latest change, all packages inside a
trusted index are automatically trusted.
This is important within the OpenWrt eco-system since signing the index
happens on another machine than the package creation.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Testing OpenWrt is important, and there is a test suite in the making.
For maximum convenience and minimal CI over-usage, make it simple to run
tests locally. The main Makefile now attempts to include
`tests/Makefile` and silently fails if it doesn't.
While the test suite[1] is still young, it provides good examples of how
to test things around OpenWrt: starting with shell scripts using
`bats`[2], followed by QEMU tests, and finally real device tests using
LabGrid[3]. This could lead to the creation of the best OpenWrt version
yet.
Please consult the `openwrt-tests.git` README.md for details on the
setup. Once installed you may run commands like the following:
* make tests/shell # run shell tests
* make tests/x86-64 # run and test x86/64 in QEMU
[1]: http://github.com/aparcar/openwrt-tests/
[2]: https://bats-core.readthedocs.io
[3]: https://labgrid.readthedocs.io
Signed-off-by: Paul Spooren <mail@aparcar.org>
Link: https://github.com/openwrt/openwrt/pull/15647
Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
This reverts commit 80d1c353b7 with the
fix which won't break running systems. A logic error on how shell
handles && and || more the init process.
Signed-off-by: Paul Spooren <mail@aparcar.org>
When running unit tests this causes trouble since `/lib/config/uci.sh`
isn't available in those cases. Instead exit with a clean status fo the
unit test framework don't wrongly interpret things as an error.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Since we set the root for APK it tries to use those during the build,
which shouldn't happen since local package are used instead.
Disable the repositories by manually setting an empty repository.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Due to missing quotes the script would wrongly assume APK to be enabled
and don't run post install scripts, breaking pretty much everything.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The host build would need Lua to compile which currently adds a race
condition. Instead of tracking that down just disable helptext for the
host build.
Signed-off-by: Paul Spooren <mail@aparcar.org>
Both package managers work slightly different, i.e. stores files at
different places. Modify the `functions.sh` file to cover those.
Signed-off-by: Paul Spooren <mail@aparcar.org>
The APK package manager does not support handling of package
alternatives itself, so implement it via a simple shell script.
Signed-off-by: Paul Spooren <mail@aparcar.org>
A new option called `USE_APK` is added which generated APK packages
(.apk) instead of OPKG packages (.ipk).
Some features like fstools `snapshot` command are not yet ported
Signed-off-by: Paul Spooren <mail@aparcar.org>
Paul Spooren
