get_bootdev() identifies the boot disk for a root=PARTUUID=xxxxxxxx-NN
command line by reading the four-byte signature at offset 440 of each
block device, but renders it with %x per byte, so a byte below 0x10 comes
out as a single digit and the string is too short to ever match. A disk
whose signature is cf0a3161 yields "cfa3161", get_bootdev() returns null,
and every backend's init fails: uvol then reports no backend at all and
no volume can be created on a perfectly good volume group.
Pad each byte to two digits, as the GPT branch of the same function
already does.
Fixes: 6350c7bc63 ("uvol: replace with re-write in ucode")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Move the storage backend probe, the ctx construction, the locking
helpers and the volume-name check out of the CLI script into
/usr/lib/uvol/common.uc, so a second entry point can reuse them
without duplicating the logic.
Publish ubus object 'uvol' with the volume operations and a readiness
query. Consumers such as uxc are pure ubus frontends and must not exec
the command line tool. Ship an rpcd exec plugin, installed by the uvol
package itself, taking the same locks as the CLI so both entry points
stay serialised. As stdout is the plugin's reply channel, point file
descriptor 1 at stderr for the duration of a call.
Send a 'uvol.ready' ubus event at the end of 'uvol boot', carrying the
active backend name and whether the .meta volume is ready. Consumers
such as uxc can wait for this event instead of polling volume state.
Sending is best-effort: boot keeps its exit code even when ubusd is
not reachable.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Volume names were interpolated into shell command lines at every
backend call site, so a name containing shell metacharacters ran
arbitrary commands as root. ucode's system() takes an array and execs
it directly, so every call site that interpolates a name, a device or
a path now passes an argument vector instead. Three sites genuinely
want a shell and keep it, with their arguments quoted: the lvm()
helper reads its JSON report via popen(), the content-addressed digest
pipeline quotes the volume path, and taking a volume down attempts a
umount whether or not anything mounted it, so that one keeps its
redirection and is now shared by both backends instead of spelt out
per call site. Suppressed errors that were only hiding a useless
diagnostic are dropped, and the filesystem grow tool is now located by
searching PATH instead of asking a shell.
lvm matches lvname with its own regular expression, and a volume name
was placed in that expression raw, so a dot in a name matched any
character instead of itself: a volume called a.b resolved to an
unrelated axb. Dotted names are the norm, since uxc composes a data
volume as <container>.<volname>; the name is now escaped where the
selection expression is built.
Every lvs, vgs and pvs caller dereferenced straight into the JSON
report, so an lvm invocation that failed to produce one, or failed to
run at all, crashed with a reference error instead of an error code.
Both now report through the .retval every caller already checks.
Volume names reaching the command line are now restricted to
^[A-Za-z0-9_][A-Za-z0-9._-]*$: a leading dot is reserved for uvol's
own volumes such as .meta, and a leading dash would be read as an
option by the backend tools. Only the read-only verbs may name an
internal volume, closing remove, down and resize being able to target
the metadata volume, which the backends never rejected the way mount
registration does; uvol's own boot path calls the backend directly
and is unaffected. The check runs before the per-volume lock, which is
keyed on the volume name too, which is why the -j flag is now consumed
before both. A rejected name exits 22 rather than falling off the end
of the script, which reports success.
Reachable only by root today, through the command line or a volume
name composed by uxc from an author-controlled registration; fixed as
defence in depth.
Fixes: 6350c7bc6 ("uvol: replace with re-write in ucode")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
manage volume state over ubus instead of the rootfs
Rework the volume lifecycle around blockd's ubus API, so volume state
lives entirely on the self-describing LVM/UBI backing store and never
leaks into the firmware rootfs.
- register active volumes with blockd over ubus (mount.uc) instead of
spooling mounts into /etc/config/fstab (uci.uc)
- bootstrap .meta from 'uvol boot' and drop the broken 90-uvol-init
uci-default
- run 'uvol boot' straight from the mount.ready trigger, and query
'ubus call block status' at service start to cover storage already
being ready
- defer removal of a volume whose backing device is still held, and
reap it from the mount.umount trigger once it is free
- enforce content-addressed "<algo>-<hexdigest>" volumes in the
dispatcher: reuse an existing volume of the same digest, verify
every write
- verify a content-addressed write in place while the volume is still
incomplete, reporting EBADMSG on mismatch instead of flipping
volume state
- purge incomplete wo/wp leftovers on boot and reclaim an exactly
matching one on create
- serialise mutating commands with a device-wide lock, plus a
per-volume lock ordered after it so a slow write cannot stall other
volumes
- add a grow-only 'resize' verb to both backends: lvextend plus the
matching fs-grow tool for lvm, ubirsvol for ubi
- create ext4 volumes with a journal (was ext2) and request check_fs
when registering a read-write volume
- declare read-only mounts to blockd rather than letting block guess
from the filesystem
- reject non-numeric and non-positive sizes in create and resize
- report its own version
- simplify command line parsing (drop compatibility with ancient ucode)
Requires the matching fstools changes from
openwrt/openwrt@9b11fa4088.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Instead of /sys/devices/virtual/ubi which will no longer be available
in future kernels, switch to /sys/class/ubi.
While at it fix unrelated arithmetic syntax error by guarding the
affected expression to not run on an empty string.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Consider only integer part of free space in megabytes when
deciding the boundaries of the to be created partition.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Bump version to reset PKG_RELEASE counter.
Add missing dependency on ucode-mod-math which is needed for the
mkdtemp() implementation used to format UBI volumes.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Fix and simplify console output, make sure uvol doesn't break in case
of LVM being installed but not used.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Shift ARGV until actual parameters start to compensate for older
versions of ucode passing the complete cmdline.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Don't use features which are not yet present in the version of ucode
currently shipped with OpenWrt (such as `fs.access()` or assuming raw
mode by default).
Fix syntax error in main executable.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* Fix build which was broken by a wrong path in the Makefile.
Adapt to ucode commit 4618807 ("main: rework CLI frontend"):
* ucode now no longer needs the {% %} around each code file, remove
that and safe one level of indentation.
* ARGV now no longer includes ucode executable and script itself
Fixes: 6350c7bc6 ("uvol: replace with re-write in ucode")
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Replace previous Shell draft-quality implementation of uvol with a
rewrite in ucode[1].
While the new code is slightly larger, it performs much better (as
we no longer fork() for parsing strings like in Shell with grep, sed
and friends).
Before:
time uvol list -j
[ ... ]
real 0m 0.82s
user 0m 0.13s
sys 0m 0.10s
After:
time uvol list -j
[ ... ]
real 0m 0.47s
user 0m 0.05s
sys 0m 0.05s
[1]: https://github.com/jow-/ucode
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
'uvol list' was broken when introducing support for the hidden volumes.
Fix that by not using 'continue' keyword to break the loop (as that
lead to skipping 'json_select ..') and using if-clause instead.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* Winchester discs can take a long time to spin up which results in
blockd also taking a while to complete to start.
* Make sure uvol only gets started once blockd is ready.
* Clean up paths (use '/tmp/run/uvol' everywhere instead of relying on
the '/var' -> '/tmp' symlink)
* Create 'apk' folder when initializing .meta volume.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Calling 'block umount' results in *all* devices listed in
/etc/config/fstab being unmounted.
Just call 'umount /dev/$devname' directly instead as that has the
desired effect.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* auto-created meta volume
Automatically create volume '.meta' sized 0.01% of the total space
managed by uvol, the minimum size being 4 MiB.
This volume should be used to keep metadata about stored volumes, such
as container runtime configuration (in /var/run/uvol/.meta/uxc) or
the database of installed container packages.
* configurable LVM volume group
Allow setting volume group to be used by uvol using UCI.
This is useful on devices which do not boot off the device which
should be used for uvol and hence cannot make use of autopart and
physical volume detection based on kernel cmdline.
Use option vg_name in uvol section of fstab UCI config package.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
As /var/state is used for UCI state, move persistent config for uxc
from /var/state/uxc to /var/run/uxc.
uxc itself has already been changed to look for it there.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Create UCI fstab mount sections for existing volumes on firstboot.
Introduce special handling for volumes with non-[:alnum:] prefix:
- do not create UCI sections for these 'hidden' volumes
- create UCI section for '.uxc' special volume with target
/var/state/uxc to be used for uxc config.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Changing a volume right after activation doesn't work well due to some
caching or race-condition issue.
Perform activation as last lvchange operation as a work-around.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* genrate UCI fstab configs for each volume before first 'up'
* remove UCI section on volume remove
* use autofs automounter for read-only volumes
* try umount on 'down'
* emulate hotplug events for UBI volume up/down
* more robust error paths
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* lvm: remove left-over ubus events
* lvm: resolve /dev/dm-* device name
As block-mount and blockd now use /dev/dm-* names, let uvol also return
those device names.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* make sure new volumes are in state 'down' until explicitely
requested to transition to 'up' state. Useful for additional
verification steps after writing a read-only volume.
* remove unused ubus events as blockd does that much better now
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Using GPT/UUID parition table is not always a possible choice.
Add support for MBR/DOS partitioned disks to make autopart work on
legacy targets like mt7623.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Free and total bytes are now properly returned as bytes by LVM2 as
requested. No longer multiply values.
Fix parameter order of 'create' command in usage output while at it.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
* quotes around all variables
* always use 'read -r' instead of 'read'
* some more minor shellcheck fixes
* reorder LVM ops for 'up' call to allow it to succeed
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Use sfdisk to get GPT partition by name as partition names are not
known by the kernel if added via partx.
Make sure physical volume names are unique, if possible correlate
with the disks serial number and/or card's cid.
mkf2fs apparently returns 134 even in case format succeeded, so don't
fail in that case (this fixes rw volumes large enough for F2FS to be
selected by the lvm scripts of uvol).
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Emmit ubus events when volumes come up/down.
Make sure volume state is always well defined by introducing
additional state 'write-prepare' (wp) during mkfs.
Add init scripts to bring up volumes at boot.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Make sure filesystem is ready when volume becomes available.
Use 'write-once' as initial state for read-only volumes, only allow
writing to volumes in that state and transision to 'read-only' once
write has completed.
Also fix a typo which prevented 'list' command from working with LVM.
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
uvol is a wrapper-script which allows automated handling of storage
volumes. uvol currently comes with backend support for LVM2 and UBI,
covering practically all options for storage large enough to be
managed (NAND, SPI-NAND, eMMC, SATA, NVME, virtio-blk, ...).
Signed-off-by: Daniel Golle <daniel@makrotopia.org>
Daniel Golle
Philip Prindeville