mirror of
https://github.com/openwrt/packages.git
synced 2025-12-22 01:44:32 +04:00
b98fb60635
We have received reports of builds of perl occasionally failing when
building with many parallel jobs, with a log like the following:
LD_LIBRARY_PATH=[...]/perl/perl-5.40.0 ./miniperl -Ilib make_ext.pl \
dist/constant/pm_to_blib MAKE="make" LIBPERL_A=libperl.so
File/Path.pm did not return a true value at [...]/hostpkg/usr/lib/perl5/5.40.0/ExtUtils/MakeMaker.pm line 13.
BEGIN failed--compilation aborted at [...]/hostpkg/usr/lib/perl5/5.40.0/ExtUtils/MakeMaker.pm line 13.
Compilation failed in require at Makefile.PL line 3.
BEGIN failed--compilation aborted at Makefile.PL line 3.
Unsuccessful Makefile.PL(dist/constant): code=65280 at make_ext.pl line 532.
The failing extension (dist/constant in the above log) would differ
between runs.
The cause of the issue is the `-Ilib` in the command line of miniperl.
In the host build, `./miniperl -I lib` will use the following include
path:
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/AutoLoader/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/dist/Carp/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/dist/PathTools
[..]/build_dir/hostpkg/perl/perl-5.40.0/dist/PathTools/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-Install/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-MakeMaker/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-Manifest/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/File-Path/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/ext/re
[..]/build_dir/hostpkg/perl/perl-5.40.0/dist/Term-ReadLine/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/dist/Exporter/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/ext/File-Find/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/Text-Tabs/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/dist/constant/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/version/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/Getopt-Long/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/Text-ParseWords/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-PL2Bat/lib
[..]/build_dir/hostpkg/perl/perl-5.40.0/lib
.
Various dependencies of the extension build scripts (Makefile.PL) -
including File-Path, which failed to be loaded in the error log - are
included in the path by buildcustomize.pl, as these extensions are only
installed to `lib` as the build proceeds.
However, in a target build, miniperl is just a symlink to the previously
built host perl. As the host perl does not implicitly load
`buildcustomize.pl`, we get the following include path for
`./miniperl -Ilib`:
lib
[..]/staging_dir/hostpkg/usr/lib/perl5/site_perl/5.40.0/x86_64-linux
[..]/staging_dir/hostpkg/usr/lib/perl5/site_perl/5.40.0
[..]/staging_dir/hostpkg/usr/lib/perl5/5.40.0/x86_64-linux
[..]/staging_dir/hostpkg/usr/lib/perl5/5.40.0
The host perl's install location is used as the default include path
which provides File-Path etc. for the target build; however, as more
and more libraries get installed into `lib` during the extension build,
they may get loaded from there instead, as `lib` is at the beginning of
the include path. When multiple extensions are built in parallel, a
Makefile.PL may attempt to load File/Path from `lib` after the file has
been created, but before its contents have been written fully, resulting
in the build to fail.
In fact, we should not load anything from `lib` during the target build,
as it is the staging directory for the target, including native
extensions built for the target architecture - with one exception: The
build scripts expect to find target information in the `Config` module,
so simply removing `lib` from the include path completely would break
the build.
Solve the issue by creating an alternative lib directory `lib_build`,
symlinking `Config.pm` and its dependencies in it, and replacing the
`-Ilib` argument with `-Ilib_build` using a wrapper script around the
host perl executable. This is similar to the approach seen in perl's own
obsolete/broken cross compile scripts (`Cross/Makefile`).
Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
-- Perl configuration --
Perl uses a huge configuration file, normally generated via the Configure script
at build-time. This fails when cross-compiling though, so we need to supply our
own.
We're using perlconfig.pl to piece together the final configuration from a bunch
of configuration files(all ending in .config). Please refer to perlconfig.pl's
POD for information on usage and syntax.
Throughout the files, you will see a bunch of references to private symbols with
the prefix "owrt". These are used to control output in an effort to both
simplify writing configuration files, as well as to provide switchable options
to select the feature set of the resulting perl installation.
The following will be a summary/quick reference of all private symbols we're
currently using:
Passed via architecture configuration file(mipsel.config, i486.config, ...)
---------------------------------------------------------------------------
Symbol Values Description
owrt:bits 32/64 Target's native word length.
owrt:endian little/big Target's endianness.
owrt:arch mipsel, i486, ... Target's architecture name.
owrt:sig_count 64/128 Number of signals the target
provides(NSIG - 1).
owrt:sigs * Symbolic names of the first 32 signals
this architecture provides, in numeric
order. Separated by whitespaces.
owrt:sig_name_extra * Symbolic names of any additional signals
this architecture provides after
owrt:sig_count. Separated by
whitespaces.
owrt:sig_num_extra * Numeric values associated with the
signal names provided in
owrt:sig_name_extra. Separated by
whitespaces.
Passed via command line
-----------------------
Symbol Values Description
owrt:libc glibc/uclibc/musl Which C library implementation is in
use.
owrt:threads yes/no Whether to enable threading support.
owrt:ipv6 define/undef Whether to enable IPv6 support.
owrt:target_cross * Target architecture's host triplet.
owrt:target_cc * C compiler to use.
owrt:gccversion * target_cc's version number.
owrt:cflags * Additional C compiler flags.
owrt:ldflags * Additional linker flags.
owrt:staging_dir * Same as OpenWRT buildroot's
$(STAGING_DIR).
owrt:host_perl_prefix * host-perl installation prefix.
Passed via version.config
-------------------------
Symbol Values Description
owrt:perllibpath * Path to perl library files, from the
target's point of view.