openwrt-mirror/packages
1
0
Fork 0
mirror of https://github.com/openwrt/packages.git synced 2026-06-17 17:00:28 +04:00
Code Issues Packages Projects Releases Wiki Activity
Files
1d33c0ed9dfdaa1ded1241b3ed707bf286374258
packages/lang/perl/Makefile
T
Matthias Schiffer b98fb60635 perl: fix parallel build race condition in target build 
We have received reports of builds of perl occasionally failing when
building with many parallel jobs, with a log like the following:

    LD_LIBRARY_PATH=[...]/perl/perl-5.40.0 ./miniperl -Ilib make_ext.pl \
        dist/constant/pm_to_blib  MAKE="make" LIBPERL_A=libperl.so
    File/Path.pm did not return a true value at [...]/hostpkg/usr/lib/perl5/5.40.0/ExtUtils/MakeMaker.pm line 13.
    BEGIN failed--compilation aborted at [...]/hostpkg/usr/lib/perl5/5.40.0/ExtUtils/MakeMaker.pm line 13.
    Compilation failed in require at Makefile.PL line 3.
    BEGIN failed--compilation aborted at Makefile.PL line 3.
    Unsuccessful Makefile.PL(dist/constant): code=65280 at make_ext.pl line 532.

The failing extension (dist/constant in the above log) would differ
between runs.

The cause of the issue is the `-Ilib` in the command line of miniperl.
In the host build, `./miniperl -I lib` will use the following include
path:

    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/AutoLoader/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/dist/Carp/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/dist/PathTools
    [..]/build_dir/hostpkg/perl/perl-5.40.0/dist/PathTools/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-Install/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-MakeMaker/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-Manifest/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/File-Path/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/ext/re
    [..]/build_dir/hostpkg/perl/perl-5.40.0/dist/Term-ReadLine/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/dist/Exporter/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/ext/File-Find/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/Text-Tabs/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/dist/constant/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/version/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/Getopt-Long/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/Text-ParseWords/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/cpan/ExtUtils-PL2Bat/lib
    [..]/build_dir/hostpkg/perl/perl-5.40.0/lib
    .

Various dependencies of the extension build scripts (Makefile.PL) -
including File-Path, which failed to be loaded in the error log - are
included in the path by buildcustomize.pl, as these extensions are only
installed to `lib` as the build proceeds.

However, in a target build, miniperl is just a symlink to the previously
built host perl. As the host perl does not implicitly load
`buildcustomize.pl`, we get the following include path for
`./miniperl -Ilib`:

    lib
    [..]/staging_dir/hostpkg/usr/lib/perl5/site_perl/5.40.0/x86_64-linux
    [..]/staging_dir/hostpkg/usr/lib/perl5/site_perl/5.40.0
    [..]/staging_dir/hostpkg/usr/lib/perl5/5.40.0/x86_64-linux
    [..]/staging_dir/hostpkg/usr/lib/perl5/5.40.0

The host perl's install location is used as the default include path
which provides File-Path etc. for the target build; however, as more
and more libraries get installed into `lib` during the extension build,
they may get loaded from there instead, as `lib` is at the beginning of
the include path. When multiple extensions are built in parallel, a
Makefile.PL may attempt to load File/Path from `lib` after the file has
been created, but before its contents have been written fully, resulting
in the build to fail.

In fact, we should not load anything from `lib` during the target build,
as it is the staging directory for the target, including native
extensions built for the target architecture - with one exception: The
build scripts expect to find target information in the `Config` module,
so simply removing `lib` from the include path completely would break
the build.

Solve the issue by creating an alternative lib directory `lib_build`,
symlinking `Config.pm` and its dependencies in it, and replacing the
`-Ilib` argument with `-Ilib_build` using a wrapper script around the
host perl executable. This is similar to the approach seen in perl's own
obsolete/broken cross compile scripts (`Cross/Makefile`).

Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
2025-05-05 11:20:38 +02:00

205 lines
7.7 KiB
Makefile
Raw Blame History

#
# Copyright (C) 2006-2018 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#
include $(TOPDIR)/rules.mk
include perlver.mk
PKG_NAME:=perl
PKG_VERSION:=$(PERL_VERSION)
PKG_RELEASE:=2
PKG_SOURCE_URL:=https://www.cpan.org/src/5.0
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION).tar.gz
PKG_HASH:=c740348f357396327a9795d3e8323bafd0fe8a5c7835fc1cbaba0cc8dfe7161f
PKG_LICENSE:=GPL-1.0-or-later Artistic-1.0-Perl
PKG_LICENSE_FILES:=Copying Artistic README
PKG_MAINTAINER:=Marcel Denia <naoir@gmx.net>, Philip Prindeville <philipp@redfish-solutions.com>
PKG_CPE_ID:=cpe:/a:perl:perl
# Build settings
PKG_BUILD_DIR:=$(BUILD_DIR)/perl/$(PKG_NAME)-$(PKG_VERSION)
HOST_BUILD_DIR:=$(BUILD_DIR_HOST)/perl/$(PKG_NAME)-$(PKG_VERSION)
PKG_INSTALL:=1
PKG_BUILD_DEPENDS:=perl/host
PKG_BUILD_PARALLEL:=1
HOST_BUILD_PARALLEL:=1
# Variables used during configuration/build
HOST_PERL_PREFIX:=$(STAGING_DIR_HOSTPKG)/usr
ifneq ($(CONFIG_USE_MUSL),)
TARGET_CFLAGS += -D_LARGEFILE64_SOURCE
endif
# Filter -g3, it will break Compress-Raw-Zlib
TARGET_CFLAGS_PERL:=$(patsubst -g3,-g,$(TARGET_CFLAGS))
TARGET_CPPFLAGS_PERL:=$(patsubst -g3,-g,$(TARGET_CPPFLAGS))
# A list of disabled tests
# ExtUtils tests are disabled for now as we don't support building
# native extensions on the target machine at the moment
PERL_DISABLEDTESTS:=cpan/ExtUtils-Constant cpan/ExtUtils-MakeMaker
# We're on Linux, so don't even package them
PERL_DISABLEDTESTS+=cpan/Win32API-File cpan/Win32 ext/VMS-DCLsym ext/VMS-Filespec ext/VMS-Stdio ext/Win32CORE os2/
# NDBM and ODBM not supported
PERL_DISABLEDTESTS+=ext/NDBM_File ext/ODBM_File
include $(INCLUDE_DIR)/package.mk
include $(INCLUDE_DIR)/host-build.mk
include perlmod.mk
define Package/perl
SUBMENU:=Perl
SECTION:=lang
CATEGORY:=Languages
TITLE:=The Perl intepreter
URL:=http://www.perl.com/
DEPENDS:=+USE_GLIBC:libbsd +PERL_THREADS:libpthread @!arc
endef
define Package/perl/description
Perl is a stable, cross platform programming language.
It is used for mission critical projects in the public and private sectors
and is widely used to program web applications of all needs.
endef
define Package/perl/config
source "$(SOURCE)/Config.in"
endef
# Static host perl
define Host/Configure
( cd $(HOST_BUILD_DIR); ./Configure -der -Uusedl -Duserelocatableinc -Dprefix=$(HOST_PERL_PREFIX) $(if $(CONFIG_PERL_THREADS),-Dusethreads,))
endef
define Host/Install
( cd $(HOST_BUILD_DIR); ./miniperl installperl )
$(INSTALL_DIR) $(HOST_PERL_PREFIX)/bin/
$(CP) $(HOST_BUILD_DIR)/generate_uudmap $(HOST_PERL_PREFIX)/bin/
# Link any possibly installed static extension in
$(MAKE) -C $(HOST_BUILD_DIR)/relink clean || true
( cd $(HOST_BUILD_DIR)/relink && $(HOST_PERL_PREFIX)/bin/perl Makefile.PL )
$(call perlmod/host/relink,$(HOST_BUILD_DIR)/relink)
endef
# Target perl
define Build/Configure
# We don't want to pass -Ilib to host perl in the target build (as lib
# contains the target libraries, and files may currently be written
# while being imported in parallel builds). We do however need the
# target versions of the Config modules at the beginning of the include
# path for the build scripts' use.
#
# Create an alternative lib_build directory that will be added to the
# include path instead of lib (using hostperl-wrapper), containing only
# the config modules.
$(INSTALL_DIR) $(PKG_BUILD_DIR)/lib_build
ln -sf ../lib/Config.pm ../lib/Config_heavy.pl ../lib/Config_git.pl $(PKG_BUILD_DIR)/lib_build/
install -m0755 files/hostperl-wrapper $(PKG_BUILD_DIR)/hostperl-wrapper
sed -i "s'@HOST_PERL@'$(HOST_PERL_PREFIX)/bin/perl'g" $(PKG_BUILD_DIR)/hostperl-wrapper
$(PERL_CMD) files/perlconfig.pl -Dowrt:target_cc='$(TARGET_CC)' \
-Dowrt:gccversion=$(CONFIG_GCC_VERSION) \
-Dowrt:target_cross='$(TARGET_CROSS)' \
-Dowrt:cflags='$(TARGET_CFLAGS_PERL) $(TARGET_CPPFLAGS_PERL)' \
-Dowrt:ldflags='$(TARGET_LDFLAGS)' \
-Dowrt:libc=$(subst uClibc,uclibc,$(CONFIG_LIBC)) \
-Dowrt:ipv6=$(if $($(CONFIG_IPV6)),define,undef) \
-Dowrt:threads=$(if $(CONFIG_PERL_THREADS),yes,no) \
-Dowrt:staging_dir='$(STAGING_DIR)' \
-Dowrt:host_perl_prefix='$(HOST_PERL_PREFIX)' \
-Dsysroot='$(TOOLCHAIN_ROOT_DIR)' \
files/version.config \
files/base.config \
files/$(patsubst i386,i486,$(ARCH)).config \
files/architecture.config \
files/signal.config \
files/threads.config \
files/libc.config \
files/misc.config \
> $(PKG_BUILD_DIR)/config.sh
(cd $(PKG_BUILD_DIR) && ./Configure -S)
install -m 0644 $(PKG_BUILD_DIR)/config.h $(PKG_BUILD_DIR)/xconfig.h
endef
define Build/Compile
# make depend is required to avoid race conditions: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=996953
+$(MAKE) -C $(PKG_BUILD_DIR) depend
+$(MAKE) $(PKG_JOBS) -C $(PKG_BUILD_DIR)
endef
ifeq ($(CONFIG_arc),)
define Build/InstallDev
$(INSTALL_DIR) $(1)/usr/lib/perl5/$(PERL_VERSION)
$(CP) $(PKG_INSTALL_DIR)/usr/lib/perl5/$(PERL_VERSION) $(1)/usr/lib/perl5/
endef
endif
define Package/perl/install
$(INSTALL_DIR) $(1)/usr/bin
$(INSTALL_BIN) $(PKG_INSTALL_DIR)/usr/bin/perl$(PKG_VERSION) $(1)/usr/bin
ln -nsf perl$(PKG_VERSION) $(1)/usr/bin/perl
$(INSTALL_DIR) $(1)/usr/lib/perl5/$(PERL_VERSION)/CORE
$(CP) $(PKG_INSTALL_DIR)/usr/lib/perl5/$(PERL_VERSION)/CORE/libperl.so $(1)/usr/lib/perl5/$(PERL_VERSION)/CORE/
endef
$(eval $(call RequireCommand,rsync, \
$(PKG_NAME) requires rsync installed on the host-system. \
))
$(eval $(call BuildPackage,perl))
$(eval $(call HostBuild))
-include perlbase.mk
# A helper package that includes all sort of supplementary files for tests
define Package/perl-tests-common
$(call Package/perlbase-template)
TITLE:=Common test support files
DEPENDS:=@PERL_TESTS
endef
define Package/perl-tests-common/install
$(INSTALL_DIR) $(1)/$(PERL_TESTSDIR)
$(INSTALL_DIR) $(1)/$(PERL_TESTSDIR)/Porting
$(INSTALL_DIR) $(1)/$(PERL_TESTSDIR)/regen
$(INSTALL_DIR) $(1)/$(PERL_TESTSDIR)/lib
$(INSTALL_DIR) $(1)/usr/lib/perl5/$(PERL_VERSION)/XS
$(INSTALL_DIR) $(1)/usr/lib/perl5/$(PERL_VERSION)/auto/XS
$(INSTALL_DIR) $(1)/usr/lib/perl5/$(PERL_VERSION)/unicore
$(CP) $(PKG_BUILD_DIR)/t $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/Porting $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/regen $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/MANIFEST $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/TestInit.pm $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/vutil.c $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/vxs.inc $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/lib/XS $(1)/usr/lib/perl5/$(PERL_VERSION)/
$(CP) $(PKG_BUILD_DIR)/lib/auto/XS $(1)/usr/lib/perl5/$(PERL_VERSION)/auto
$(CP) $(PKG_BUILD_DIR)/lib/vmsish.pm $(1)/usr/lib/perl5/$(PERL_VERSION)/
$(CP) $(PKG_BUILD_DIR)/lib/vmsish.t $(1)/$(PERL_TESTSDIR)/lib
$(CP) $(PKG_BUILD_DIR)/lib/Internals.t $(1)/$(PERL_TESTSDIR)/lib
$(CP) $(PKG_BUILD_DIR)/lib/unicore/TestProp.pl $(1)/usr/lib/perl5/$(PERL_VERSION)/unicore
$(CP) files/perl-run_tests.sh $(1)/$(PERL_TESTSDIR)/run_tests.sh
sed \
-e 's!%%PERL_DISABLEDTESTS%%!$(PERL_DISABLEDTESTS)!' \
-e 's!%%PERL_VERSION%%!$(PERL_VERSION)!' \
-i $(1)/$(PERL_TESTSDIR)/run_tests.sh
$(CP) $(PKG_BUILD_DIR)/config_h.SH $(1)/$(PERL_TESTSDIR)
$(CP) $(PKG_BUILD_DIR)/perl.h $(1)/$(PERL_TESTSDIR)
endef
$(eval $(call BuildPackage,perl-tests-common))
Reference in New Issue View Git Blame Copy Permalink