Includes fixes for:
* CVE-2021-3177 - ctypes: Buffer overflow in PyCArg_repr
* CVE-2021-23336 - urllib parse_qsl(): Web cache poisoning - semicolon
as a query args separator
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This backports fixes for security issues, including:
* CVE-2020-14422: Hash collisions in IPv4Interface and IPv6Interface
* CVE-2019-20907: Infinite loop in the tarfile module
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This lets the Python build process set _PYTHON_HOST_PLATFORM instead of
forcing an explicit value.
Also:
* Save the target _PYTHON_HOST_PLATFORM value during Build/InstallDev
for use when building target Python packages (in python3-package.mk).
* Use the (mostly) default PYTHON_FOR_BUILD value, instead patch
configure to remove the platform triplet from the sysconfigdata file
name.
* Remove the "CROSS_COMPILE=yes" make variable (there is no indication
that this variable is necessary).
* Force host pip to build packages from source instead of downloading
binary wheels.
Previously, host pip can download universal (platform-independent)
wheels but not platform-specific wheels, because of the custom
_PYTHON_HOST_PLATFORM value. (Packages that do not have universal
wheels would be compiled from source.)
With a correct _PYTHON_HOST_PLATFORM, host pip can install
platform-specific wheels as well. However, the pre-built shared object
(.so) files in these wheels will have the host's platform triplet in
their file names. When target Python packages are built (using the
target's _PYTHON_HOST_PLATFORM), Python will not use these shared
object files.
By forcing host pip to build packages from source, the built shared
object files will not have the platform triplet in their file names.
(Host Python has been patched to remove the platform triplet from file
names.) This allows these packages to be used when building target
Python packages.
(The net effect of this complete change is that platform-dependent
packages will continue to be compiled from source, while
platform-independent packages will now also be compiled from source.)
Fixes https://github.com/openwrt/packages/issues/12680.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
When a Python package is installed from source (i.e. using setup.py)
into a custom location (with --home), setuptools may want to create a
site.py file in the custom location. This file is created based on the
source code of site-patch.py, a file bundled with setuptools.
Because the normal OpenWrt setuptools package does not contain Python
source code, this file is missing and the installation will end with an
error.
This copies site-patch.py to site-patch.py.txt so that it will be
included in python3-setuptools, and patches setuptools to look for this
file.
See https://github.com/openwrt/packages/issues/12223
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 70a7f736c5)
Python will record the values of CC, CXX, AR, and READELF (and other
configure options) used during compilation. pip will use these programs
when asked to compile extension modules on the target device.
* If ccache is used during build, CC and CXX will be ccache_cc and
ccache_cxx, respectively, which are not available on-device (#11912).
* If an external toolchain is used during build, the values of these
variables will contain the external toolchain prefix, which may not be
available on target.
* If the normal toolchain is used during build, AR and READELF will
contain the toolchain prefix, but the names of ar and readelf
on-device do not contain the prefix; they are named "ar" and
"readelf".
This changes the values of these variables in Python's files to match
the names available on-device, and without any toolchain prefix.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
(cherry picked from commit 9f81ab895e)
This package is required by other packages to run some binaries via
`load_entry_point`.
So, this splits this package away from setuptools.
setuptools is pretty big, akd pkg-resources is also big, but not as big.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
(cherry picked from commit ed0e77f3c3)
Reference to discussion at
c61579b564 (commitcomment-36665837)
Adjusted python PKG_RELEASE items to current situation
Signed-off-by: Hannu Nyman <hannu.nyman@iki.fi>
setuptools: Updated to version 41.2.0
pip: Updated to version 19.2.3
Removed patches:
- 017: code was refactored
- 02{5,6,7,8}: those patches were backported from upstream and they are
included in 3.7.5
Refreshed patches
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
* Remove $$$$(pkg-config --static --libs libcrypto libssl) from
HOST_LDFLAGS
Having this leads to an "unknown type name 'u_int'" error on Mac.
Removing it doesn't appear to affect Python's ability to find
buildroot LibreSSL.
* Change -Wl,-rpath=... to -Wl,-rpath,... in HOST_LDFLAGS
The equals sign version is not supported by the Mac linker (appears to
be an GNU extension). The comma version is supported; -rpath and its
argument will be separated by a space when passed to the linker.
* Add ac_cv_header_libintl_h=no to HOST_CONFIGURE_VARS for Mac
Python on Mac doesn't expect to use libintl, but if gettext-full is
compiled for host, it will try, leading to undefined symbol errors
during compilation. This prevents configure from finding libintl.h.
Fixes#7171.
Fixes#9621.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
- Update setuptools to 40.8.0
- Update pip to 19.0.3
- Refreshed patches
- Removed 4 patches (2 of them was included in 3.7.3 and other two are
included in this release)
Makefile python3:
- Move PKG_MAINTAINER above PKG_LICENSE
Signed-off-by: Josef Schlehofer <pepe.schlehofer@gmail.com>
(cherry-picked from ce769db007)
This patch, taken from buildroot, avoids the use of host paths when
compiling third-party extensions.
Signed-off-by: Eneas U de Queiroz <cotequeiroz@gmail.com>
(cherry-picked from 523c52f6f2)
Build/InstallDev is passed a second argument, a path where host binaries
should be placed (ultimately $(STAGING_DIR)/host).
This change moves python[3]-config to that directory.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
These patches address issue:
CVE-2019-9948: Unnecessary URL scheme exists to allow local_file://
reading file in urllib
Link to Python issue:
https://bugs.python.org/issue35907
Issue 35907 is still currently open, waiting for a decision for
Python 3.5; these patches for Python 2.7 and 3.7 have been merged.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
These patches address issues:
CVE-2019-9740: Python urllib CRLF injection vulnerability
CVE-2019-9947: Header Injection in urllib
Links to Python issues:
https://bugs.python.org/issue36276 (resolved duplicated of 30458)
https://bugs.python.org/issue35906 (resolved duplicated of 30458)
https://bugs.python.org/issue30458
Issue 30458 is still currently open, waiting for a decision for
Python 3.5; these patches for Python 2.7 and 3.7 have been merged.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes the "patched" indicator files for host setuptools and pip
to include their PKG_RELEASE values. This also removes host setuptools
and/or pip before host install, if the installed copy does not match the
version (and PKG_RELEASE) of the copy to be installed.
This will allow added or removed patches to affect host setuptools /
pip, since these changes will cause PKG_RELEASE to be incremented.
This also fixes the host install error, when the install tries to patch
an already patched copy of setuptools. (This error occurs because the
existing indicator files do not have version numbers in their file
names, whereas host install expected version numbers to be present.)
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds the current setuptools/pip version numbers to the indicator
files' names, which should allow upgraded versions to be patched.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This uses two find commands to delete __pycache__ contents then the
__pycache__ directories, rather than a for loop.
The second command omits a -empty test, so that if the first command
doesn't remove all directory contents for some reason, the second
command will return an error (find will not delete a non-empty
directory).
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes the --prefix option, passed to host pip when "installing"
target setuptools and pip, to /usr, in case the prefix is recorded in
the packages.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds --cache-dir and --disable-pip-version-check options for host
pip, when "installing" target setuptools and pip.
This also changes the pip command to use $(HOST_PYTHON[3]_PIP) from
python[3]-host.mk.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Normally, Python will include the user's site-packages directory
(~/.local/lib/python$(PYTHON_VERSION)/site-packages) in it's internal
search path for modules.
This disables this default inclusion for host Python.
This change is applied during Host/Configure instead of as a patch to
keep this setting unchanged for target Python.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes --with-ensurepip=install to upgrade, to upgrade host
versions of setuptools and pip to the Python-bundled versions.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This adds the ability to patch setuptools (and pip), and adds 3
reproducibility patches from Debian[1].
(003-PKG-INFO-output-reproducible.patch addresses the issue identified
in #9039.)
The patching is not perfect, in that the patches are applied to
setuptools and pip after they have been installed, since they are
installed from wheels which are already "precompiled".
Also, patching for the host install cannot be updated in place, for
example if a patch is added or removed.
[1]: https://sources.debian.org/patches/python-setuptools/40.8.0-1/
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Changed PKG_LICENSE to reflect spdx license tag, and PKG_LICENSE_FILES
to include all lincense-related files applicable to the parts of the
code we are actually using to build and/or distributing. The
Windows-only files, and the python-bundled Tools we're not using have
been left out.
Signed-off-by: Eneas U de Queiroz <cote2004-github@yahoo.com>
This installs python{2.7,3.7}-config in $(STAGING_DIR)/usr/bin as part
of Build/InstallDev, to be used by other packages to get build
configuration for target Python.
The treatment for Python 2 and 3 are a bit different:
* For Python 2, python-config is a Python script that is expected to be
run with, and return data for, the installed Python interpreter. This
installs a modified version of this script, to be run using host
Python, and read/return data for target Python.
* Python 3 includes a shell script version of python-config (expected to
be used in cross-compilation scenarios). This simply installs the
script into the right place.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
This changes Build/InstallDev for both Python 2 and 3 to only copy files
from target Python, not from host Python, since InstallDev files are
used for target packages to link to other target packages.
In particular, usr/lib/python{2.7,3.7}/_sysconfigdata.py holds system
configuration data generated at build time, and is different for target
Python and host Python.
Signed-off-by: Jeffery To <jeffery.to@gmail.com>
Fixes: https://github.com/openwrt/packages/issues/8399
These 2 patches cause some breakage for other packages.
For now, we drop them and wait for upstream to finalize a fix.
We can live with deprecated SSL APIs for a while. No need to hurry, since
this doesn't seem to help.
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
For a while now, Jeffery has helped quite a lot with Python, and is now
unofficial go-to guy [for problems] with Python packages.
This change adds him as co-maintainer [if he also agrees].
I'm not going away; I'll be still doing the same work for Python.
This change serves to recognize Jeffery in an official way, since he's
already taking on these things. And 2 co-maintainers is better in case one
kicks the bucket [by accident].
Signed-off-by: Alexandru Ardelean <ardeleanalex@gmail.com>
