Repository navigation
BSD support #1
Description
Activity
Hello @flit :-)
Here you go :-) I am working on FreeBSD 13-STABLE with GENERIC kernel configuration (some customizations for my hardware added):
uname -a FreeBSD 0xCFMX4 13.0-STABLE FreeBSD 13.0-STABLE #0 stable/13-n247979-3e322ded35f: Thu Nov 4 12:28:58 CET 2021 root@0xCFMX4:/usr/obj/usr/src/amd64.amd64/sys/GENERIC amd64There are some dependencies that needs to be installed prior build / install (only
setuptoolsandsetuptools_scm?). I have provided initialrequirements.txtfile in #2. Is there any other / new way to detect and install python dependencies?Without dependencies I got:
(venv38zephyr) python setup.py develop Traceback (most recent call last): File "setup.py", line 18, in <module> import setuptools_scm # noqa: F401 ModuleNotFoundError: No module named 'setuptools_scm'With dependencies installed:
(venv38zephyr) python setup.py package usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: setup.py --help [cmd1 cmd2 ...] or: setup.py --help-commands or: setup.py cmd --help error: invalid command 'package' (venv38zephyr) python setup.py develop running develop running egg_info creating src/libusb_package.egg-info writing src/libusb_package.egg-info/PKG-INFO writing dependency_links to src/libusb_package.egg-info/dependency_links.txt writing requirements to src/libusb_package.egg-info/requires.txt writing top-level names to src/libusb_package.egg-info/top_level.txt writing manifest file 'src/libusb_package.egg-info/SOURCES.txt' warning: the 'license_file' option is deprecated, use 'license_files' instead adding license file 'LICENSE' (matched pattern 'LICENSE') reading manifest template 'MANIFEST.in' warning: no previously-included files matching '__pycache__' found anywhere in distribution warning: no previously-included files matching '*.py[cod]' found anywhere in distribution warning: no directories found matching 'src/libusb' writing manifest file 'src/libusb_package.egg-info/SOURCES.txt' running build_ext build_temp = /XXX/libusb-package.git/src/libusb build_lib = /XXX/libusb-package.git/src bash /XXX/libusb-package.git/src/libusb/bootstrap.sh bash: /XXX/libusb-package.git/src/libusb/bootstrap.sh: No such file or directory Error while building libusb: command 'bash' failed with exit status 127 Ignoring build failure and creating system-only libusb-package Creating /XXX/libusb-package.egg-link (link to src) Adding libusb-package 1.0.24.1 to easy-install.pth file Installed /XXX/libusb-package.git/src Processing dependencies for libusb-package==1.0.24.1 Finished processing dependencies for libusb-package==1.0.24.1 (venv38zephyr) python Python 3.8.12 (default, Oct 14 2021, 01:14:41) [Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c on freebsd13 Type "help", "copyright", "credits" or "license" for more information. >>> import libusb-package File "<stdin>", line 1 import libusb-package ^ SyntaxError: invalid syntax >>> import libusb_package >>> dir(libusb_package) ['Any', 'Optional', 'TYPE_CHECKING', '_LIBRARY_EXT', '_LIBRARY_MAP_EXT', '_LIBRARY_NAME', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__version__', '_version', 'atexit', 'ctypes', 'find', 'find_library', 'functools', 'get_library_path', 'get_libusb1_backend', 'platform', 'resources', 'sys']There seems no
python setup.py packagetarget.This error is a bit worrying during
python setup.py develop(development debug capable install):bash: /XXX/libusb-package.git/src/libusb/bootstrap.sh: No such file or directory Error while building libusb: command 'bash' failed with exit status 127 Ignoring build failure and creating system-only libusb-packageIs there any test I could run to verify created package operations?
Please note that FreeBSD that I work on provides its own LibUSB in the base OS install (0.x and 1.x compatible).
On NetBSD and OpenBSD there is a dedicated package for LibUSB.
Also please note that there is already
pyusbpackage that wraps LibUSB for Python. Here is BSD related thread: pyusb/pyusb#362On BSD systems static libraries use
*.aand dynamic libraries use*.sofile extensions, see built-in libusb as example:ls /usr/lib/libusb libusb_p.a libusb.a libusb.so@ libusb.so.3 libusbhid_p.a libusbhid.a libusbhid.so@ libusbhid.so.4Please let me know if you need any other information / feedback / testing :-)
I have read the project description:
This Python package functions as an installation vehicle for libusb shared libraries, to simplify installation of tools that require libusb. The main use case is so that users don't have to install libusb manually for projects that use pyusb.
On FreeBSD there is a BSD licensed LibUSB (created by @hselasky) already pre-installed with the base OS, so no need to install from package, its here already with the OS.
Maybe on OpenBSD and NetBSD this package needs to be added by hand. But these are system provided ports / packages.. and these must only be installed by
root.Anyways on BSD local user packages should not touch anything system wide, this is not allowed and unacceptable. Any local python package should be installed in local user created virtual environment. Any custom dynamic library should be placed in local application bundle. Local means nothing outside
/usr/home/user//home/userusually that is/home/user/.localor application directory.On BSD only
rootcan install system packages and/or modify system. Anything else is not allowed. I am not sure if this package is feasible here.Do I miss anything? :-)
(venv38zephyr) python Python 3.8.12 (default, Oct 14 2021, 01:14:41) [Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c on freebsd13 Type "help", "copyright", "credits" or "license" for more information. >>> import platform >>> platform.system() 'FreeBSD'Please note that there may be different flavors of BSD out there, mostly OpenBSD and NetBSD that are different OS, others are FreeBSD derivatives in most cases :-)
Even though there was some build error your package seems to work on FreeBSD:
(venv38zephyr) python Python 3.8.12 (default, Oct 14 2021, 01:14:41) [Clang 11.0.1 (git@github.com:llvm/llvm-project.git llvmorg-11.0.1-0-g43ff75f2c on freebsd13 Type "help", "copyright", "credits" or "license" for more information. >>> import libusb_package >>> >>> for dev in libusb_package.find(find_all=True): ... print(dev) ... DEVICE ID 1050:0111 on Bus 000 Address 015 ================= ... DEVICE ID 0bda:0411 on Bus 000 Address 014 ================= ... ... >>> import libusb_package >>> import usb.core >>> import usb.backend.libusb1 >>> >>> libusb1_backend = usb.backend.libusb1.get_backend(find_library=libusb_package.find_library) >>> # -> calls usb.libloader.load_locate_library( >>> # ('usb-1.0', 'libusb-1.0', 'usb'), >>> # 'cygusb-1.0.dll', 'Libusb 1', >>> # win_cls=win_cls, >>> # find_library=find_library, check_symbols=('libusb_init',)) >>> # >>> # -> calls find_library(candidate) with candidate in ('usb-1.0', 'libusb-1.0', 'usb') >>> # returns lib name or path (as appropriate for OS) if matching lib is found >>> >>> # It would also be possible to pass the output of libusb_package.get_libsusb1_backend() >>> # to the backend parameter here. In fact, that function is simply a shorthand for the line >>> # above. >>> print(list(usb.core.find(find_all=True, backend=libusb1_backend))) [<DEVICE ID 1050:0111 on Bus 000 Address 015>, <DEVICE ID 0bda:0411 on Bus 000 Address 014>, <DEVICE ID 0bda:0411 on Bus 000 Address 013>, <DEVICE ID 04dd:9761 on Bus 000 Address 012>, <DEVICE ID 1199:9041 on Bus 000 Address 011>, <DEVICE ID 5986:054c on Bus 000 Address 010>, <DEVICE ID 8087:0a2a on Bus 000 Address 009>, <DEVICE ID 04b5:0680 on Bus 000 Address 008>, <DEVICE ID 0bda:5411 on Bus 000 Address 007>, <DEVICE ID 046d:c52b on Bus 000 Address 006>, <DEVICE ID 041e:3040 on Bus 000 Address 005>, <DEVICE ID 8087:8001 on Bus 000 Address 004>, <DEVICE ID 0bda:5411 on Bus 000 Address 003>, <DEVICE ID 0000:0000 on Bus 000 Address 002>, <DEVICE ID 0000:0000 on Bus 000 Address 001>, <DEVICE ID 0000:0000 on Bus 000 Address 001>]Hi @cederom, apologies it took me so long to get back to this!
Thanks so much for testing! 🙏🏽
The build error (
src/libusb/bootstrap.shnot found) is almost certainly caused bysrc/libusbbeing a git submodule. If you check out submodules (git submodule update --init) and runpip install .it should get past this error. (I'll add a note to the readme.)The reason your test worked even though the build failed is that if the build if libusb fails when installing libusb-package, the install of libusb-package will still succeed. It just won't have a libusb shared library, and will always fall back to the system-provided libusb (if there is one).
System libusb and the point of libusb-package
The use case for libusb-package is that when a project like pyocd uses pyusb, there is normally no way to guarantee a known good version of libusb on all OSes. Users have to take an extra step of installing libusb before they can use pyocd.
Like the BSDs, Linux also normally comes with libusb installed. However, the system-provided version of libusb may be older and will likely change depending on the OS version. Using the libusb that comes with libusb-package, you know which version of libusb you'll be getting, and thus the limitations and bugs.
pyusb is the libusb wrapper that libusb-package is primarily designed to work with. libusb-package doesn't itself contain a Python wrapper around libusb—it's just the shared library and a few functions to conveniently get its path or create a pyusb
IBackendinstance.The libusb shared library is contained within the Python package, and is not installed anywhere system-wide. So it's only used by Python tools that explicitly want to use it.
Installing and building
Running
setup.pyisn't supported any more for Python packages that use the (relatively) new PEP 517 style packaging (pyproject.toml). The only supported method is using a standards-compliant tool like pip, e.g.pip install .orpip install -e .. Runningsetup.pydirectly isn't guaranteed to work any more. When you install with pip, the build dependencies such as setuptools_scm are installed automatically (they are listed inpyproject.toml).Similarly, the way to build either source distributions or wheels has changed now. You have to use the
buildpackage (install with pip). For examplepython -m buildwill build an sdist from the current directory, then build a wheel from the sdist, all done in an isolated virtual env.Testing
The
test.pyscript in the repo root can be run after libusb-package is installed in
Add BSD support to libusb-package. I'd like to make sure that libusb-package can install and work successfully from the source distribution on BSD systems, since prebuilt wheels for BSD are highly unlikely.
@cederom I don't have a BSD environment to test with. Could you help here?
One question is: what does
platform.system()return for BSD, and what is the file extension for shared libraries? This will allow an entry to be added to the library extension mapping dict:libusb-package/src/libusb_package/__init__.py
Lines 40 to 45 in 7b3a847