Develop Reference

ROS2 Yocto - no providers found in RDEPENDS_?

A Bit Backstory
I'm trying to install ROS2 package to a Yocto image using Devtools. I follow through this tutorial here with some modification according to my project.
The devtools built successfully, but when I run
ros2 pkg list
My ROS2 package wasn't there.
I've added my layers on the bblayers.conf etc.
So i thought that I need to bitbake the image again to install the ros2 package inside the Yocto image.
This is where the problem is
As I run
bitbake core-image-sato
I got an error
py-pubsub-0.0.0+git999-r0 do_package_qa: QA Issue: /usr/lib/py_pubsub/talker contained in
package py-pubsub requires /home/<PC_name>/yocto/build/tmp/work
/aarch64-poky-linux/py-pubsub/0.0.0+git999-r0/recipe-sysroot-native/usr/bin/python3-native/python3, but no providers found in RDEPENDS_py-
pubsub? [file-rdeps]
I'm guessing the problem is missing dependency added to RDEPENDS on the py-pubsub_git.bb file, so I added
RDEPENDS_${PN} += "/home/<PC_name>/yocto/build/tmp/work/aarch64-poky-
linux/py-pubsub/0.0.0+git999-r0/recipe-sysroot-native/usr/bin/python3-
native/python3"
but another error popped out. I also tried to add python3 to the RDEPENDS but I still got the same error.
Any help would be appreciated!
Thank you.
py-pubsub_git.bb
SUMMARY = "ROS2YOCTO: Examples of minimal publisher/subscriber using rclpy"
HOMEPAGE = "UNKNOWN"
LICENSE = "CLOSED"
LIC_FILES_CHKSUM = ""
SRC_URI = "git://github.com/<username>/py-pubsub.git;protocol=https;branch=master"
# Modify these as desired
PV = "0.0.0"
S = "${WORKDIR}/git"
inherit setuptools3
DEPENDS += "python3-setuptools-native ament-copyright ament-flake8 ament-pep257 ament-cmake-python python3-distlib python3 python3-native"
# WARNING: the following rdepends are from setuptools install_requires. These
# upstream names may not correspond exactly to bitbake package names.
RDEPENDS_${PN} += "python3-setuptools python3 ament-cmake-python"
# WARNING: the following rdepends are determined through basic analysis of the
# python sources, and might not be 100% accurate.
RDEPENDS_${PN} += "python3-core rclpy std-msgs"
RDEPENDS_${PN} += "python3-six python3-debugger python3-modules"
#RDEPENDS_${PN} += "bc python flex bison ${TCLIBC}-utils"
#RDEPENDS_${PN} += "openssl-dev util-linux"
#RDEPENDS_${PN} += "${#bb.utils.contains('ARCH', 'x86', 'elfutils', '', d)}"
FILES_${PN} += "/usr/share/ament_index/*"
#ALLOW_EMPTY_${PN} = "1"
#INSANE_SKIP_${PN} += "file-rdeps"
#INSANE_SKIP_${PN} += "build-deps"
Edit:
I found a package that also produces the exact same error as mine here, and I tried to do what they suggested
do_install () {
# Specify install commands here
rm -r ${D}/usr/lib/py-pubsub
}
but it produces this error
No such file or directory
which doesn't make sense to me since the directory is there, I even copy pasted the directory

Can you try adding the following rdepends to your recipe:
RDEPENDS_${PN} = "python3-core python3-native"
And see if that solves this issue?

ta-lib replit python install problem, ERROR: No matching distribution found for talib-binary

I use it on my windows machine by downloading its binary. I also use it in Heroku from its herokus build pack. I don't know what operating system replit use. But I try every possible commed like.
!pip install ta-lib
!pip install talib-binary
It's not working with replit. I thought it work like google co-lab but its not the same.
can anyone use TA-LIB with replit. if so. How you install it?

Getting TA-Lib work on Replit
(by installing it from sources)
Create a new replit with Nix toolset with a Python template.
In main.py write:
import talib
print (talib.__ta_version__)
This will be our test case. If ta-lib is installed the python main.py (executed in Shell) will return something like:
$ python main.py
b'0.6.0-dev (Jan 1 1980 00:00:00)'
We need to prepare a tools for building TA-Lib sources. There is a replit.nix file in your project's root folder (in my case it was ~/BrownDutifulLinux). Every time you execute a command like cmake the Nix reports that:
cmake: command not installed. Multiple versions of this command were found in Nix.
Select one to run (or press Ctrl-C to cancel):
cmake.out
cmakeCurses.out
cmakeWithGui.out
cmakeMinimal.out
cmake_2_8.out
If you select cmake.out it will add a record about it into the replit.nix file. And next time you call cmake, it will know which cmake version to launch. Perhaps you may manually edit replit.nix file... But if you're going to add such commands in a my way, note that you must execute them in Shell in your project root folder as replit.nix file is located in it. Otherwise Nix won't remember your choice.
After all my replit.nix file (you may see its content with cat replit.nix) content was:
{ pkgs }: {
deps = [
pkgs.libtool
pkgs.automake
pkgs.autoconf
pkgs.cmake
pkgs.python38Full
];
env = {
PYTHON_LD_LIBRARY_PATH = pkgs.lib.makeLibraryPath [
# Needed for pandas / numpy
pkgs.stdenv.cc.cc.lib
pkgs.zlib
# Needed for pygame
pkgs.glib
# Needed for matplotlib
pkgs.xorg.libX11
];
PYTHONBIN = "${pkgs.python38Full}/bin/python3.8";
LANG = "en_US.UTF-8";
};
}
Which means I executed libtool, autoconf, automake and cmake in Shell. I always choose a generic suggestion from Nix, without a specific version. Note: some commands may report errors as we executing them in a wrong way just to add to a replit.nix.
3.
Once build tools are set up we need to get and build TA-Lib C library sources. To do that execute in Shell:
git clone https://github.com/TA-Lib/ta-lib.git
then
cd ta-lib/
libtoolize
autoreconf --install
./configure
If configure script is completed without any problems, build the library with:
make -j4
It will end up with some compilation errors, but they are related to some additional tools which are used to add new TA-Lib indicators and build at the end, but not the library itself. The library will be successfully compiled and you should be able to see it with:
$ ls ./src/.libs/
libta_lib.a libta_lib.lai libta_lib.so.0
libta_lib.la libta_lib.so libta_lib.so.0.0.0
Now we have our C library built, but we can't install it to a system default folders. So we have to use the library as is from the folders where it was build. All we need is just one more additional preparation:
mkdir ./include/ta-lib
cp ./include/*.h ./include/ta-lib/
This will copy a library headers to a subfolder, as they are designed to be used from a such subfolder (which they don't have due to impossibility to perform the installation step).
4.
Now we have TA-Lib C library built and prepared to be used locally from its build folders. All we need after that - is to compile the Python wrapper for it. But Python wrapper will look for a library only in system default folders, so we need to instruct it where our library is.
To do this, execute pwd and remember the absolute path to your project's root folder. In my case it was:
/home/runner/FormalPleasedOffice
Then adjust the paths (there are two) in a following command to lead to your project path:
TA_INCLUDE_PATH=/home/runner/FormalPleasedOffice/ta-lib/include/ TA_LIBRARY_PATH=/home/runner/FormalPleasedOffice/ta-lib/src/.libs/ pip install ta-lib
This is one line command, not a two commands.If the paths would be shorter it would look like:
TA_INCLUDE_PATH=/path1/ TA_LIBRARY_PATH=/path2/ pip install ta-lib.
After execution of this command the wrapper will be installed with two additional paths where it will look for a library and its header files.
That's actually all.
An alternative way would be to clone the wrapper sources, edit its setup.py and install wrapper manually. Just for the record this would be:
cd ~/Your_project
git clone https://github.com/mrjbq7/ta-lib.git ta-lib-wrapper
cd ta-lib-wrapper
Here edit the setup.py. Find the lines include_dirs = [ and library_dirs = [ and append your paths to these lists. Then you just need to:
python setup.py build
pip install .
Note the dot at the end.
5.
Go to the project's folder and try our python script:
$python main.py
b'0.6.0-dev (Jan 1 1980 00:00:00)'
Bingo!

The #truf answer is correct.
after you add the
pkgs.libtool
pkgs.automake
pkgs.autoconf
pkgs.cmake
in the replit.nix dippendancies.
git clone https://github.com/TA-Lib/ta-lib.git
cd ta-lib/
libtoolize
autoreconf --install
./configure
make -j4
mkdir ./include/ta-lib
cp ./include/*.h ./include/ta-lib/
TA_INCLUDE_PATH=/home/runner/FormalPleasedOffice/ta-lib/include/ TA_LIBRARY_PATH=/home/runner/FormalPleasedOffice/ta-lib/src/.libs/ pip install ta-lib
Note : FormalPleasedOffice should be your project name
Done.
Here is the youtube video :
https://www.youtube.com/watch?v=u20y-nUMo5I

How to download executable(ANTLRv4) to generate files during pip install

The library I'm working on generates python files according to an executable (which turns ANTLRv4 .g4 files into python files), and I have the following install step:
import os
import subprocess
from setuptools import setup
from setuptools.command.install import install
class AntlrInstallCommand(install):
def run(self):
output_dir = compile_grammar()
print(f"Compiled ANTLRv4 grammar in {output_dir}")
install.run(self)
def compile_grammar():
parser_dir = os.path.join(os.path.dirname(__file__), "my_project/grammar")
subprocess.check_output(
["antlr", "MyGrammar.g4", "-Dlanguage=Python3", "-visitor", "-o", "gen"],
cwd=parser_dir,
)
# The result is created in the subfolder `gen`
return os.path.join(parser_dir, "gen")
setup(
...
install_requires=[
"setuptools",
"antlr4-python3-runtime==4.9.2",
...
],
cmdclass={"install": AntlrInstallCommand},
license="MIT",
python_requires=">=3.6",
)
Which works great if I'm pip install'ing the project on a machine that has antlr installed (since I'm calling it via subprocess).
Ideally, attempting to do this on a machine that doesn't have antlr installed would first install the executable(with the correct version) in either a system directory like /usr/bin, or whatever relevant python bin directory we're working in, but right now it errors out with the following message(which is expected):
running install
error: [Errno 2] No such file or directory: 'antlr'
----------------------------------------
ERROR: Failed building wheel for my_project
I see a couple of solutions each with slight caveats:
sympy uses ANTLR, but it requires the user to install antlr first. See here
setuptools-antlr allows me to download an antlr jar as a giant blob in a python package, and then I can invoke it here. However, the version doesn't match mine (which is 4.9.2).
java2python precompiles the files for me and writes them into the github repo. However, these files are extremely large and are very hard to read as they're autogenerated. If I slightly modify the grammar and don't modify the parser it would also lead to unexpected bugs. As a result, I would like to hide this complexity from the repository as it's tangential to development.
If I can get the right version of the antlr binary and be able to invoke it at install time, that would be optimal. Otherwise I'm okay with picking one of these alternatives. Any suggestions for either case would be appreciated.

Error installing/compiling grpc on Ubuntu

I have protobuf compiler version 3.0 and need to install grpc and grpc python plugin. Following the tutorial, I added deb http://http.debian.net/debian jessie-backports main to my sources.list file and did sudo apt-get update and sudo apt-get install libgrpc-dev which returned
Package libgrpc-dev is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'libgrpc-dev' has no installation candidate
So, I decided to compile it from source as mentioned in INSTALL notes and did:
$ git clone https://github.com/grpc/grpc.git
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
However, on the make step, I get
[MAKE] Generating cache.mk
make: Circular /home/vagrant/grpc2/grpc/libs/opt/libboringssl.a <- /home/vagrant/grpc2/grpc/libs/opt/libboringssl.a dependency dropped.
[C] Compiling third_party/boringssl/crypto/bio/connect.c
third_party/boringssl/crypto/bio/connect.c: In function 'split_host_and_port':
third_party/boringssl/crypto/bio/connect.c:127:17: error: declaration of 'close' shadows a global declaration [-Werror=shadow]
cc1: all warnings being treated as errors
make: *** [/home/vagrant/grpc2/grpc/objs/opt/third_party/boringssl/crypto/bio/connect.o] Error 1
On switching to the release-0_11 branch, running make results in
[HOSTCXX] Compiling src/compiler/csharp_generator.cc
src/compiler/csharp_generator.cc:47:43: error: 'google::protobuf::compiler::csharp::GetUmbrellaClassName' has not been declared
src/compiler/csharp_generator.cc: In function 'void grpc_csharp_generator::{anonymous}::GenerateServiceDescriptorProperty(grpc::protobuf::io::Printer*, const ServiceDescriptor*)':
src/compiler/csharp_generator.cc:237:62: error: 'GetUmbrellaClassName' was not declared in this scope
make: *** [/home/vagrant/grpc2/grpc/objs/opt/src/compiler/csharp_generator.o] Error 1
I can't figure out how to install this. Any help would be appreciated.

For me the issue got fixed after i made change in the file as:
diff --git a/src/compiler/csharp_generator.cc
b/src/compiler/csharp_generator.cc
index 7b497df..5a8746d 100644
--- a/src/compiler/csharp_generator.cc
+++ b/src/compiler/csharp_generator.cc
## -44,7 +44,7 ##
using google::protobuf::compiler::csharp::GetFileNamespace;
using google::protobuf::compiler::csharp::GetClassName;
-using google::protobuf::compiler::csharp::GetUmbrellaClassName;
+using google::protobuf::compiler::csharp::GetReflectionClassName;
using grpc::protobuf::FileDescriptor;
using grpc::protobuf::Descriptor;
using grpc::protobuf::ServiceDescriptor;
## -234,7 +234,7 ## void GenerateServiceDescriptorProperty(Printer* out, const ServiceDescriptor *se
out->Print("public static global::Google.Protobuf.Reflection.ServiceDescriptor Descriptor\n");
out->Print("{\n");
out->Print(" get { return $umbrella$.Descriptor.Services[$index$]; }\n",
- "umbrella", GetUmbrellaClassName(service->file()), "index",
+ "umbrella", GetReflectionClassName(service->file()), "index",
index.str());
out->Print("}\n");
out->Print("\n");
More specifically, please open the file src/compiler/csharp_generator.cc and replace all references of GetUmbrellaClassName by GetReflectionClassName

Working backwards:
for release-0_11: it looks like you're trying to compile against the most recent protobuf. Since we're both in development right now there's occasional breakage - but grpc does track the version of protobuf it's tested against in third_party/protobuf. Try checking out and installing that version. I filed https://github.com/grpc/grpc/issues/4697 to update to the latest protobuf 3.0 version.
for master from github: which compiler and OS are you using? I recently checked in the boringssl integration work, so it's fresh, and not nearly as battle-tested. I'd like to get it battle-tested. That said, if you do a 'make EMBED_OPENSSL=false' then things should work out for you.
for the debian package problem: I'm not sure what's going on. I'm happy to try and spin up a VM with your OS and repro if you can let me know which OS it is.

How to fix "python does not have .gnu.prelink_undo section"

I need to package a virtualenv as an rpm. I found a sample spec file for plone here
My project uses python 2.7 and for that I've built python from source. Therefore I changed some of the spec file to
/usr/local/bin/virtualenv-3.4 --no-site-packages --distribute %{_builddir}/usr/local/virtualenvs/%{shortname}
I'm getting the following error on rpmbuild -bb requirements.spec
+ /usr/sbin/prelink -u /var/lib/jenkins/rpmbuild/BUILDROOT/requirements-1.0-1.x86_64/usr/local/virtualenvs/requirements/bin/python
/usr/sbin/prelink: /var/lib/jenkins/rpmbuild/BUILDROOT/requirements-1.0-1.x86_64/usr/local/virtualenvs/requirements/bin/python does not have .gnu.prelink_undo section
I'm assuming I need to rebuild python and enable the prelinking during the ./configure. How can I do that?

I had a similar issue recently with a SPEC file that was also based on this example from plone.
In my case I'm using python27 RPMs from IUS repository and want to avoid building it from source.
My workaround was to disable prelink completely in my SPEC file:
add this: %define __prelink_undo_cmd %{nil}
comment out this:
# # This avoids prelink & RPM helpfully breaking the package signatures:
# /usr/sbin/prelink -u $RPM_BUILD_ROOT/usr/local/virtualenvs/%{shortname}/bin/python