awscli fails to execute within python virtual environment - python

I'm really bad with python. This is on a CentOS7 vm
Problem:
When trying to use awscli in a python virtual environment, I get an error:
(python3ve) [user#ncwv-jlnxnode01 ~]$ aws --version
Traceback (most recent call last):
File "/home/user/venv/python3ve/bin/aws", line 27, in
sys.exit(main())
File "/home/user/venv/python3ve/bin/aws", line 23, in main
return awscli.clidriver.main()
File "/home/user/venv/python3ve/lib64/python3.6/site-packages/awscli/clidriver.py", line 69, in main
driver = create_clidriver()
File "/home/user/venv/python3ve/lib64/python3.6/site-packages/awscli/clidriver.py", line 79, in create_clidriver
event_hooks=session.get_component('event_emitter'))
File "/home/user/venv/python3ve/lib64/python3.6/site-packages/awscli/plugin.py", line 44, in load_plugins
modules = _import_plugins(plugin_mapping)
File "/home/user/venv/python3ve/lib64/python3.6/site-packages/awscli/plugin.py", line 61, in _import_plugins
module = __import__(path, fromlist=[module])
ModuleNotFoundError: No module named '/root/'
ultimately i'm trying to put together a step by step method in an ansible playbook for not only installing awscli, but also awscli-plugin-endpoint, so i'd prefer to install this through pip instead of the centos repos and instead of just downloading the binaries.
Installation Steps:
remove python3 and everything python3 related on the system.
~$ rm -rf ~/venv/python3ve/
~$ sudo yum remove -y python3
~$ sudo yum autoremove -y
~$ sudo find / -name "python3*" > ~/file
~$ sudo xargs rm -r ~/file (missing the arrow because stackoverflow formatting is freaking out with it)
install
~$ sudo yum install -y python3
~$ /usr/bin/python3 -m venv ~/venv/python3ve
~$ source ~/venv/python3ve/bin/activate
~$ ~/venv/python3ve/bin/python3 -m pip install --upgrade pip
~$ ~/venv/python3ve/bin/python3 -m pip install --upgrade awscli
~$ which aws
~/venv/python3ve/bin/aws
~$ aws --version
---output is in the problem description above---
suggestions?

ultimately found that the error was stemming from my ~/.aws/config which I wasnt removing when I reinstalled. that had a reference to the plugin not yet installed and also the old site-packages path (pre venv)
cli_legacy_plugin_path=/root/.local/lib/python3.6/site-packages/
endpoint = awscli_plugin_endpoint
Once I removed those, it worked fine again.
~$ aws --version
aws-cli/1.24.10 Python/3.6.8 Linux/3.10.0-957.el7.x86_64 botocore/1.26.10
The error was referencing /root/ because of how _import_plugins within /awscli/plugin.py splits the path based on . if present
module = path.rsplit('.', 1)

Related

Can't install Python modules in Gitlab CI

I can't get Gst (or seemingly any Python module) to install inside Gitlab CI
Here is my .gitlab-ci.yml file
image: "python:3.7"
before_script:
- apt-get -qq update
- apt-get -qq install -y python3-dev python3-pip libgirepository1.0-dev
- python3 --version
- python3 -m pip install --upgrade pip
- pip3 install -r requirements.txt
test:
script:
- pip3 install flake8 # you can also use tox
- flake8 --max-line-length=254 --extend-ignore=F401,E402 matinee.py
run:
script:
- ./matinee.py -h
But no matter what I tried the closest I got to was:
$ ./matinee.py -h
Traceback (most recent call last):
File "./matinee.py", line 6, in <module>
gi.require_version('Gst', '1.0')
File "/usr/local/lib/python3.7/site-packages/gi/__init__.py", line 126, in require_version
raise ValueError('Namespace %s not available' % namespace)
ValueError: Namespace Gst not available
Before that it complained that gi didn't exist (?) so basically it seems like python modules are not installable this way.
It's been a week now, and frankly I'm giving up on setting up Gitlab CI for python ; I have nodejs projects with tons of deps passing installing without a glitch ; This post is my last hail Mary attempt :|
Naturally, everything works perfectly locally (Ubuntu 22)
My requirements file
# cat requirements.txt
PyGObject==3.42.1
What I tried
image: "python:3.7"
image: "python:latest"
image: "ubuntu:latest"
venv
And everything on this page. Please, help me understand why I can't install Python modules in Gitlab CI.

I tried to run a python script, but got ImportError: attempted relative import with no known parent package

I found other stackoverflow questions and answers for this error, but none of them worked for me. My python package problem:
I tried to install packages with these commands so I could run tests:
$ pip3 install virtualenv
$ virtualenv -p $(which python3.9) venv
$ source venv/bin/activate
$ pip install --upgrade pip # stop local environments from messing things up
$ pip install -r path/to/requirements.txt
I tried to run my tests:
$ python -m unittest discover
I got this error:
======================================================================
ERROR: some_file (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: some_file
Traceback (most recent call last):
File "/usr/local/opt/python#3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
module = self._get_module_from_name(name)
File "/usr/local/opt/python#3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
__import__(name)
File "/Users/path/some_file.py", line 3, in <module>
from .other_file import some_function, SomeClass
ImportError: attempted relative import with no known parent package
----------------------------------------------------------------------
As the error said, I was trying to use a relative path in my file:
from .other_file import some_function, SomeClass
I don't know about other versions of python, but I expected this to be possible in python3.
I needed to add this line to my installation commands:
$ pip install --editable .
That is:
$ pip3 install virtualenv
$ virtualenv -p $(which python3.9) venv
$ source venv/bin/activate
$ pip install --upgrade pip # stop local environments from messing things up
$ pip install -r path/to/requirements.txt
$ pip install --editable .
I don't yet understand why it works. I'll update this answer if I find out more.

python install pip with error "ModuleNotFoundError: No module named '_struct'"

I install python from source:
$ wget -c https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz
$ tar xf Python-3.6.1.tar.xz
$ cd Python-3.6.1
$ ./configure --enable-optimizations --prefix=$HOME/.local/ --with-pydebug
$ make altinstall -j8
Then I install pip with get-pip.py
$ python3.6 get-pip.py --prefix=~/.local/
I met the error of:
Traceback (most recent call last): File "get-pip.py", line 27, in
import struct File "/home/zhangzy/.local/lib/python3.6/struct.py", line 13, in
from _struct import * ModuleNotFoundError: No module named '_struct'
What is the cause of this problem and how could it solve it?
You should always use your system package manager when possible. There is a safe way to get newer versions of Python on CentOS/RHEL, as proper RPM packages. It is called the IUS project, and we offer multiple versions of Python.
After setting up the repo, you can get Python 3.5 by running the command yum install python35u. You will then have the command python3.5 available, as well as the pyvenv-3.5 command for creating virtual environments.

Error in installing PyQt

I am following this manual for installing PyQt in Mac but I got the following error:
Monas-MacBook-Pro:sip-4.16.7 mona$ sudo make install
Password:
cp -f sip /Library/Frameworks/Python.framework/Versions/3.4/bin/sip
cp -f sip.so /Library/Python/3.3/site-packages/sip.so
cp -f /Users/mona/OpenSource/sip-4.16.7/siplib/sip.h /Library/Frameworks/Python.framework/Versions/3.4/include/python3.4m/sip.h
cp -f sipconfig.py /Library/Python/3.3/site-packages/sipconfig.py
cp -f /Users/mona/OpenSource/sip-4.16.7/sipdistutils.py /Library/Python/3.3/site-packages/sipdistutils.py
Monas-MacBook-Pro:sip-4.16.7 mona$ cd ..
Monas-MacBook-Pro:OpenSource mona$ ls
PyQt-mac-gpl-4.11.3 sip-4.16.7
PyQt-mac-gpl-4.11.3.tar.gz sip-4.16.7.tar.gz
Monas-MacBook-Pro:OpenSource mona$ cd PyQt-mac-gpl-4.11.3
Monas-MacBook-Pro:PyQt-mac-gpl-4.11.3 mona$ python3 configure.py -d /Library/Python/3.3/site-packages/ --use-arch x86_64
Traceback (most recent call last):
File "configure.py", line 32, in <module>
import sipconfig
ImportError: No module named 'sipconfig'
Any idea how to fix it?
To get the new version of pyqt wich is pyqt5, you start by installing a version of python 3.5 https://www.python.org/downloads/release/python-350/
And you can run the command line: pip3 install pyqt5 (sometimes you will need to write it in the correct format so it will look like : pip3 install PyQt5)
Otherwise you can use brew, first install brew :/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
And when it's done. run : brew install pyqt
It should work

mysqlfailover: No module named mysql.utilities.common.tools

I am trying to setup mysqlfailover utility on ec2 instance running Ubuntu 12.04. I performed the following steps:
Downloaded mysql-connector-python:
wget http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python_1.0.12-1debian6.0_all.deb -o mysql-connector-python_1.0.12-1debian6.0_all.deb
(Installed using dpkg -i command.)
Download mysql-utilities:
wget http://cdn.mysql.com/Downloads/MySQLGUITools/mysql-utilities_1.3.5-1debian6.0_all.deb -o mysql-utilities_1.3.5-1debian6.0_all.deb
(Installed using dpkg -i command.)
There were some dependencies issues. I ran the following commands:
sudo apt-get install python-pip
easy_install -U distribute
sudo apt-get install libmysqlclient-dev
sudo apt-get install python-dev
pip install mysql-python
Now, when I run the command mysqlfailover, it gives the following error:
Traceback (most recent call last):
File "/usr/bin/mysqlfailover", line 24, in <module>
from mysql.utilities.common.tools import check_python_version
ImportError: No module named mysql.utilities.common.tools
I have been banging my head around quite some time now, but no success. Any help in the matter is highly appreciated. Thanks.
Try env PYTHONPATH=/usr/share/pyshared/ mysqlfailover
Thers is no utilities in mysql modul, you can check this do:
ls /usr/lib/python2.7/dist-packages/mysql
So you should change mysql package:
cp -r /usr/share/pysharedmysql /usr/lib/python2.7/dist-packages/mysql

Categories

Resources