When executing python3 (Python 3.6.8) script on a local directory, it works well, but when running sbatch job in slurm, complains about certifi.
python3 -m pip install certifi
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: certifi in /usr/local/lib/python3.6/site-packages (2020.12.5)
After adding to the python code this:
import sys
import os
sys.path.append(os.getcwd())
or this:
import sys
import os
module_path = os.path.abspath(os.getcwd())
if module_path not in sys.path:
sys.path.append(module_path)
the same error occurs. It seems that certifi is installed.
pip show certifi
Name: certifi
Version: 2020.12.5
Summary: Python package for providing Mozilla's CA Bundle.
Home-page: https://certifiio.readthedocs.io/en/latest/
Author: Kenneth Reitz
Author-email: me#kennethreitz.com
License: MPL-2.0
Location: /usr/local/lib/python3.6/site-packages
Requires:
Required-by: requests
The error after running python code (without having the line 'import certifi' in python code):
Traceback (most recent call last):
File "/home/username/test/test.py", line 19, in <module>
from textattack.augmentation import WordNetAugmenter, EmbeddingAugmenter, EasyDataAugmenter, CharSwapAugmenter
File "/home/username/.local/lib/python3.6/site-packages/textattack/__init__.py", line 12, in <module>
from . import (
File "/home/username/.local/lib/python3.6/site-packages/textattack/attack_recipes/__init__.py", line 21, in <module>
from .attack_recipe import AttackRecipe
File "/home/username/.local/lib/python3.6/site-packages/textattack/attack_recipes/attack_recipe.py", line 9, in <module>
from textattack.shared import Attack
File "/home/username/.local/lib/python3.6/site-packages/textattack/shared/__init__.py", line 11, in <module>
from . import utils
File "/home/username/.local/lib/python3.6/site-packages/textattack/shared/utils/__init__.py", line 1, in <module>
from .install import *
File "/home/username/.local/lib/python3.6/site-packages/textattack/shared/utils/install.py", line 9, in <module>
import requests
File "/home/username/.local/lib/python3.6/site-packages/requests/__init__.py", line 118, in <module>
from . import utils
File "/home/username/.local/lib/python3.6/site-packages/requests/utils.py", line 25, in <module>
from . import certs
File "/home/username/.local/lib/python3.6/site-packages/requests/certs.py", line 15, in <module>
from certifi import where
ModuleNotFoundError: No module named 'certifi'
The error (with having the line 'import certifi' in python code):
Traceback (most recent call last):
File "/home/username/projecttest_LR_attack/LR_attack.py", line 17, in <module>
import certifi
ModuleNotFoundError: No module named 'certifi'
What could be the solution to the issue?
for anyone on MacOS and already ran pip install certifi and still doesn't work
Go to your applications folder > find your python version folder -> double click on the file Install Certificates.command inside the python folder to install the certificate.
wait for it to complete the installation
After that, you can try running your code with requests package again
Are the same modules installed on the compute nodes as locally? You may need to check with the Slurm admins.
This could mean that /usr/local/lib/python3.6/site-packages/ is not your PYTHONPATH environment variable that sbatch job in slurm has access to. You can either add it or append it during runtime:
import sys
sys.path.append('/usr/local/lib/python3.6/site-packages/')
pip install certifi
Try this command on the CLI
Related
I have trouble running a (working) python-script as the user www-data on a standard Raspi4-Debian-installation, which is necessary, since I want to run it from a php-file. I found a lot of questions regarding the same here, but I can't get it to work sadly. I'd be happy if somebody could help.
So, when I run my python-script on the commandline as myself it works perfectly.
When I start it with "sudo" oder "sudo -u www-data" I get the following error:
File "/usr/local/lib/python3.9/dist-packages/firebase_admin/firestore.py", line 22, in <module>
from google.cloud import firestore # pylint: disable=import-error,no-name-in-module
File "/usr/local/lib/python3.9/dist-packages/google/cloud/firestore.py", line 18, in <module>
from google.cloud.firestore_v1 import __version__
File "/usr/local/lib/python3.9/dist-packages/google/cloud/firestore_v1/__init__.py", line 30, in <module>
from google.cloud.firestore_v1._helpers import GeoPoint
File "/usr/local/lib/python3.9/dist-packages/google/cloud/firestore_v1/_helpers.py", line 22, in <module>
from google.api_core import gapic_v1 # type: ignore
File "/usr/local/lib/python3.9/dist-packages/google/api_core/gapic_v1/__init__.py", line 16, in <module>
from google.api_core.gapic_v1 import config
File "/usr/local/lib/python3.9/dist-packages/google/api_core/gapic_v1/config.py", line 23, in <module>
import grpc
File "/usr/local/lib/python3.9/dist-packages/grpc/__init__.py", line 22, in <module>
from grpc import _compression
File "/usr/local/lib/python3.9/dist-packages/grpc/_compression.py", line 15, in <module>
from grpc._cython import cygrpc
ImportError: /usr/local/lib/python3.9/dist-packages/grpc/_cython/cygrpc.cpython-39-arm-linux-gnueabihf.so: undefined symbol: __atomic_exchange_8
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/var/www/html/private/./weddingr_uploader2.py", line 26, in <module>
from firebase_admin import credentials, initialize_app, storage, firestore
File "/usr/local/lib/python3.9/dist-packages/firebase_admin/firestore.py", line 28, in <module>
raise ImportError('Failed to import the Cloud Firestore library for Python. Make sure '
ImportError: Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module.
So I researched and thought: "okay, the user www-data" can't access the module google-cloud-firestore". I tried to deinstall it with
sudo pip3 uninstall google-cloud-firestore"
and installing it again with
sudo pip3 install google-cloud-firestore"
Although it tells me uninstall is successfull, when I install it again, it tells me, the requirement is already satisfied:
...
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.9/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3.0dev,>=1.25.0->google-api-core[grpc]!=2.0.*,!=2.1.*,!=2.2.*,!=2.3.0,<3.0.0dev,>=1.31.5->google-cloud-firestore) (0.4.8)
Installing collected packages: google-cloud-firestore
Successfully installed google-cloud-firestore-2.5.3
I am running out of ideas, how to deinstall the module as my user and install it with su-rights in the correct path. Has somebody a hint in the right direction for me? Thanks!
I wanted to run a google.cloud.storage dependency to my project so I tried to install this dependency with
sudo pip install --upgrade google-api-python-client
Running the above command, we installed all the required packages in the apache airflow dist-packages directory ex:- /usr/local/python2.7/dist-packages/
When i imported libraries into my project using the below code.
from airflow.contrib.operators.mysql_to_gcs import MySqlToGoogleCloudStorageOperator
running the project on Python, i am getting the error ImportError: No module named api_core.exceptions
here is the trace
#python mysql_to_gcs.py
[2019-04-30 06:45:09,668] {settings.py:182} INFO - settings.configure_orm(): Using pool settings. pool_size=5, pool_recycle=1800, pid=15440
Traceback (most recent call last):
File "mysql_to_gcs.py", line 2, in <module>
from airflow.contrib.operators.mysql_to_gcs import MySqlToGoogleCloudStorageOperator
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/operators/mysql_to_gcs.py", line 25, in <module>
from airflow.contrib.hooks.gcs_hook import GoogleCloudStorageHook
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcs_hook.py", line 25, in <module>
from airflow.contrib.hooks.gcp_api_base_hook import GoogleCloudBaseHook
File "/usr/local/lib/python2.7/dist-packages/airflow/contrib/hooks/gcp_api_base_hook.py", line 30, in <module>
from google.api_core.exceptions import GoogleAPICallError, AlreadyExists, RetryError
ImportError: No module named api_core.exceptions
Instead of --upgrade google-api-python-client Just reinstall api-core using
pip install dialogflow
Upgrade has some weird problems which does not update api-core.
I'm trying to execute ansible in my system. I guess I messed up something and unable to run ansible. When I run ansible --version, I see the following error:
krishnapatamset:bin krishna.patamsetti$ ansible --version
[WARNING]: log file at /var/log/ansible/ansible.log is not writeable and we cannot create it, aborting
ERROR! Unexpected Exception: No module named markupsafe
the full traceback was:
Traceback (most recent call last):
File "/usr/local/bin/ansible", line 80, in <module>
from ansible.cli.adhoc import AdHocCLI as mycli
File "/usr/local/lib/python2.7/site-packages/ansible/cli/adhoc.py", line 28, in <module>
from ansible.executor.task_queue_manager import TaskQueueManager
File "/usr/local/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 29, in <module>
from ansible.executor.play_iterator import PlayIterator
File "/usr/local/lib/python2.7/site-packages/ansible/executor/play_iterator.py", line 30, in <module>
from ansible.playbook.block import Block
File "/usr/local/lib/python2.7/site-packages/ansible/playbook/__init__.py", line 27, in <module>
from ansible.playbook.play import Play
File "/usr/local/lib/python2.7/site-packages/ansible/playbook/play.py", line 27, in <module>
from ansible.playbook.base import Base
File "/usr/local/lib/python2.7/site-packages/ansible/playbook/base.py", line 29, in <module>
from jinja2.exceptions import UndefinedError
File "/usr/local/lib/python2.7/site-packages/jinja2/__init__.py", line 33, in <module>
from jinja2.environment import Environment, Template
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 13, in <module>
from jinja2 import nodes
File "/usr/local/lib/python2.7/site-packages/jinja2/nodes.py", line 19, in <module>
from jinja2.utils import Markup
File "/usr/local/lib/python2.7/site-packages/jinja2/utils.py", line 531, in <module>
from markupsafe import Markup, escape, soft_unicode
ImportError: No module named markupsafe
Can anyone please help me with this situation. Thanks in advance
I even tried pip install markupsafe but still I am unable to get ansible working
Ansible is based on python and there is a python module that's missing called markupsafe
install it via pip, you may need to escalate privileges
pip2 install -I markupsafe
pip2 for python 2.x
-I to ignore and reinstall package if it's already installed
I uninstalled pip and installed it again.
brew uninstall pip
brew doctor
brew install pip
I am trying to install CKAN onto Ubuntu 13.10. (64bit) following these instructions. Python 2.7.5+ is installed on the machine. The following errors stop me from installing the CKAN package:
$ sudo dpkg -i python-ckan_2.2_amd64.deb
Selecting previously unselected package python-ckan.
(Reading database ... 289528 files and directories currently installed.)
Unpacking python-ckan (from python-ckan_2.2_amd64.deb) ...
Setting up python-ckan (2.2-1) ...
Traceback (most recent call last):
File "/usr/lib/ckan/default/bin/paster", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/ckan/default/local/lib/python2.7/site-packages/ \
distribute-0.6.24-py2.7.egg/pkg_resources.py", line 16, in <module>
import sys, os, zipimport, time, re, imp, types
File "/usr/lib/ckan/default/lib/python2.7/re.py", line 105, in <module>
import sre_compile
File "/usr/lib/ckan/default/lib/python2.7/sre_compile.py", line 14, in <module>
import sre_parse
File "/usr/lib/ckan/default/lib/python2.7/sre_parse.py", line 17, in <module>
from sre_constants import *
File "/usr/lib/ckan/default/lib/python2.7/sre_constants.py", line 18, in <module>
from _sre import MAXREPEAT
ImportError: cannot import name MAXREPEAT
System start/stop links for /etc/init.d/nginx already exist.
System start/stop links for /etc/init.d/apache2 already exist.
ERROR: Site ckan_default does not exist!
ERROR: Site datapusher does not exist!
Finally, I installed CKAN from source as suggested by the documentation:
This is the quickest and easiest way to install CKAN, but it requires Ubuntu 12.04 64-bit.
I'm following this post to make one ipython rule all the virtualenvs.
From what I understand, the main idea of the post is that when in a virtualenv, ipython can not find its modules.
(a-virtualenv)me#pc:~$ ipython
Traceback (most recent call last):
File "/usr/bin/ipython", line 19, in <module>
from IPython.frontend.terminal.ipapp import launch_new_instance
ImportError: No module named IPython.frontend.terminal.ipapp
To work around this, I add this to my /usr/bin/ipython
import sys
if "/usr/lib/python2.7/dist-packages" not in sys.path:
sys.path.append("/usr/lib/python2.7/dist-packages")
This trick works for Balthazar. But in my case, I get this
Traceback (most recent call last):
File "/usr/bin/ipython", line 18, in <module>
from IPython.frontend.terminal.ipapp import launch_new_instance
File "/usr/lib/python2.7/dist-packages/IPython/__init__.py", line 46, in <module>
from .frontend.terminal.embed import embed
File "/usr/lib/python2.7/dist-packages/IPython/frontend/terminal/embed.py", line 37, in <module>
from IPython.frontend.terminal.ipapp import load_default_config
File "/usr/lib/python2.7/dist-packages/IPython/frontend/terminal/ipapp.py", line 38, in <module>
from IPython.core.completer import IPCompleter
File "/usr/lib/python2.7/dist-packages/IPython/core/completer.py", line 84, in <module>
from IPython.utils import generics
File "/usr/lib/python2.7/dist-packages/IPython/utils/generics.py", line 19, in <module>
from IPython.external.simplegeneric import generic
File "/usr/lib/python2.7/dist-packages/IPython/external/simplegeneric/__init__.py", line 4, in <module>
from _simplegeneric import *
ImportError: No module named _simplegeneric
I thought maybe I should add /usr/lib/python2.7/dist-packages/IPython/external to sys.path in /usr/bin/ipython as well. What I get is
Traceback (most recent call last):
File "/usr/bin/ipython", line 18, in <module>
from IPython.frontend.terminal.ipapp import launch_new_instance
File "/usr/lib/python2.7/dist-packages/IPython/__init__.py", line 43, in <module>
from .config.loader import Config
File "/usr/lib/python2.7/dist-packages/IPython/config/loader.py", line 27, in <module>
from IPython.utils.path import filefind, get_ipython_dir
File "/usr/lib/python2.7/dist-packages/IPython/utils/path.py", line 24, in <module>
from IPython.utils.process import system
File "/usr/lib/python2.7/dist-packages/IPython/utils/process.py", line 27, in <module>
from ._process_posix import _find_cmd, system, getoutput, arg_split
File "/usr/lib/python2.7/dist-packages/IPython/utils/_process_posix.py", line 27, in <module>
from IPython.utils import text
File "/usr/lib/python2.7/dist-packages/IPython/utils/text.py", line 27, in <module>
from IPython.external.path import path
ImportError: cannot import name path
This error occurred on line 43 of IPython/__init__py, but the previous one occurred on line 46.
How can I make ipython work with virtualenv?
How can adding /usr/lib/python2.7/dist-packages/IPython/external to sys.path resulting a new error?
What should I read to get a better understanding of python import path?
Thanks!
PS:
normal IPython sys.path
['',
'/usr/bin',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-linux2',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages',
'/usr/lib/python2.7/dist-packages/PIL',
'/usr/lib/python2.7/dist-packages/gst-0.10',
'/usr/lib/python2.7/dist-packages/gtk-2.0',
'/usr/lib/pymodules/python2.7',
'/usr/lib/python2.7/dist-packages/ubuntu-sso-client',
'/usr/lib/python2.7/dist-packages/ubuntuone-client',
'/usr/lib/python2.7/dist-packages/ubuntuone-control-panel',
'/usr/lib/python2.7/dist-packages/ubuntuone-couch',
'/usr/lib/python2.7/dist-packages/ubuntuone-installer',
'/usr/lib/python2.7/dist-packages/ubuntuone-storage-protocol',
'/usr/lib/python2.7/dist-packages/IPython/extensions']
IPython seems to work now!
Because ipython can not find simplegeneric. I tried to locate simplegeneric and found simplegeneric is in '/usr/lib/pymodules/python2.7'. After '/usr/lib/pymodules/python2.7' being added, ipython works fine.
Jeff Tratner's comment really helps!
If you install ipython3 using apt-get, you would not face any problems like that.
Instead of ipython, you should type ipython3 on the terminal.
For Python 2 use ipython2.
Making changes to the path did not help in any way. However running ipython as a super user worked (although I'm not proud of it!)
sudo ipython
pip install ipython instead of apt-get install ipython. The former installs into the virtualenv, the latter installs into your system python packages.
I had the same problem, in my case the cause of it was that the directory: /usr/lib/python2.7/dist-packages was not added to the path when the virtual environment was created (I would like to know why)
In my case the following in the terminal solved the problem:
workon 'your_environment_here'
add2virtualenv /usr/lib/python2.7/dist-packages
Note: This assumes you are using virtualenvwrapper
In my case, it was a package higher up in my customized Python path named "path" that was causing the ImportError from:
import IPython.external.path as path
See https://github.com/ipython/ipython/issues/3994
I experienced some weirdness in a new ubuntu host, and realized I was getting different behaviors for mkvirtualenv on mac versus the ubuntu. I got past this difference with
'mkvirtualenv --system-site-packages mynewenv'
For me, the solution was to upgrade IPython from 0.13.2 to the currently newest 4.0.0.
Similar traceback I experienced with python 3.5.4 and pip 10.0.1 after installing ipython successfully.
line 32, in <module>
from .debugger import TerminalPdb, Pdb
File "/usr/lib/python3.5/site-packages/IPython/terminal/debugger.py", line 6, in <module>
from IPython.core.completer import IPCompleter
File "/usr/lib/python3.5/site-packages/IPython/core/completer.py", line 137, in <module>
from IPython.utils import generics
File "/usr/lib/python3.5/site-packages/IPython/utils/generics.py", line 8, in <module>
from simplegeneric import generic
ImportError: No module named 'simplegeneric'
updating or installing the simplegeneric i.e sudo pip install simplegeneric --upgrade I get
Requirement already up-to-date: simplegeneric in /usr/lib/python3.5/site-packages (0.8.1)
After some initial trial I searched on google/stackoverflow to see if anyone else faced similar problem and found this. And its #zjk answer that helps me to find solution.
Solutions:
Found only egg folder in the python3.5 installed package dir. So I removed them and fresh install the simplegeneric.
sudo rm -rf /usr/lib/python3.5/site-packages/simplegeneric-0.8.1-py3.5.egg-info/
sudo pip install simplegeneric
after that ipython worked just like charm.
N.B: I install the package at the system level that's why sudo required.