Develop Reference

Huggingface Transformers not getting imported in VS Code

My VS Code Editor for Python is not able to import transformers even though I have done a conda install and giving me the following error
Traceback (most recent call last):
File "c:/Users/I323017/Documents/Studies/question_answering_kinnal/src/main.py", line 3, in <module>
import transformers
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\transformers\__init__.py", line 107, in <module>
from .pipelines import (
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\transformers\pipelines.py", line 40, in <module>
from .tokenization_auto import AutoTokenizer
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\transformers\tokenization_auto.py", line 49, in <module>
from .tokenization_flaubert import FlaubertTokenizer
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\transformers\tokenization_flaubert.py", line 23, in <module>
from .tokenization_xlm import XLMTokenizer
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\transformers\tokenization_xlm.py", line 26, in <module>
import sacremoses as sm
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\sacremoses\__init__.py", line 2, in <module>
from sacremoses.tokenize import *
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\sacremoses\tokenize.py", line 10, in <module>
from sacremoses.util import is_cjk
File "C:\Users\I323017\AppData\Local\Continuum\anaconda3\lib\site-packages\sacremoses\util.py", line 11, in <module>
from joblib import Parallel, delayed
ModuleNotFoundError: No module named 'joblib'
May I know the problem here?

The error clearly suggests:
ModuleNotFoundError: No module named 'joblib'
Try pip install joblib
Also make sure you have latest torch and transformers library installed.

This seems to be an installation issue. If you have already installed transformers using conda install -c conda-forge transformers, an additional upgradation from the source using the below resolved my issue.
git clone https://github.com/huggingface/transformers.git
cd transformers
pip install .

As #Vishnukk has stated, this seems like an installation problem.
HuggingFace has now published transformers officially via their own conda channel
Doing conda install transformers -c huggingface should then work after removing the old version of transformers.

AttributeError: module 'importlib._bootstrap' has no attribute 'SourceFileLoader'

Im new to programming, so I have met this trouble.
I ve upgraded my pip and then used pip list, ipython notebook, jupyter notebook to use anaconda, so after all this commands I got one answer :
Traceback (most recent call last):
File "/home/bakhytgul/anaconda3/bin/pip", line 7, in <module>
from pip._internal import main
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/__init__.py", line 40, in <module>
from pip._internal.cli.autocompletion import autocomplete
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/cli/autocompletion.py", line 8, in <module>
from pip._internal.cli.main_parser import create_main_parser
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/cli/main_parser.py", line 12, in <module>
from pip._internal.commands import (
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/commands/__init__.py", line 6, in <module>
from pip._internal.commands.completion import CompletionCommand
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/commands/completion.py", line 6, in <module>
from pip._internal.cli.base_command import Command
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/cli/base_command.py", line 19, in <module>
from pip._internal.download import PipSession
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_internal/download.py", line 15, in <module>
from pip._vendor import requests, six, urllib3
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_vendor/requests/__init__.py", line 97, in <module>
from pip._vendor.urllib3.contrib import pyopenssl
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pip/_vendor/urllib3/contrib/pyopenssl.py", line 46, in <module>
import OpenSSL.SSL
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/OpenSSL/__init__.py", line 8, in <module>
from OpenSSL import rand, crypto, SSL
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/OpenSSL/crypto.py", line 13, in <module>
from cryptography.hazmat.primitives.asymmetric import dsa, rsa
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/primitives/asymmetric/rsa.py", line 14, in <module>
from cryptography.hazmat.backends.interfaces import RSABackend
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/cryptography/hazmat/backends/__init__.py", line 7, in <module>
import pkg_resources
File "/home/bakhytgul/anaconda3/lib/python3.6/site-packages/pkg_resources.py", line 1435, in <module>
register_loader_type(importlib_bootstrap.SourceFileLoader, DefaultProvider)
AttributeError: module 'importlib._bootstrap' has no attribute 'SourceFileLoader'
Can u help me to solve this issue, please? Thank u very much.
Im using:
- Ubuntu 16.04. (64bit)
- Python 2.7.12
- Python 3.6.0 :: Anaconda 4.3.1 (64-bit)
P.S: Ive tried to fix with dotenv, ensurepip(this got me answer that I dont have this package).

Not a full answer because I cannot tell you why this is happening, but hopefully this will help you find a solution.
I had been running into this problem with an venv. When I started from scratch trying to build rjsmin, everything went all right, that is when I noticed that the reason was that the venv that was failing was using Python 3.6, whereas the new venv was using Python 3.7.
According to your console output the anaconda you have is using 3.6, I think you might have break your install by manually upgrading packages using pip. You could try getting the newer version of anaconda, or go with a new, fresh install.

"No module named zlib" error anytime I use pip

I'm using terminal on mac, and anytime I attempt to use pip, I get the following error message telling me that it cannot find a module named zlib:
$ pip --version
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/packages/__init__.py", line 27, in <module>
from . import urllib3
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/__init__.py", line 8, in <module>
from .connectionpool import (
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/connectionpool.py", line 42, in <module>
from .response import HTTPResponse
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/packages/urllib3/response.py", line 3, in <module>
import zlib
ModuleNotFoundError: No module named 'zlib'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.5/bin/pip", line 7, in <module>
from pip import main
File "/usr/local/lib/python3.6/site-packages/pip/__init__.py", line 21, in <module>
from pip._vendor.requests.packages.urllib3.exceptions import DependencyWarning
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/__init__.py", line 62, in <module>
from .packages.urllib3.exceptions import DependencyWarning
File "/usr/local/lib/python3.6/site-packages/pip/_vendor/requests/packages/__init__.py", line 29, in <module>
import urllib3
File "/usr/local/lib/python3.6/site-packages/urllib3/__init__.py", line 8, in <module>
from .connectionpool import (
File "/usr/local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 36, in <module>
from .response import HTTPResponse
File "/usr/local/lib/python3.6/site-packages/urllib3/response.py", line 3, in <module>
import zlib
ModuleNotFoundError: No module named 'zlib'
I've read several SO posts with the same error message, and have tried their solutions, including attempting to install zlib using homebrew, which fails, and updating my Xcode developer tools. I also installed pip earlier using python get-pip.py, and it said it installed successfully.
The reason I need pip is to install a package for use on python 2.7, but doing $ pip install "package-name" produces the same error as above. I have both Python 2 and 3 installed on my machine, if that helps. I'm kind of new to pip and using the command line in general, and I'm super confused; if anyone could help me out I'd really appreciate it.

This is the OP. I was able to fix the issue, and wanted to post my solution here in case it would help others.
Essentially I believe it occurred because Python2 is default on my machine, but pip is by default pointing to resources pertaining to Python3, or something like that. Doing pip2 --version instead provided the expected response.

pip commands return a TypeError

I needed to install hashlib for one of my projects, and used easy_install hashlib (as was stated by this Stack Exchange post) to do so. However, right after I installed this package, I haven't been able to use pip at all. Whenever I type a command with pip (like pip install), I get the following error message:
Traceback (most recent call last):
File "/usr/local/bin/pip", line 5, in <module>
from pkg_resources import load_entry_point
File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources/__init__.py", line 37, in <module>
import email.parser
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/parser.py", line 12, in <module>
from email.feedparser import FeedParser
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/feedparser.py", line 27, in <module>
from email import message
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/message.py", line 16, in <module>
import email.charset
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/charset.py", line 13, in <module>
import email.base64mime
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/base64mime.py", line 40, in <module>
from email.utils import fix_eols
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/email/utils.py", line 27, in <module>
import random
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/random.py", line 49, in <module>
import hashlib as _hashlib
File "build/bdist.macosx-10.13-intel/egg/hashlib.py", line 115, in <module>
"""
TypeError: 'frozenset' object is not callable
This also now occurs when I'm using easy_install- same error. I'm also not able to uninstall pip and reinstall- any command with pip results in the error above. Even executing
pip --version yields the same error. I know it's something to do with the hashlib package, but I'm not sure how to delete it and reinstall without messing up my entire Python configurations. My hypothesis is that the version of hashlib I installed is either too old or too new for my Python 2.7, and doesn't play well with it.
Does anyone know what exactly is going on behind the scenes here in this stack trace, and how I can fix pip?
Is the best thing to do to literally go in and delete hashlib.py from my lib/python2.7 folder?

So, this doesn't solve the issue with hashlib, but it does get you a working pip again: literally download and reinstall Python 2.7 all over again from the source. If anyone comes up with a better, more elegant solution for fixing pip, I am all for it.
However, at least I can use pip again...

Ansible is not working on mac

I'm trying to execute ansible in my system. I guess I messed up something and unable to run ansible. When I run ansible --version, I see the following error:
krishnapatamset:bin krishna.patamsetti$ ansible --version
[WARNING]: log file at /var/log/ansible/ansible.log is not writeable and we cannot create it, aborting
ERROR! Unexpected Exception: No module named markupsafe
the full traceback was:
Traceback (most recent call last):
File "/usr/local/bin/ansible", line 80, in <module>
from ansible.cli.adhoc import AdHocCLI as mycli
File "/usr/local/lib/python2.7/site-packages/ansible/cli/adhoc.py", line 28, in <module>
from ansible.executor.task_queue_manager import TaskQueueManager
File "/usr/local/lib/python2.7/site-packages/ansible/executor/task_queue_manager.py", line 29, in <module>
from ansible.executor.play_iterator import PlayIterator
File "/usr/local/lib/python2.7/site-packages/ansible/executor/play_iterator.py", line 30, in <module>
from ansible.playbook.block import Block
File "/usr/local/lib/python2.7/site-packages/ansible/playbook/__init__.py", line 27, in <module>
from ansible.playbook.play import Play
File "/usr/local/lib/python2.7/site-packages/ansible/playbook/play.py", line 27, in <module>
from ansible.playbook.base import Base
File "/usr/local/lib/python2.7/site-packages/ansible/playbook/base.py", line 29, in <module>
from jinja2.exceptions import UndefinedError
File "/usr/local/lib/python2.7/site-packages/jinja2/__init__.py", line 33, in <module>
from jinja2.environment import Environment, Template
File "/usr/local/lib/python2.7/site-packages/jinja2/environment.py", line 13, in <module>
from jinja2 import nodes
File "/usr/local/lib/python2.7/site-packages/jinja2/nodes.py", line 19, in <module>
from jinja2.utils import Markup
File "/usr/local/lib/python2.7/site-packages/jinja2/utils.py", line 531, in <module>
from markupsafe import Markup, escape, soft_unicode
ImportError: No module named markupsafe
Can anyone please help me with this situation. Thanks in advance
I even tried pip install markupsafe but still I am unable to get ansible working

Ansible is based on python and there is a python module that's missing called markupsafe
install it via pip, you may need to escalate privileges
pip2 install -I markupsafe
pip2 for python 2.x
-I to ignore and reinstall package if it's already installed

I uninstalled pip and installed it again.
brew uninstall pip
brew doctor
brew install pip