Develop Reference

Jupyter packages

I am trying to import certain packages as I am working with Jupyter notebook files, and most of the packages seem to be missing, even though I have installed them. For example, when I do the command: from bs4 import BeautifulSoup or import requests
I get the error saying ModuleNotFoundError: No module named 'bs4' for the first one and a similar one for importing requests as well. I have tried pip install requests and pip install bs4, but same issue persists. I have installed them on:
"(base) aminnazemzadeh#amins-MacBook-Pro ~ % " which seems to be my home directory, and I also have anaconda3 installed alongside python3. What is the issue that I cannot import these modules.
I am using visual studio if it makes any difference
Once I add :
!pip install requests
!pip install bs4
I get:
/Users/aminnazemzadeh/.zshenv:.:1: no such file or directory: /Users/aminnazemzadeh/.cargo/env
Requirement already satisfied: requests in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (2.28.1)
Requirement already satisfied: charset-normalizer<3,>=2 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (2.0.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (1.26.11)
Requirement already satisfied: idna<4,>=2.5 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (3.3)
Requirement already satisfied: certifi>=2017.4.17 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from requests) (2022.9.24)
/Users/aminnazemzadeh/.zshenv:.:1: no such file or directory: /Users/aminnazemzadeh/.cargo/env
Requirement already satisfied: bs4 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (0.0.1)
Requirement already satisfied: beautifulsoup4 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from bs4) (4.11.1)
Requirement already satisfied: soupsieve>1.2 in /Users/aminnazemzadeh/opt/anaconda3/lib/python3.9/site-packages (from beautifulsoup4->bs4) (2.3.1)
followed by this warning:
ModuleNotFoundError Traceback (most recent call last)
Cell In[7], line 4
2 get_ipython().system('pip install bs4')
3 from urllib.request import urlopen
----> 4 from bs4 import BeautifulSoup
ModuleNotFoundError: No module named 'bs4'
Thanks

probably you're installing the packages on an environment other than the one vs code is using. you can try installing the packages directly from your jupyter notebook by running the following code in a notebook cell. the current best practice to be running installs in the notebook is using the magic commands %pip or %conda:
%pip install requests beautifulsoup4
# or
%conda install requests beautifulsoup4
this should install the packages in the same environment that the notebook is running on.
note that you may need to restart the kernel to use the affected packages.
sources:
Jupyter Discourse Forum - Location of libraries or extensions installed in JupyterLab
Jupyter Discourse Forum - Why users can install modules from pip but not from conda?
Installing Beautiful Soup
ps: thanks #wayne for the comments regarding the current best practices for installing on the current running environment.

If you're using conda, you should install via conda whenever possible. When you install via pip, conda loses some of its ability to manage dependency versions.
Try creating a new conda environment, install the needed packages via conda, then set the kernel to your new environment in vscode. Dedicate conda environments to specific projects. It is okay to have a default/generic environment for playing around but not for any significant work as you can easily create errors in your other work if a dependency changes to an incompatible version.
Conda cheet sheet for reference if you need it: https://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf
You will need the Jupyter extension in vscode if you do not already have it installed.
You will also need to install the corresponding jupyter package in your conda environment.

ERROR: Could not find a version that satisfies the requirement time (from versions: none)

I'm getting this type of error when installing the library.
Not for all packages, but only for some. Is it because pip version? or Because of python interpreter?
Plz guide me what should I do.

You don't need to install this package because it's already in the standard library.
Just import time in any Python file.

where to get and install crypto.dll on 64 bit Windows

Note: This is not an issue with Visual Studio, but rather with incompatible dll versions. The steps below replicate the problem since running in Visual Studio in debug mode breaks on exceptions being thrown. If you just run, the thrown exception is handled elsewhere and the program works fine. But since I spend a lot of time in debug mode, I would prefer to fix this problem.
When debugging, I want to be able to step into modules I have added to my Virtual Environment in Visual Studio. I get a 'library not found' error that I am not able to fix. Here are the steps:
In Visual Studio create a new Python Application.
Create a virtual environment for that application (Python 3.6 64
bit).
pip install twilio into your virtual environment. You get the
following output.
...
----- Installing 'twilio' -----
Collecting twilio
Using cached twilio-6.10.5-py2.py3-none-any.whl
Collecting pytz (from twilio)
Using cached pytz-2018.3-py2.py3-none-any.whl
Collecting six (from twilio)
Using cached six-1.11.0-py2.py3-none-any.whl
Collecting PyJWT>=1.4.2 (from twilio)
Using cached PyJWT-1.6.0-py2.py3-none-any.whl
Collecting requests>=2.0.0; python_version >= "3.0" (from twilio)
Using cached requests-2.18.4-py2.py3-none-any.whl
Collecting pysocks; python_version >= "3.0" (from twilio)
Using cached PySocks-1.6.8.tar.gz
Collecting certifi>=2017.4.17 (from requests>=2.0.0; python_version >= "3.0"->twilio)
Using cached certifi-2018.1.18-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.0.0; python_version >= "3.0"->twilio)
Using cached chardet-3.0.4-py2.py3-none-any.whl
Collecting urllib3<1.23,>=1.21.1 (from requests>=2.0.0; python_version >= "3.0"->twilio)
Using cached urllib3-1.22-py2.py3-none-any.whl
Collecting idna<2.7,>=2.5 (from requests>=2.0.0; python_version >= "3.0"->twilio)
Using cached idna-2.6-py2.py3-none-any.whl
Installing collected packages: pytz, six, PyJWT, certifi, chardet, urllib3, idna, requests, pysocks, twilio
Running setup.py install for pysocks: started
Running setup.py install for pysocks: finished with status 'done'
Successfully installed PyJWT-1.6.0 certifi-2018.1.18 chardet-3.0.4 idna-2.6 pysocks-1.6.8 pytz-2018.3 requests-2.18.4 six-1.11.0 twilio-6.10.5 urllib3-1.22
----- Successfully installed 'twilio' -----
Add the following line to the top of your .py file:
from twilio.rest import Client
In Visual Studio go to tools > options > python > debugging. Make
sure 'Enable debugging of Python standard library' is checked
Run the application. You get the following error:
ModuleNotFoundError: No module named 'OpenSSL'
pip install pyopenssl You get the following output:
...
----- Installing 'pyopenssl' -----
Collecting pyopenssl
Using cached pyOpenSSL-17.5.0-py2.py3-none-any.whl
Requirement already satisfied: six>=1.5.2 in c:\users\x\source\repos\pythonapplication9\pythonapplication9\env\lib\site-packages (from pyopenssl)
Collecting cryptography>=2.1.4 (from pyopenssl)
Using cached cryptography-2.1.4-cp36-cp36m-win_amd64.whl
Requirement already satisfied: idna>=2.1 in c:\users\x\source\repos\pythonapplication9\pythonapplication9\env\lib\site-packages (from cryptography>=2.1.4->pyopenssl)
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=2.1.4->pyopenssl)
Using cached cffi-1.11.5-cp36-cp36m-win_amd64.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=2.1.4->pyopenssl)
Using cached asn1crypto-0.24.0-py2.py3-none-any.whl
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=2.1.4->pyopenssl)
Using cached pycparser-2.18.tar.gz
Installing collected packages: pycparser, cffi, asn1crypto, cryptography, pyopenssl
Running setup.py install for pycparser: started
Running setup.py install for pycparser: finished with status 'done'
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.1.4 pycparser-2.18 pyopenssl-17.5.0
----- Successfully installed 'pyopenssl' -----
Run the application. You get the following error:
asn1crypto._ffi.LibraryNotFoundError: The library libcrypto could not be found
The error is thrown in the file named _big_num_ctypes.py in asn1crypto. The code line where this is thrown is:
libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')
if not libcrypto_path:
raise LibraryNotFoundError('The library libcrypto could not be found')
Update: I was asked to provide the full backtrace. I modified the code in this way to print it:
import unittest
import traceback
class Test_test1(unittest.TestCase):
def test_A(self):
try:
from twilio.rest import Client
except Exception as e:
print('foo')
foo = traceback.extract_stack()
traceback.print_exc(e)
if __name__ == '__main__':
unittest.main()
As before the import line throws the exception but the exception is not caught and the lines in 'except' clause are never executed
from twilio.rest import Client
update 2: I somehow had gotten this to work following #Prateek and #user8212173. But now it is not working again. As both suggested, the problem is that crypto.dll is not there. So I went thru the steps below to add it with no success:
I installed Win64 OpenSSL v1.1.0j from https://slproweb.com/products/Win32OpenSSL.html (pointed to from https://wiki.openssl.org/index.php/Binaries). It does not contain crypto.dll.
I then installed crypto.dll from http://www.dlldownloader.com/crypto-dll/ (as #user8212173 suggested) (there is only a 32 bit version) and followed the instructions. I then got a new error message "ImportError: DLL load failed: %1 is not a valid Win32 application" which means that the crypto.dll I installed has a version conflict (I am running 64bit python on a 64bit computer). I remember installing it from Unofficial Windows Binaries for Python Extension Packages I can't find it there. So where do I get a working 64bit version of crypto.dll?

I searched a lot and could find that you are missing crypto.dll file. Your code is looking for this dll file and it is unable to find it.
Please note this wont be installed by pip install crypto as this is python library and the code is looking for a dll file.
ctypes.util.find_library searches for dll file from windows environment path variable.
Reference : find_library() in ctypes
To verify I checked.
find_library('l2gpstore')
>>'C:\\WINDOWS\\system32\\l2gpstore.dll'
find_library('java')
>>'C:\\Program Files\\Java\\jdk-9.0.4\\bin\\java.dll'
Furthermore you should install OpenSSL with libcrypto module from here
OpenSSL
OpenSSL installation instructions
The master sources are maintained in our git repository, which is
accessible over the network and cloned on GitHub, at
https://github.com/openssl/openssl. Bugs and pull patches (issues and
pull requests) should be file on the GitHub repo. Please familiarize
yourself with the license.
libcrypto with respect to OpenSSL
reference : GitHub
libcrypto (with platform specific naming):
Provides general cryptographic and X.509 support needed by SSL/TLS but
not logically part of it.
Once you install binaries and check crypto.dll is available in one of the path strings in your environment variables this issue should be resolved.
If not add it into path variable and check.
Update:
Update since the question has been updated and the issue has recurred.
There are architectural changes with OpenSSL 1.1.0 as compared to 1.0.2
September 13, 2018 - OpenSSL 1.1.0 and later are quite different from previous releases. Users should install BOTH the 1.0.2 series (LTS) and the 1.1.1 (LTS) series for maximum application compatibility. Developers need to recompile their software to support 1.1.1. See the official OpenSSL release strategy document for more details. – Prateek yesterday
If you open 1.0.2 from Github you can see crypto.h file , the same file is missing in latest version. Also OpenSSL there is change in DLL names , they renamed libeay32 to libcrypto
You need to post code which makes use of asn1crypto library in the post. There is no code that explicitly uses asn1crypto in your post. So, not able to reproduce your issue using pipenv.
Make sure you are using updated libraries too.
I would not recommend downloading DLL source from unreliable source like DLLdownloader
If you are having issues with latest version of OpenSSL and asn1crypto its better to downgrade OpenSSL to 1.0.2 ,I think that would work considering it ships with crypto.h file.
Good luck!

I tried to reproduce the error on my computer and was successful when I ran the "error-producing" file _big_num_ctypes.py. Although, I do not have Visual Studio, the error stems from the missing crypto.dll file. We will deduce this step-by-step. Let's first examine the error causing code snippet in the file _big_num_ctypes.py:
#imports
from ctypes.util import find_library
.
.
from .._ffi import LibraryNotFoundError, FFIEngineError
try:
# On Python 2, the unicode string here may raise a UnicodeDecodeError as it
# tries to join a bytestring path to the unicode name "crypto"
libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')
if not libcrypto_path:
raise LibraryNotFoundError('The library libcrypto could not be found')
.
.
except (AttributeError):
raise FFIEngineError('Error initializing ctypes')
I ran the file:
C:\>cd "C:\ProgramData\Anaconda3\Lib\site-packages\asn1crypto\_perf"
C:\ProgramData\Anaconda3\Lib\site-packages\asn1crypto\_perf>python _big_num_ctypes.py
and had a Traceback for the library import:
Traceback (most recent call last):
File "_big_num_ctypes.py", line 27, in <module>
from .._ffi import LibraryNotFoundError, FFIEngineError
ValueError: attempted relative import beyond top-level package
So, I changed the import path for.ffito:
from asn1crypto._ffi import LibraryNotFoundError, FFIEngineError
On the second run, the missing libcrypto library error appeared:
asn1crypto._ffi.LibraryNotFoundError: The library libcrypto could not be found
The exception is raised when the dll library named crypto could not be found at C:\Windows\System32 and/or SYSWOW64(for 64-bit)
libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')
The purpose of find_library is to find a specified library and return a pathname. The behavior of this method varies with OS as described in the docs. If this method cannot find any packages, it returns None.
>>> from ctypes.util import find_library
>>> print(find_library("c"))
None
In our case, the search is for crypto.dll and I couldn't find this file on my computer. So, I downloaded and installed it exactly according to the instructions here. When I checked again:
>>> find_library('crypto')
'C:\\windows\\system32\\crypto.dll'
Now I ran python _big_num_ctypes.py again and got a different Traceback:
Traceback (most recent call last):
File "_big_num_ctypes.py", line 37, in <module>
libcrypto = CDLL(libcrypto_path)
File "C:\ProgramData\Anaconda3\lib\ctypes\__init__.py", line 348, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 193] %1 is not a valid Win32 application
A further investigation into the above error revealed that if I'm using a 32bit DLL with 64bit Python, or vice-versa, then I may get such errors as explained here. So, I installed Python 3.6 32-bit and tried again with py -3.6-32 _big_num_ctypes.py. I also installed all the required packages, but this error persisted.
Could it be possible that we may require 32-bit binaries for the Crypto package? This answer and this give more information.
I realized that Pycryptodome is a regularly maintained package and is preferred over the old Crypto package but can still be installed under Crypto. Another point to notice is that one of the requirements for this package is MS Visual Studio 2015 (Community Edition) and the C/C++ compilers and the redistributable only. It could be possible that some C++ compiler files or MS Visual Studio files are missing at present and causing these issues to happen.
If you install all the above prerequisites, the crypto.dll file and the Pycryptodomepackage, I believe this error will be Resolved. You have already installed other required packages OpenSSL & Twilio. Unfortunately, I am restricted to install MS Visual Studio on my computer and so I couldn't test this further.
I also ran the unittest code and it ran successfully for me:
#Output
.
----------------------------------------------------------------------
Ran 1 test in 0.771s
OK

Here is my solution for windows 10, visual studio community 2017,
open file C:\Program Files (x86)\Microsoft Visual Studio\Shared\Python36_64\Lib\site-packages\asn1crypto_perf_big_num_ctypes.py and then change the code from :
libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'crypto')
to:
libcrypto_path = find_library(b'crypto' if sys.version_info < (3,) else 'libcrypto')
then the error msg is gone.
The libcrypto.dll is already under folder C:\Windows\System32\

PyInstaller won't install, Python 3.6.0a4 and x64 Windows

I have said Python version (from https://www.python.org/downloads/windows/), and x64 Windows 10.
Every time I try to execute "pip install pyinstaller" it crashes with an error:
C:\WINDOWS\system32>pip install pyinstaller
Collecting pyinstaller
Using cached PyInstaller-3.2.tar.gz
Requirement already satisfied (use --upgrade to upgrade): setuptools in c:\users\jskurski\appdata\local\programs\python\python36\lib\site-packages (from pyinstaller)
Collecting pefile (from pyinstaller)
Using cached pefile-2016.3.28.tar.gz
Collecting pypiwin32 (from pyinstaller)
Using cached pypiwin32-219.zip
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\jskurski\AppData\Local\Temp\pip-build-y9lsbd5f\pypiwin32\setup.py", line 121
print "Building pywin32", pywin32_version
^
SyntaxError: Missing parentheses in call to 'print'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\jskurski\AppData\Local\Temp\pip-build-y9lsbd5f\pypiwin32\
So, for me it seems there is a version msmatch or something. Unofortunately, I can not figure it out myself.
Any suggestions?
Has anybody sucessfully used PyInstaller with latest 3.6 Python on Windows? Or maybe I should downgrade Python to older version?
edit: tested on another PC (same enviroment) and it was the same.
edit2: seems to work on 3.5.2 version, so it's probably a way to go, for now.

pyinstaller needs pypiwin32 module.
when pip tries to install it, it shows an error because there is no pypiwin32 for python3.6

Case is closed for me, as I downgraded to stable 3.5.2. Probably some inconsistency in that alpha release, which caused this. I just wanted to write a simple GUI Windows program, so I will not investigate further.

You have to install manually pywin32 according to your version of python. The following link you can download.
https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/
Once installed pywin32 with your right version of python. Pyinstaller must be installed

As cdarke pointed out, you are running python 2 code on Python 3.
Try this instead:
pip3 install pyinstaller

Getting Started with Python-Twitch 1.2

I've been programming with Python for a few months now but I'm having trouble figuring out the Python-Twitch library for the Twitch API. I've installed version 1.2 and it gives instructions here: http://ingwinlu.github.io/python-twitch/quickstart.html#dependencies
However, I'm completely inexperienced with doing this type of thing. Even just using their example of
from twitch.api import v3
returns a "no module named twitch" error..
Where would I get started with this? Any help would be greatly appreciated, sorry for my lack of knowledge.

If calling from twitch.api import v3 returns the following error:
no module named twitch
Then you do not have the python-twitch module properly installed.
First make sure you have an up to date version of Python, such as 3.5.1 from https://www.python.org/downloads/
Then install it (if not already installed) using the recommended options including PIP
Then go to the Python\Scripts directory, such as:
C:\Users\me\AppData\Local\Programs\Python\Python35-32\Scripts>
Then run the following:
pip install python-twitch
Now you should see something like this:
Collecting python-twitch
Downloading python-twitch-1.2.0.tar.gz
Collecting six>=1.9.0 (from python-twitch)
Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six, python-twitch
Running setup.py install for python-twitch
Successfully installed python-twitch-1.2.0 six-1.10.0\
Now re-run your test python file:
from twitch.api import v3