Develop Reference

Module not found - how to install a package in a Django + poetry project?

This is such a basic question, I'm sorry. I installed django-parsley with poetry (poetry add django-parsley). It's clearly installed in my pyproject.toml file.
In my django project files, in forms.py, I have a line of code that imports a module from parsley: from parsley.decorators import parsleyfy
However, when I try to run python manage.py runserver, I get the following error:
from parsley.decorators import parsleyfy
ModuleNotFoundError: No module named 'parsley'
I also tried adding 'parsley' to my INSTALLED_APPS in settings.py. That gives me this error (which is maybe due to not adding it globally with pip install?):
...some more errors...
File "C:\Program Files\Python310\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'parsley'
What do I need to do to be able to import it in a python file in my project?

I figured it out. It's actually a VSCode issue - normally, VSCode automatically identifies the right virtual environment for the project (in this case, Poetry's default auto-created project-specific venv).
However, in this project, it didn't switch over. To fix, I ran the Python: Select Interpreter command and switched the venv over to the right project. It then recognized the site-packages folder and was able to import normally.

Pyinstaller and Python-Markdown - ImportError: no module named 'extra'

I'm running into an issue trying to get python-markdown to work in pyinstaller. I have this code to demonstrate the issue in file called test.py:
import markdown
testMarkdown = "blahdy blah blah"
print(markdown.markdown(testMarkdown))
print(markdown.markdown(testMarkdown, extensions=["extra"]))
When I run it using python3, I get as desired:
(venv) C:\Users\madgrizzle>python3 test.py
<p>blahdy blah blah</p>
<p>blahdy blah blah</p>
I run pyinstaller as follows:
(venv) C:\Users\madgrizzle>pyinstaller test.py
and run the resulting code, I get the following:
(venv) C:\Users\madgrizzle\dist\test>test
<p>blahdy blah blah</p>
Traceback (most recent call last):
File "test.py", line 5, in <module>
File "lib\site-packages\markdown\core.py", line 390, in markdown
File "lib\site-packages\markdown\core.py", line 100, in __init__
File "lib\site-packages\markdown\core.py", line 126, in registerExtensions
File "lib\site-packages\markdown\core.py", line 166, in build_extension
File "importlib\__init__.py", line 126, in import_module
File "<frozen importlib._bootstrap>", line 985, in _gcd_import
File "<frozen importlib._bootstrap>", line 968, in _find_and_load
File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
ImportError: No module named 'extra'
[14432] Failed to execute script test
I tried to rebuild using:
(venv) C:\Users\madgrizzle>pyinstaller --hidden-import="markdown.extensions.extra" test.py
but I get the same error message.
Is there something special that's needed for including markdown extensions?
Additional Information:
It appears that the 'extra' extension might be causing the problem. Per https://python-markdown.github.io/extensions/extra/, 'extra' is a compilation of multiple extensions, including fenced_code and tables. If I just try to use the tables extension by itself, pyinstaller works IF I use the full-name as follows:
markdown.markdown(testMarkdown, extensions=["markdown.extensions.tables"])
If instead of using 'markdown.extensions.tables' I use 'markdown.extensions.extra', compile using pyinstaller, and run it, it responds back with a missing "fenced_code" module. Basically, it seems I have to avoid 'extra' with pyinstaller.

Short names for extensions like extra and table are setuptools entrypoints. I expect that pyinstaller does not play nice with entrypoints. Therefore, you need to use the full importable string name for all extensions.
However, as extra uses the short names internally, that means you cannot use extra. You will need to call each of the nested extensions separately by their full importable string name:
markdown.markdown(
testMarkdown,
extensions=[
"markdown.extensions.abbr",
"markdown.extensions.attr_list",
"markdown.extensions.def_list",
"markdown.extensions.fenced_code",
"markdown.extensions.footnotes",
"markdown.extensions.tables"
]
)
The only problem with that is that you won't get any extra specific behavior. However, when version 3.2 is released, the changes here will be available and you can add markdown.extensions.md_in_html to the list of extensions. At that point, you will get all of extra without needing to include extra at all.

Package Pathing Errors in Python

So I am working on implementing a file structure to my Python and am having an error with doing imports. The file structure looks something like this:
Dirs(Folder)
╘ run.py
Vers(Folder)
╘ __init__.py
1_1(Folder)
╘ Main.py
secondary.py
__init__.py
1_2(Folder)
╘ Main.py
secondary.py
__init__.py
This is the contents of run.py
import importlib
print("This is the main module")
A = importlib.import_module(str("Vers.1_1.Main"))
A.start()
B = importlib.import_module(str("Vers.1_2.Main"))
B.start()
Each Main.py and secondary.py contain code that is the same except for the version number in the print statements, which is changed depending on the version number of the folder they are in.
Main.py
import secondary
class start():
def __init__(self):
print("This is version 1.2 main")
secondary.start()
secondary.py
class start():
def __init__(self):
print("This is version 1.1 secondary")
This is the output I get when I execute run.py
This is the main module
Traceback (most recent call last):
File "Python\Python36-32\lib\importlib\__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 978, in _gcd_import
File "<frozen importlib._bootstrap>", line 961, in _find_and_load
File "<frozen importlib._bootstrap>", line 950, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 678, in exec_module
File "<frozen importlib._bootstrap>", line 205, in _call_with_frames_removed
File "Dirs\Vers\1_1\Main.py", line 1, in <module>
import secondary
ModuleNotFoundError: No module named 'secondary'
So from my understanding, run.py is able to successfully find and attempt to import Main.py from the 1_1 Folder. However, when executing the Main.py file, it is unable to see that secondary.py is in the same directory to import it. I've tried looking for how to fix this problem, but I really don't know what my issue is. Is how I am attempting to set up packages wrong? If so what do I need to change in order to make it work properly?

Short answer: Use
from . import secondary
Longer version: It looks like you assume Python 2 relative import semantics, when this is Python 3 with absolute import semantics. Have a look e.g. here for a more detailed explanation.
If you do
import secondary
this is an absolute import, and thus is not resolved relative to the current package, but rather from sys.path.
When you execute run.py as a Python script (not as a module with -m), the directory the script resides in gets added to sys.path automatically, which is why importing Vers.1_1.Main works, but import secondary does not.

After the import your code is executing under Dirs, where there is no secondary module to import
Change
import secondary
in Main.py
to
import Vers.1_1.secondary
and to
import Vers.1_2.secondary
in the other Main.py

pyinstaller 3.2 with django 1.10.1

System: windows 7 64 bit, python 3.5, anaconda 3 (64 bit) , django 1.10.1
I'm trying to compile my django project in 2 ways:
First:
[Anaconda3] c:\compilation\Gui>pyinstaller --name=gui --exclude-module=PyQt4 --exclude-module=matplotlib --clean --win-private-assemblies manage.py
Second according to this soloution:
[Anaconda3] c:\compilation\Gui>pyinstaller --name=gui --exclude-module=PyQt4 --exclude-module=matplotlib --clean --win-private-assemblies --runtime-hook=pyi_rth_django.py manage.py
When I try to run the output:
c:\compilation\Gui\dist\gui>gui.exe runserver
I get (for the 2 versions I get the same output):
c:\compilation\Gui\dist\gui>gui.exe runserver
Unhandled exception in thread started by <function check_errors.<locals>.wrapper at 0x00000000044E9D90>
Traceback (most recent call last):
File "site-packages\django\utils\autoreload.py", line 226, in wrapper
File "site-packages\django\core\management\commands\runserver.py", line 113, in inner_run
File "site-packages\django\utils\autoreload.py", line 249, in raise_last_exception
File "site-packages\django\utils\six.py", line 685, in reraise
File "site-packages\django\utils\autoreload.py", line 226, in wrapper
File "site-packages\django\__init__.py", line 27, in setup
File "site-packages\django\apps\registry.py", line 85, in populate
File "site-packages\django\apps\config.py", line 116, in create
File "importlib\__init__.py", line 126, in import_module
File "<frozen importlib._bootstrap>", line 986, in _gcd_import
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 956, in _find_and_load_unlocked
ImportError: No module named 'django.contrib.admin.apps'
Please advice.

What is your files layout? According to these pyinstaller docs https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Executable-From-Django there could be two solutions.
run your command from parent directory, i.e. instead of
c:\compilation\Gui>pyinstaller --name=gui manage.py
do
c:\compilation>pyinstaller --name=gui Gui\manage.py
try to add import django.contrib.admin.apps to your manage.py and make sure it exists
report bug

To fix the "ImportError: No module named django.contrib.admin.apps" problem you have to create a directory, let's call it "your_project/other_hooks".
Inside that directory create a file called hook-django.contrib.py with this content:
#!/usr/bin/python
# -*- coding: utf-8 -*-
from PyInstaller.utils.hooks import collect_submodules
hiddenimports = collect_submodules('django.contrib')
Now call pyinstaller in this way:
pyinstaller --name=yourProject --additional-hooks-dir=your_project\other_hooks your_project\manage.py
While you can use any name for the directory, the file name is mandatory, it has to be "hook-django.contrib.py".
Hope this helps.
References:
https://github.com/pyinstaller/pyinstaller/wiki/Recipe-Executable-From-Django
https://github.com/pyinstaller/pyinstaller/issues/2332
https://pythonhosted.org/PyInstaller/hooks.html#how-a-hook-is-loaded

ImportError: cannot import name 'ddt'

I'm using Python3.5 in Windows with pip version 8.0.2. I installed ddt library using 'pip install ddt'. While using ddt library in code, getting import error. How to get rid of this error?
import unittest
from selenium import webdriver
from ddt import ddt, data ,unpack
import time
#ddt
class Search(unittest.TestCase):
def setUp(self):
#some code
#data(("phones",2),("music", 5))
#unpack
def test_searchproducts(self, searchterm, results):
#some code
def tearDown(self):
self.driver.quit()
if __name__ == "__main__":
unittest.main()
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.5.4\helpers\pycharm\utrunner.py", line 120, in <module>
modules = [loadSource(a[0])]
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 4.5.4\helpers\pycharm\utrunner.py", line 41, in loadSource
module = imp.load_source(moduleName, fileName)
File "C:\Program Files (x86)\Python 3.5\lib\imp.py", line 172, in load_source
module = _load(spec)
File "<frozen importlib._bootstrap>", line 693, in _load
File "<frozen importlib._bootstrap>", line 673, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 662, in exec_module
File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed
File "C:\Users\murugamx\PycharmProjects\New Project\Selenium Learning\ddt.py", line 3, in <module>
from ddt import ddt, data ,unpack
ImportError: cannot import name 'ddt'

The name of your py file is ddt. This is an error. You cannot name your file after the name of a library that you are importing.
From the Python Doc:
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
So when you use import, the first place it searches is your current directory. This is why your error occurred.