Develop Reference

"python -m idlelib.idle" not working - Windows 10

I'm running Python 3.10.2288.0 (x64) on a Windows 10 OS.
I use IDLE as interpreter.
So far, I have been running IDLE (in and out venv) by:
C:path\to\venv venv\Scripts\Activate
(venv) C:path\to\venv python -m idlelib.idle
All of a sudden, this command didn't work anymore.
The following error appeared:
Traceback (most recent call last):
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 196, in _run_module_as_main
return _run_code(code, main_globals, None,
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\runpy.py", line 86, in _run_code
exec(code, run_globals)
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\idlelib\idle.py", line 13, in <module>
from idlelib.pyshell import main # This is subject to change
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\idlelib\pyshell.py", line 53, in <module>
from idlelib import debugger
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\idlelib\debugger.py", line 7, in <module>
from idlelib import macosx
File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.2288.0_x64__qbz5n2kfra8p0\lib\idlelib\macosx.py", line 7, in <module>
from test.support import requires, ResourceDenied
ModuleNotFoundError: No module named 'test'
I tried to reinstall Python, but nothing changed.
The same error appears when running command outside the venv.
It seems that everything is up-to-date.
What do you suggest?
Thanks,
idromv

This is the result of a buggy fix to a bug that is triggered when Python is installed without the test module. The bug appeared in 3.10.8 and 3.12.0a1, released a week ago, and at least in Microsoft Store Python 3.10.2288.0.
Omitting the test module seem to not currently be an option on Mac, so this should not be a problem there.
One fix anywhere is to edit /Lib/idlelib/macosx.py: delete line 7, from test.support import requires, ResourceDenied, and replace line 23, requires('gui') with pass. This may require admin privileges.
On Windows, rerun the python.org installer, chose Modify, and check the box to install the test suite.
For the Windows store Python, including the test suite may not be an option. One can, as mentioned in the comments, instead get the python.org installer and install with the test suite.
I don't know what the situation is for *nix.

Django manage.py 1.6.11 suddenly crashes when importing settings

I have an old DJango application that used work fine but suddenly crashes. I use to test it with:
cd /opt/formshare/src/formshare/
export PYTHONPATH=$PYTHONPATH:/opt/formshare/src/formshare
export DJANGO_SETTINGS_MODULE=formshare.settings.default_settings
python manage.py validate
But now I get this error:
(formshare) bash-4.3$ python manage.py validate
Your environment is:"formshare.settings.default_settings"
Traceback (most recent call last):
File "manage.py", line 21, in <module>
execute_from_command_line(sys.argv)
File "/opt/formshare/lib/python2.7/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/opt/formshare/lib/python2.7/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/opt/formshare/lib/python2.7/site-packages/django/core/management/__init__.py", line 261, in fetch_command
commands = get_commands()
File "/opt/formshare/lib/python2.7/site-packages/django/core/management/__init__.py", line 107, in get_commands
apps = settings.INSTALLED_APPS
File "/opt/formshare/lib/python2.7/site-packages/django/conf/__init__.py", line 54, in __getattr__
self._setup(name)
File "/opt/formshare/lib/python2.7/site-packages/django/conf/__init__.py", line 49, in _setup
self._wrapped = Settings(settings_module)
File "/opt/formshare/lib/python2.7/site-packages/django/conf/__init__.py", line 132, in __init__
% (self.SETTINGS_MODULE, e)
ImportError: Could not import settings 'formshare.settings.default_settings' (Is it on sys.path? Is there an import error in the settings file?): cannot import name OrderedDict
I am using:
virtualenv=15.0.2
Python=2.7.11
amqp==2.1.1
billiard==3.3.0.23
celery==3.1.15
dict2xml==1.3
Django==1.6.11
django-braces==1.10.0
django-celery==3.1.16
django-cors-headers==0.13
django-db-readonly==0.3.2
django-digest==1.13
django-filter==0.7
django-guardian==1.2.4
django-nose==1.4.2
django-oauth-toolkit==0.7.2
django-registration-redux==1.1
django-reversion==1.8.4
django-taggit==0.12.1
django-templated-email==0.4.9
djangorestframework==2.4.3
djangorestframework-csv==1.3.3
dpath==1.2.post70
elaphe==0.5.6
FormEncode==1.3.1
gdata==2.0.18
httmock==1.2.2
httplib2==0.9
jdcal==1.3
jsonfield==0.9.23
kombu==4.0.0
librabbitmq==1.5.2
linecache2==1.0.0
lxml==3.4.0
Markdown==2.5
mock==1.0.1
modilabs-python-utils==0.1.5
nose==1.3.7
numpy==1.11.2
oauthlib==2.0.1
openpyxl==2.0.5
pandas==0.12.0
Pillow==2.5.3
poster==0.8.1
psycopg2==2.5.4
pybamboo==0.5.8.1
pylibmc==1.3.0
pymongo==2.7.2
python-dateutil==2.6.0
python-digest==1.7
-e git+https://github.com/qlands/python-json2xlsclient.git#5a39387752d819cb6387f75569dbea9a5288aa6f#egg=python_json2xlsclient
pytz==2014.7
-e git+https://github.com/XLSForm/pyxform.git#cfe8589f40319fa3279b0a83e0d23d49bcbe8408#egg=pyxform
recaptcha-client==1.0.6
requests==2.4.1
savReaderWriter==3.4.2
simplejson==2.6.2
six==1.10.0
South==1.0
traceback2==1.4.0
unicodecsv==0.9.4
unittest2==1.1.0
vine==1.1.3
xlrd==0.9.3
xlwt==0.7.5
I tried to add set the path and setting in the validate but nothing works.
Any idea what could have changed and how to correct it?

The django process doesn't appear to be seeing your formshare.settings.default_settings. Without seeing the project structure, it's hard to know exactly what's going on, but a few things you might check.
If you are using python 2.x, does the formshare/ and the settings/ directory both have an __init__.py file in them?
Try adding /opt/formshare/src/ to the python path instead of (or in addition to) what you have.
One trick that I often try to make sure the python path is correct: while I'm active in my virtualenv, I'll enter a python shell (not django shell) and try something like the following:
import formshare
from formshare import settings
from fomrmshare.settings import default_settings
The error messages that might be generated from those might also help isolate the issue.

Since you activated virtualenv, you have to do something different. Try add2virtualenv in virtualenvwrapper.For more info
http://virtualenvwrapper.readthedocs.io/en/latest/command_ref.html#add2virtualenv

How do I install PDFNet python module?

I have been running in circles trying to figure out how to get my Django app to recognise the trial version of PDFNet:
https://pypi.python.org/pypi/PDFTron-PDFNet-SDK-for-Python/5.7
I tried adding the files to my ~/usr/bin directory, I tried dropping them into my virtualenv's bin directory. Neither has worked. I have read all the documentation i can find. I am too new of a Python developer to look at this package and know how to install it and utilize it in my project.
Please help!
update I attempted to create an app folder, and list this component as an app in the app list, but when I run the code to start the application, I get the following error:
Library not loaded: #rpath/libPDFNetC.dylib
I placed all of the lib files into this folder within my project:
__init.py (empty)
_PDFNetPython.so
libPDFNetC.dylib
PDFNetPython.py
PDFNetRuby.bundle
I used the following import code at the top of the py file I was attempting to use the component on:
import site
site.addsitedir("../pdfnetc")
from pdfnetc.PDFNetPython import *
I put the lib files into a app folder named pdfnetc. once I had this, the import statements were no longer listed as unfound by pycharm.
here is the stack trace:
Traceback (most recent call last):
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 2217, in <module>
globals = debugger.run(setup['file'], None, None)
File "/Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py", line 1643, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/Users/ntregillus/myapp/manage.py", line 12, in <module>
execute_from_command_line(sys.argv)
File "/Users/ntregillus/Envs/myapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/Users/ntregillus/Envs/myapp/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
django.setup()
File "/Users/ntregillus/Envs/myapp/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/Users/ntregillus/Envs/myapp/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/Users/ntregillus/Envs/myapp/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/Users/ntregillus/myapp/statements/models.py", line 12, in <module>
from statements.managers import StatementTemplateManager, StatementManager
File "/Users/ntregillus/myapp/statements/managers.py", line 8, in <module>
from statements.utils import render_to_pdf, StatementContextBuilder
File "/Users/ntregillus/myapp/statements/utils.py", line 23, in <module>
from pdfnetc.PDFNetPython import *
File "/Users/ntregillus/myapp/pdfnetc/PDFNetPython.py", line 28, in <module>
_PDFNetPython = swig_import_helper()
File "/Users/ntregillus/myapp/pdfnetc/PDFNetPython.py", line 24, in swig_import_helper
_mod = imp.load_module('_PDFNetPython', fp, pathname, description)
ImportError: dlopen(/Users/ntregillus/myapp/pdfnetc/_PDFNetPython.so, 2): Library not loaded: #rpath/libPDFNetC.dylib
Referenced from: /Users/ntregillus/myapp/pdfnetc/_PDFNetPython.so
Reason: image not found

python -m pip install PDFNetPython3
home page link here

Below are steps to install the PDFNet module:
Download the Python and Ruby prebuilt binaries. Make sure you download the right architecture for your Python interpreter.
Extract the downloaded zip file and navigate to it.
Navigate to the /PDFNetC/Lib directory of the Python SDK download and execute:
> chmod a+x fix_rpaths.sh
./fix_rpaths.sh

I am not knowledgable of django, but take a close look at the sample py files that are in the SDK. For example, take a look at PDFNetC/Samples/AddImageTest/PYTHON folder. In there is a .py file, and look at the first 4 lines of code, they indicate what needs to load.
Please note that you need all the files that are in PDFNetC/Lib folder, including the PDFNetC.dll file.
If this does not help, please post the exact (first) error message you get.

It just works fine for me when i try to run the shell script, btw iam using the python as my programming language for pdfnet,
Try to use these commands to run the pdfn
#!/bin/sh
TEST_NAME=PDFATest
export LD_LIBRARY_PATH=../../../PDFNetC/Lib
python -u $TEST_NAME.py
The PDFATest specifies the python code, and then the export will get the required packages such as the PDFNet which is in
../../Lib
folder so make sure you use that file in the lib and then run your code and you will not have any issues with PDFNet, btw i dont answer this with respect to django but sure this will help you out.

How to debug ImportError (with sys.path being correct)

I serve django pages via CherryPy. Everything works when CherryPy is started in the foreground. When I daemonize CherryPy with
Daemonizer(cherrypy.engine).subscribe()
I get an ImportError.
sys.path is exactly the same in both cases (daemonized and non-daemonized). How can I debug this, what else than sys.path affects python imports?
Additional Information
Traceback:
[02/Sep/2014:03:08:46] ENGINE ImproperlyConfigured('Error importing module plinth.modules.first_boot.middleware: "No module named middleware"',)
Traceback (most recent call last):
File "/usr/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1353, in communicate
req.respond()
File "/usr/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 868, in respond
self.server.gateway(self).respond()
File "/usr/lib/python2.7/dist-packages/cherrypy/wsgiserver/wsgiserver2.py", line 2267, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "/usr/lib/python2.7/dist-packages/cherrypy/_cptree.py", line 299, in call
return app(environ, start_response)
File "/usr/lib/python2.7/dist-packages/django/core/handlers/wsgi.py", line 187, in call
self.load_middleware()
File "/usr/lib/python2.7/dist-packages/django/core/handlers/base.py", line 45, in load_middleware
mw_class = import_by_path(middleware_path)
File "/usr/lib/python2.7/dist-packages/django/utils/module_loading.py", line 26, in import_by_path
sys.exc_info()[2])
File "/usr/lib/python2.7/dist-packages/django/utils/module_loading.py", line 21, in import_by_path
module = import_module(module_path)
File "/usr/lib/python2.7/dist-packages/django/utils/importlib.py", line 40, in import_module
import(name)
ImproperlyConfigured: Error importing module plinth.modules.first_boot.middleware: "No module named middleware"
The file to import: /home/fbx/code/plinth/plinth/modules/first_boot/middleware.py
The relevant sys.path entry (also present when the ImportError occurs): '/home/fbx/code/plinth'
The ImportError occurs in djangos import_module function in https://github.com/django/django/blob/master/django/utils/importlib.py.
The parameter name for import_module is "plinth.modules.first_boot.middleware"
the django MIDDLEWARE_CLASSES setting is 'plinth.modules.first_boot.middleware.FirstBootMiddleware'
One more note:
I run the daemonized server with python -m plinth in the directory /home/fbx/code/plinth.
When I start the daemonized server with /usr/bin/python /home/fbx/code/plinth/plinth/__main__.py everything works! In this case, sys.path has one additional entry: /home/fbx/code/plinth/plinth. But adding this path manually on startup doesn't fix the ImportError when run as python -m plinth.
I'm running this code: https://github.com/freedombox/Plinth/tree/0b5af376102f4210395c15b2366b96a6e56fefb2
update
Thanks #cyraxjoe,os.chdir() in combination with the module missing in __init__.py was the problem. For me this behavior is unexpected and I did not find a lot of useful information/documentation, so I set up a github repo to easier demonstrate the issue: https://github.com/fonfon/ImportError-demo

This is just a theory, but it could be the reason.
Given that:
The Deamonizer plugin changes the directory to the root os.chdir('/').
The package plinth.modules.first_boot is explicitly importing first_boot and not the middleware on the __init__.py of the package.
It could be that before the Daemonizer plugins changes the directory, the module plinth.modules.first_boot is imported but without the middleware, so when django y trying to dynamically import the module it just find the module on the import cache but it can't find the middleware because the path was relative and when the Daemonizer plugin changes the directory then it becomes inaccessible.
Try importing the middleware module on the __init__ of the package.
Basically add a from . import middleware on the __init__

Django-admin.py producing errors on my shared hosting environment

I tried to install django on my shared hosting environment, which has SSH access.
I downloaded the newest version of Django using
svn export http://code.djangoproject.com/svn/django/trunk/django django
I can import Django in python shell without any error throwing.
But then I tried to start a new project using django-admin.py and it started throwing me a bunch of error, in which I tried to google for the past few hours. I still can't find it
Below is the actual error:
k4660061#server42222:~$ django-admin.py startproject klikevent
/u/k4660061/home/local/lib/python/site-packages/django/core/management/templates.py:155: Warning: 'with' will become a reserved keyword in Python 2.6
Traceback (most recent call last):
File "/u/k4660061/home/local/lib/python/site-packages/django/bin/django-admin.py", line 5, in <module>
management.execute_from_command_line()
File "/u/k4660061/home/local/lib/python/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
utility.execute()
File "/u/k4660061/home/local/lib/python/site-packages/django/core/management/__init__.py", line 381, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/u/k4660061/home/local/lib/python/site-packages/django/core/management/__init__.py", line 261, in fetch_command
klass = load_command_class(app_name, subcommand)
File "/u/k4660061/home/local/lib/python/site-packages/django/core/management/__init__.py", line 69, in load_command_class
module = import_module('%s.management.commands.%s' % (app_name, name))
File "/u/k4660061/home/local/lib/python/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/u/k4660061/home/local/lib/python/site-packages/django/core/management/commands/startproject.py", line 2, in <module>
from django.core.management.templates import TemplateCommand
File "/u/k4660061/home/local/lib/python/site-packages/django/core/management/templates.py", line 155
with open(old_path, 'r') as template_file:
^
SyntaxError: invalid syntax
k4660061#server42222:~$
Thanks
UPDATE
After a good night sleep, I got it working! See answer by Bernie. I have to see the stacktrace over and over, but I just need to add the import statement to multiple file
the file are
django/utils/archive.py
django/core/management/templates.py
and any file that throw the warning with_statement
Thanks #Bernie

If the syntax error is related to the with statement, it could be that your prod server has an older version of Python than your dev server.
If your prod server is running version 2.5 or higher you can do this:
from __future__ import with_statement
Please note: any from __future__ imports must be the very first import.
Reference: http://docs.python.org/reference/simple_stmts.html#future