Develop Reference

How to install downloaded django project in virtualenv

I am trying to follow the instructions from this Django/Ajax tutorial: https://realpython.com/blog/python/django-and-ajax-form-submissions/. They say:
Download the compressed pre-ajax Django Project from the repo https://github.com/realpython/django-form-fun/tree/master/part1
Activate a virtualenv
Install the requirements
Sync the database
Fire up the server
Here is what I am doing:
Create new virtualenv using virtualenv -p /usr/bin/python3 ajaxenv inside home/ajaxtutorial/ folder
Download the repo and unpack its django-form-fun/part1/pre-ajax/talk_project/ folder in the home/ajaxtutorial folder.
Put the requirements.txt provided with the downloaded repo in ajaxenv/bin
Run pip install -r requirements.txt inside ajaxenv. At this point I can see that Django gets installed.
At this point the tutorial says to "sync the database". The only way I know how to do this is to use python manage.py migrate, but this throws an error. I guess this method requires starting the project or an app first, which is not a case here, since both the app and the project are downloaded, not created by me.
I don't know how to proceed from here. python manage.py migrate throws the following error:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/core/management/__init__.py", line 399, in execute_from_command_line
utility.execute()
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/core/management/__init__.py", line 392, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/core/management/__init__.py", line 261, in fetch_command
commands = get_commands()
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/core/management/__init__.py", line 107, in get_commands
apps = settings.INSTALLED_APPS
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/conf/__init__.py", line 54, in __getattr__
self._setup(name)
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/conf/__init__.py", line 50, in _setup
self._configure_logging()
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/conf/__init__.py", line 72, in _configure_logging
from django.utils.log import DEFAULT_LOGGING
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/utils/log.py", line 7, in <module>
from django.views.debug import ExceptionReporter, get_exception_reporter_filter
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/views/debug.py", line 12, in <module>
from django.template import Template, Context, TemplateDoesNotExist
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/template/__init__.py", line 53, in <module>
from django.template.base import (ALLOWED_VARIABLE_CHARS, BLOCK_TAG_END,
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/template/base.py", line 19, in <module>
from django.utils.html import escape
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/utils/html.py", line 14, in <module>
from .html_parser import HTMLParser, HTMLParseError
File "/home/bart/ajaxtutorial/ajaxenv/lib/python3.5/site-packages/django/utils/html_parser.py", line 12, in <module>
HTMLParseError = _html_parser.HTMLParseError
AttributeError: module 'html.parser' has no attribute 'HTMLParseError'

DjangoQ run manage.py qcluster results in PermissionError: [WinError5] Access is denied

I've just recently started played around with web application development using Django and now got to Django-Q for async tasks.
I followed this tutorial up to slide 13. Unfortunately then, when I try to run python manage.py qcluster I get these error messages:
(djangoq_tut) D:\Code\Python\DjangoQ_Tutorial\djangoq_demo>python manage.py qcluster
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "D:\Code\Python\DjangoQ_Tutorial\djangoq_tut\lib\site-packages\django\core\management\__init__.py", line 350, in execute_from_command_line
utility.execute()
File "D:\Code\Python\DjangoQ_Tutorial\djangoq_tut\lib\site-packages\django\core\management\__init__.py", line 342, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "D:\Code\Python\DjangoQ_Tutorial\djangoq_tut\lib\site-packages\django\core\management\base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "D:\Code\Python\DjangoQ_Tutorial\djangoq_tut\lib\site-packages\django\core\management\base.py", line 399, in execute
output = self.handle(*args, **options)
File "D:\Code\Python\DjangoQ_Tutorial\djangoq_tut\lib\site-packages\django_q\management\commands\qcluster.py", line 22, in handle
q.start()
File "D:\Code\Python\DjangoQ_Tutorial\djangoq_tut\lib\site-packages\django_q\cluster.py", line 57, in start
self.sentinel.start()
File "c:\program files (x86)\python35-32\Lib\multiprocessing\process.py", line 105, in start
self._popen = self._Popen(self)
File "c:\program files (x86)\python35-32\Lib\multiprocessing\context.py", line 212, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "c:\program files (x86)\python35-32\Lib\multiprocessing\context.py", line 313, in _Popen
return Popen(process_obj)
File "c:\program files (x86)\python35-32\Lib\multiprocessing\popen_spawn_win32.py", line 66, in __init__
reduction.dump(process_obj, to_child)
File "c:\program files (x86)\python35-32\Lib\multiprocessing\reduction.py", line 59, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: can't pickle _thread.lock objects
(djangoq_tut) D:\Code\Python\DjangoQ_Tutorial\djangoq_demo>Traceback (most recent call last):
File "<string>", line 1, in <module>
File "c:\program files (x86)\python35-32\Lib\multiprocessing\spawn.py", line 100, in spawn_main
new_handle = steal_handle(parent_pid, pipe_handle)
File "c:\program files (x86)\python35-32\Lib\multiprocessing\reduction.py", line 86, in steal_handle
_winapi.DUPLICATE_SAME_ACCESS | _winapi.DUPLICATE_CLOSE_SOURCE)
PermissionError: [WinError 5] Access is denied
I'm using a Windows 7 machine and Python 3.5.1 (32 bit) with following requirements:
appdirs==1.4.3
arrow==0.10.0
blessed==1.14.1
Django==1.9
django-picklefield==0.3.2
django-q==0.7.18
future==0.16.0
packaging==16.8
pyparsing==2.2.0
python-dateutil==2.6.0
six==1.10.0
wcwidth==0.1.7
I tried looking up the source of the error, it looks to me as it might not be exclusively related to Django or Django-Q. Other than that I don't really understand what it means.
Also tried with Python 2.7.12, outside of a virtualenv, there I get a TypeError: can't pickle _thread.lock objects, which I don't really get either.
Any ideas how to solve this?
[Edit]
After experimenting with a DjangoQ alternative - Celery - I as able to recreate the problem:
The same error occurs if using Celery 4.0.2, but not with Celery 3.1.25
I still haven't figured out how to fix this, but decided to stick to Celery for the time being, since it kinda works...

Install the latest version with pip:
$ pip install django-q
Add django_q to INSTALLED_APPS in your projects settings.py:
INSTALLED_APPS = (
# other apps
'django_q',
)
Run Django migrations to create the database tables:
$ python manage.py migrate
Choose a message broker , configure it and install the appropriate client library.
Run Django Q cluster in order to handle tasks async:
$ python manage.py qcluster

travic-ci build failing with django project

I am trying to setup travis-ci for the first time with a django project of mine and I am getting this trace in the travis output...
0.17s$ python manage.py migrate
Traceback (most recent call last):
File "manage.py", line 13, in <module>
execute_from_command_line(sys.argv)
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/core/management/__init__.py", line 316, in execute
settings.INSTALLED_APPS
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/conf/__init__.py", line 53, in __getattr__
self._setup(name)
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/conf/__init__.py", line 41, in _setup
self._wrapped = Settings(settings_module)
File "/home/travis/virtualenv/python2.7.9/lib/python2.7/site-packages/django/conf/__init__.py", line 97, in __init__
mod = importlib.import_module(self.SETTINGS_MODULE)
File "/opt/python/2.7.9/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named py
The command "python manage.py migrate" exited with 1.
Now I can see that it is looking for a module named py instead of running the manage.py file but I don't see why that would happen, and I have found numerous resources online that have stated to run a command like that in the travi.yml file (heres one: https://gist.github.com/ndarville/3625246)
here is my travis.yml file:
script:
- python manage.py migrate
- python manage.py test
Now before I go running off and trying to get it to run by installing another module like tox or nose I would like to know if it is possible to do it this why and why it is failing and where to find the info in the docs... Thanks

Installing psycopg2 in a virtualenv in ubuntu os

I using a virtualenv and I am trying to set up postgresql. I have psycopg2 installed on my system, but it appears that my virtual env does not recognize psycopg2. Is it possible to just copy the system wide psycopg2 files into my django project directory or is this a bad approach to take?
This is the relevant part of my settings.py file in my django project.
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql_psycopg2',
'NAME': 'django_db' ,
'USER': 'myusername' ,
'PASSWORD': 'mypassword' ,
'HOST': 'localhost' ,
'PORT': '' ,
}
}
When I run python manage.py migrate I get this error:
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 354, in execute
django.setup()
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/__init__.py", line 21, in setup
apps.populate(settings.INSTALLED_APPS)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/apps/config.py", line 197, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 40, in <module>
class Permission(models.Model):
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/models/base.py", line 125, in __new__
new_class.add_to_class('_meta', Options(meta, **kwargs))
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/models/base.py", line 300, in add_to_class
value.contribute_to_class(cls, name)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/models/options.py", line 166, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/__init__.py", line 40, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/utils.py", line 242, in __getitem__
backend = load_backend(db['ENGINE'])
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/utils.py", line 108, in load_backend
return import_module('%s.base' % backend_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 27, in <module>
raise ImproperlyConfigured("Error loading psycopg2 module: %s" % e)
django.core.exceptions.ImproperlyConfigured: Error loading psycopg2 module: No module named psycopg2
When I do pip install psycopg2 in my virtual env, I get the following error:
Exception:
Traceback (most recent call last):
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/commands/install.py", line 278, in run
requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/req.py", line 1153, in prepare_files
location = req_to_install.build_location(self.build_dir, not self.is_download)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/req.py", line 218, in build_location
_make_build_dir(build_dir)
File "/uwsgi-tutorial/local/lib/python2.7/site-packages/pip/req.py", line 1527, in _make_build_dir
os.makedirs(build_dir)
File "/uwsgi-tutorial/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/uwsgi-tutorial/build'
Storing debug log for failure in /home/user/.pip/pip.log

This is an old thread but some people might experience Python.h not found type of error while pip-installing psycopg2 to virtualenv on Ubuntu based systems.
Check your virtualenv python version by running python -V inside virtualenv.
Assuming it is Python 3.8.0, try installing libpython3.8-dev to your system (not virtualenv) by:
sudo apt-get install libpython3.8-dev
Probably similar approach with other versions.
That solved it for me.

Based on the combination of your question and one of your comments, it appears that when you ran sudo pip install psycopg2, it was using the system install, as it referred to psycopg2 in /usr/lib/python2.7/dist-packages which is different from the environment from which you're trying to run Django.
Your permissions error, which is from the Django environment, seems likely to be due to not using sudo (among other things, some C compilation needs to take place, during the psycopg2 install, and so sudo is often required whereas it isn't with many other packages).
I would recommend you refer to the virtualenv install's pip binary explicitly when running the sudo command.
i.e.: sudo <virtualenv_root>/bin/pip install psycopg2

Something that worked for me (python 3.8 in virtualenv):
Inside the virtual environment type:
pip install psycopg2-binary
Best.

Django - First project - runserver is an invalid syntax

I decided to try developing on python with django, while python is new to me right now.
[I'm using Windows 7 64bit]
Then, i installed python 2.7, added C:\Python27\;C:\Python27\Scripts; to PATH variable.
Next, i installed Eclipse Classic (last version) and installed PyDev, and Django.
I created my first project on Eclipse as a Django project but couldn't run it.
I'm trying (Right click on my project)-> Run as -> PyDev: Django, and i have the following console log:
Validating models...
Traceback (most recent call last): File "C:\Users\Matan\workspace\FirstSample\manage.py", line 10, in <module>
execute_from_command_line(sys.argv) File "C:\Python27\Lib\site-packages\django\core\management\__init__.py", line 443, in execute_from_command_line
utility.execute() File "C:\Python27\Lib\site-packages\django\core\management\__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv) File "C:\Python27\Lib\site-packages\django\core\management\base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__) File "C:\Python27\Lib\site-packages\django\core\management\base.py", line 232, in execute
output = self.handle(*args, **options) File "C:\Python27\Lib\site-packages\django\core\management\commands\runserver.py", line 69, in handle
self.run(*args, **options) File "C:\Python27\Lib\site-packages\django\core\management\commands\runserver.py", line 80, in run
and the list goes on...
So i tried to run the project with the python console, using:
import Django
manage.py runserver
and i have the following syntax error:
File "<stdin>", line 1
manage.py runserver
^ SyntaxError: invalid syntax
What is the problem with running my project ? How can i solve this?
Thanks.
Edit:
After trying to run the project with the following command:
python manage.py runserver on the project directory, i receive the following traceback log: Validating models...
Unhandled exception in thread started by <bound method Command.inner_run of <dja ngo.contrib.staticfiles.management.commands.runserver.Command object at 0x000000 0002688748>> Traceback (most recent call last): File "C:\Python27\lib\site-packages\django\core\management\commands\runserver. py", line 91, in inner_run
self.validate(display_num_errors=True) File "C:\Python27\lib\site-packages\django\core\management\base.py", line 266, in validate
num_errors = get_validation_errors(s, app) File "C:\Python27\lib\site-packages\django\core\management\validation.py", lin e 23, in get_validation_errors
from django.db import models, connection File "C:\Python27\lib\site-packages\django\db\__init__.py", line 40, in <modul e>
backend = load_backend(connection.settings_dict['ENGINE']) File "C:\Python27\lib\site-packages\django\db\__init__.py", line 34, in
__geta ttr__
return getattr(connections[DEFAULT_DB_ALIAS], item) File "C:\Python27\lib\site-packages\django\db\utils.py", line 92, in
__getitem
__
backend = load_backend(db['ENGINE']) File "C:\Python27\lib\site-packages\django\db\utils.py", line 24, in load_back end
return import_module('.base', backend_name) File "C:\Python27\lib\site-packages\django\utils\importlib.py", line 35, in im port_module
__import__(name) File "C:\Python27\lib\site-packages\django\db\backends\mysql\base.py", line 16 , in <module>
raise ImproperlyConfigured("Error loading MySQLdb module: %s" % e) django.core.exceptions.ImproperlyConfigured: Error loading MySQLdb module: No mo dule named MySQLdb

Download the pre-compiled python-mysql package for Windows, install it, then run the command again.
While you are there, you might as well also install setuptools if you haven't already.

Error loading MySQLdb module: No mo dule named MySQLdb
You have configured your settings to you MySQL but you have not installed MySQL-python:
pip install MySQL-python

You must be in the project directory to run the python manage.py runserver command. You also need to (1) set the PYTHONPATH environment variable to include your project directory's parent and (2) set the DJANGO_SETTINGS_MODULE environment variable to yourproject.settings
If you've done it correctly, you should be able to go to an unrelated directory (i.e. not your project directory) and do the following import in a python shell:
c:\temp> python
...
>>> from django.conf import settings
the settings module that is imported should then be the one in your project directory...