Running Django custom manage.py task on Heroku - Importing Issues - python

I'm trying to run a custom django command as a scheduled task on Heroku. I am able to execute the custom command locally via: python manage.py send_daily_email. (note: I do NOT have any problems with the custom management command itself)
However, Heroku is giving me the following exception when trying to "Run" the task through Heroku Scheduler addon:
Traceback (most recent call last):
File "bin/send_daily_visit_email.py", line 2, in <module>
from django.conf import settings
ImportError: No module named django.conf
I placed a python script in /bin/send_daily_email.py, and it is the following:
#! /usr/bin/python
from django.conf import settings
settings.configure()
from django.core import management
management.call_command('send_daily_email') #delegates off to custom command
Within Heroku, however, I am able to run heroku run bin/python - launch the python shell - and successfully import settings from django.conf
I am pretty sure it has something to do with my PYTHON_PATH or visibility to Django's SETTINGS_MODULE, but I'm unsure how to resolve the issue. Could someone point me in the right direction? Is there an easier way to accomplish what I'm trying to do here?
Thank you so much for your tips and advice in advance! New to Heroku! :)
EDIT:
Per Nix's comment, I made some adjustments, and did discover that specifying my exact python path, I did get past the Django setup.
I now receive:
File "/app/lib/python2.7/site-packages/django/core/management/__init__.py", line 155, in call_command
raise CommandError("Unknown command: %r" % name)
django.core.management.base.CommandError: Unknown command: 'send_daily_email'
Although, I can see 'send_daily_email' when I run ``heroku run bin/python app/manage.py```.
I'll keep an update if I come across the answer.

You are probably using a different interpreter.
Check to make sure shell python is the same as the one you reference in your script /usr/bin/python . It could be that there is a different one in your path, which would explain why it works when you run python manage.py but not your shell scrip which you explicitly reference /usr/bin/python.
Typing which python will tell you what interpreter is being found on your path.

In addition, this can also be resolved by adding your home directory to your Python path. A quick and unobtrusive way to accomplish that is to add it to the PYTHONPATH environment variable (which is generally /app on the Heroku Cedar stack).
Add it via the heroku config command:
$ heroku config:add PYTHONPATH=/app
That should do it! For more details: http://tomatohater.com/2012/01/17/custom-django-management-commands-on-heroku/

Related

after formating my laptop, my Django projects doesn't work

Before I format my laptop, I cut & paste my Django project (folder that include mysite and myvenv folders) to another partition. It was built on python 2.7 and used virtual environment and Django 1.7.
Now I have python 3.4, and after activating virtual environment and run:
$ python manage.py makemigrations
I get this error:
Traceback (most recent call last):
File "manage.py", line 12, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named django.core.management
So I tried and installed python2.7 but still get the same error.
Is there a way to correct what happend to my project and if not I need your advice for best practise on how can I move Django project without break it?
In my Django project I followed the official tutorial on djangoproject.com
This is my first question on stackoverflow
I revoke my answer. I no longer want to support this site.
My guess would be that your manage.py is not configured properly. The first line must define the Python exec to run the script, and not another location like /user/bin/python. If it is set to /user/bin/python you will almost certainly have certain packages missing. Ensure that the path you specify is to your virtualenv python location.

sniffer can't find DJANGO_SETTINGS_MODULE

I'm trying to automate the test rerun after a change while developing. After searching around a little sniffer seemed fine. But if I run it my tests fail with this error:
ERROR: Failure: ImportError (Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.)
if I run them manually they pass. Do you have a clue why sniffer won't work?
Something like the following as your scent.py should work:
from subprocess import call
from sniffer.api import runnable
#runnable
def execute_tests(*args):
fn = [ 'python', 'manage.py', 'test' ]
fn += args[1:]
return call(fn) == 0
Which you can then call as sniffer -x appName.
You can get sniffer to read your settings by creating a scent.py file in the same directory as manage.py.
Here's what mine looks like:
import os
os.environ["DJANGO_SETTINGS_MODULE"] = 'myapp.settings'
Which will get you as far as sniffer reading your settings, but then you'll run into other problems — basically, sniffer just runs your tests using nose, which isn't the same thing that the manage.py test does when django-nose is installed.
Anybody know what else needs to be in scent.py for snigger to with with Django?
Trying to guess where the problem may reside: it seems you need to explicitly set the position of your settings.py file.
if you're running your test from a subprocess' call you can use the following command:
call(["django-admin.py", "test --settings=your_project.settings"])
otherwise you can set environment variables with the following command:
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'your_project.settings'
(change your_project with the name of your django project)
if you're running a command like "./manage.py tests" you can add the former lines at the beginning of manage.py (there are other ways but I need to see the code to provide a more precise solution)

python scripts issue (no module named ...) when starting in rc.local

I'm facing of a strange issue, and after a couple of hour of research I'm looking for help / explanation about the issue.
It's quite simple, I wrote a cgi server in python and I'm working with some libs including pynetlinux for instance.
When I'm starting the script from terminal with any user, it works fine, no bug, no dependency issue. But when I'm trying to start it using a script in rc.local, the following code produce an error.
import sys, cgi, pynetlinux, logging
it produce the following error :
Traceback (most recent call last):
File "/var/simkiosk/cgi-bin/load_config.py", line 3, in
import cgi, sys, json, pynetlinux, loggin
ImportError: No module named pynetlinux
Other dependencies produce similar issue.I suspect some few things like user who executing the script in rc.local (root normaly) and trying some stuff found on the web without success.
Somebody can help me ?
Thanx in advance.
Regards.
Ollie314
First of all, you need to make sure if the module you want to import is installed properly. You can check if the name of the module exists in pip list
Then, in a python shell, check what the paths are where Python is looking for modules:
import sys
sys.path
In my case, the output is:
['', '/usr/lib/python3.4', '/usr/lib/python3.4/plat-x86_64-linux-gnu', '/usr/lib/python3.4/lib-dynload', '/usr/local/lib/python3.4/dist-packages', '/usr/lib/python3/dist-packages']
Finally, append those paths to $PATH variable in /etc/rc.local. Here is an example of my rc.local:
#!/bin/sh -e
#
# rc.local
#
# This script is executed at the end of each multiuser runlevel.
# Make sure that the script will "exit 0" on success or any other
# value on error.
#
# In order to enable or disable this script just change the execution
# bits.
#
# By default this script does nothing
export PATH="$PATH:/usr/lib/python3.4:/usr/lib/python3.4/plat-x86_64-linux-gnu:/usr/lib/python3.4/lib-dynload:/usr/local/lib/python3.4/dist-packages:/usr/lib/python3/dist-packages"
# Do stuff
exit 0
The path where your modules are install is probably normally sourced by .bashrc or something similar. .bashrc doesn't get sourced when it's not an interactive shell. /etc/profile is one place that you can put system wide path changes. Depending on what Linux version/distro it may use /etc/profile.d/ in which case /etc/profile runs all the scripts in /etc/profile.d, add a new shell script there with execute permissions and a .sh extention.

Django: manage.py ImportError because environment variable is undefined

I've just cloned a Django project using Mercurial to a Windows machine that I have set up Python 2.7 on. When I try to run manage.py (with or without a command), I get the following error:
Traceback (most recent call last):
File "C:\Users\jes.000\Documents\project\manage.py", line 30, in <module>
import django.core.servers.basehttp
File "C:\Python27\lib\site-packages\django\core\servers\basehttp.py", line 26, in <module>
from django.views import static
File "C:\Python27\lib\site-packages\django\views\static.py", line 95, in <module>
template_translatable = ugettext_noop(u"Index of %(directory)s")
File "C:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 75, in gettext_noop
return _trans.gettext_noop(message)
File "C:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 48, in __getattr__
if settings.USE_I18N:
File "C:\Python27\lib\site-packages\django\utils\functional.py", line 184, in inner
self._setup()
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 40, in _setup
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." %ENVIRONMENT_VARIABLE)
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
I understand that manage.py is supposed to set that environment variable. I also understand that I might be getting this error because the project is not in the python path... but adding it manually seems like it shouldn't be necessary. Isn't manage.py supposed to resolve this issue as well?
Edit: I've just tested and found that it does the exact same thing under Linux.
Edit: Here's a useful discovery: I only get this error if I install Django via PIP, it works fine if I install the python-django package on Ubuntu. So there's something that the debian package is doing that PIP isn't. Maybe the debian package sets some environment variables? I need to figure that out, since I'm trying to develop on Windows.
Edit: I think I've found the problem, but not a solution. Working on an Ubuntu machine, when I install Django via PIP (pip install django), this project doesn't work with the error I gave. If I create a new project (django-admin startproject testproject) and try that it works, and it consists of manage.py in the created folder, with another folder that contains everything else. Like this:
+-testproject
+-manage.py
+-testproject
+-__init__.py
+-etc.
When I install the Debian package (apt-get install python-django) the project I'm trying to work on works, and if I create a test project (django-admin startproject testproject) it has a folder structure like this:
+-testproject
+-manage.py
+-__init__.py
+-etc.
Notice the difference: the PIP Django package seems to put the manage.py outside of the python module that is the app, while the Debian package puts manage.py inside the module. I assume that this is due to the two packages being of a different version of Django, in between which they changed the structure, because the two packages doing this differently doesn't make any sense.
So my problem is that the project I'm trying to develop was originally generated using the version that's been Debian packaged, while on my Windows machine I'm trying to use the version of Django from pip.
The trouble is that I'm not certain how to fix the problem. Naively moving the manage.py up a directory so it's above the project folder doesn't work, because this 'debian-package version' manage.py tries to just import settings, which fails if it's not in the same folder as settings.py. But somewhere else in Django, something seems to be expecting the project itself to be at //settings.py (as would be the case if the pip version of django-admin was used) rather than at /settings.py (as is the case if the debian-package version is used).
If I look it up, the version in the Ubuntu repositories is 1.3.1, while the version in PyPI is 1.4. Look at the release notes for 1.4, and hey:
An updated default project layout and manage.py that removes the
“magic” from prior versions. And for those who don’t like the new
layout, you can use custom project and app templates instead!
But... shouldn't projects generated with the 1.3 layout still work under 1.4..?
I'm not sure if this has changed in more recent version of django, but the default manage.py simply tries to import the settings.py file and start the manager. You would be expected to put the project in your PYTHONPATH, or to put the settings.py into your DJANGO_SETTINGS_MODULE variable, by default.
You can modify the manage.py to be a bit smarter with this snippet:
manage.py
#!/usr/bin/env python
from django.core.management import execute_manager
...
import settings
import os
project = os.path.basename(os.path.dirname(__file__))
os.environ['DJANGO_SETTINGS_MODULE'] = '%s.settings' % project
if __name__ == "__main__":
execute_manager(settings)
Now you can run manage.py from any location, since it will always explicitly set up the path to the settings.py file.
You may add the following code to the file manage.py ,like this:
from django.conf import settings
settings.configure()
This is indeed due to the difference in default layouts between 1.3 and 1.4. Normally Django 1.3 projects will work under 1.4, so something else is going on here that I can't quite figure out. I have established that the problem isn't platform-dependent.
I couldn't figure out the exact problem but I did figure out a fix. That was:
Create a new project under Django 1.4 and poach its manage.py file. After changing the project name, this file works as a manage.py. I named it 'manage14.py' so that the original manage.py would be left for my colleagues using Django 1.3.
Add the directory above the one that the project was in to the global PYTHONPATH. This is not an ideal solution, but it allowed an unmodified Django 1.4 manage.py to load the project without having to convert the project to Django 1.4's organization.
As soon a practical, I plan to convert the project to the Django 1.4 format since it seems to be better anyway, particularly from the perspective of avoiding this kind of problem. Unfortunately I'll have to wait until the Ubuntu repo's get 1.4 so that my colleagues will be up to the same version I am on Windows using PyPI.

django import error - No module named core.management

Ok, I see plenty of these errors around. I have tried everything I know to do and have yet to figure this out.
I am working on a development server running python 2.5 and Django 1.3. Django 1.3 was installed using python setup.py install after unpacking the tar.gz download.
All works well, I seldom have the need to run manage.py but am trying to use the new staticfiles app and am running into problems.
python manage.py collectstatic
Traceback (most recent call last):
File "manage.py", line 2, in <module>
from django.core.management import execute_manager
ImportError: No module named core.management
Ok, so I have PATH issue.
From Django install I double check my site-packages directory.
python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"
/usr/lib/python2.5/site-packages
Ok, let's check out what I have, echo $PYTHON_PATH was empty, so I set it
export PYTHON_PATH=/usr/lib/python2.5/site-packages/django
Still no luck. Lets check what sys.path has to say
>>> import sys
>>> print sys.path
['', '/usr/lib/python2.5', '/usr/lib/python2.5/plat-linux2', '/usr/lib/python2.5/lib-tk', '/usr/lib/python2.5/lib-dynload', '/usr/local/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages', '/usr/lib/python2.5/site-packages/PIL', '/usr/lib/python2.5/site-packages/django', '/var/lib/python-support/python2.5']
path is there, I even created /usr/lib/python2.5/site-packages/django.pth with contents
cat /usr/lib/python2.5/site-packages/django.pth
/usr/lib/python2.5/site-packages/django/
Anyone got an clues to what is going on here?
I found a symlink further up the path that was getting in the way, but no on to a new error.
python manage.py collectstatic
Traceback (most recent call last):
File "manage.py", line 14, in <module>
execute_manager(settings)
File "/usr/lib/python2.5/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/usr/lib/python2.5/site-packages/django/core/management/__init__.py", line 349, in execute
version=get_version(),
File "/usr/lib/python2.5/site-packages/django/__init__.py", line 12, in get_version
from django.utils.version import get_svn_revision
ImportError: No module named utils.version
I also tried creating a new project to see if there were any issues there and get the same utils.version error.
Side node: Unode from #django helped me a bit, set up virtualenv on same machine and got past the errors so still not sure what is up with this actual install here, but it seems to not be in the django projects but in the django/python install.
If, like me, you are running your django in a virtualenv, and getting this error, look at your manage.py.
The first line should define the python executable used to run the script.
This should be the path to your virtualenv's python, but it is something wrong like /usr/bin/python, which is not the same path and will use the global python environment (and packages will be missing).
Just change the path into the path to the python executable in your virtualenv.
You can also replace your shebang line with #!/usr/bin/env python. This should use the proper python environment and interpreter provided that you activate your virtualenv first (I assume you know how to do this).
If you are in a virtualenv you need to activate it before you can run ./manage.py 'command'
source path/to/your/virtualenv/bin/activate
if you config workon in .bash_profile or .bashrc
workon yourvirtualenvname
*please dont edit your manage.py file maybe works by isnt the correct way and could give you future errors
I had the same problem because I was installing Django as a super user, thus not in my virtualenv. You should not do sudo pip install Django
Instead, install it this way:
$ source ./bin/activate
$ pip install Django
Please, reinstall django with pip:
sudo pip install --upgrade django==1.3
(Replace 1.3 to your django version)
As known this was a path issue.
the base of my custom packages shared a name with a directory set in a /etc/profile. The packages were in a different location however for the webserver. So I removed the offending entries from my $PYTHONPATH and was good to go!
Thanks for the help.
Another possible reason for this problem is that your OS runs python3 by default.
Either you have to explicitly do: python2 manage.py
or you need to edit the shebang of manage.py, like so:
#!/usr/bin/env python2
or if you are using python3:
#!/usr/bin/env python3
You are probably using virtualenvwrapper. Don't forget to select your enviroment by running:
$ workon env_name
I had this error while trying to run an embedded system (using django of course) on a Raspberry Pi 2 (and not a VM)
Running this:
sudo pip install Django
Made the trick!
just in case a fellow using Raspbian/Jessie gets this
python3 manage.py runserver
Check version of Python
For me, my server was using Python 2.4. I simply looked up Python 2.7, which was installed on my server, and created an alias.
alias python=python2.7
If you need to know more, I found the solution here
I was getting the same problem while I trying to create a new app. If you write python manage.py startapp myapp, then it looks for usr/bin/python. But you need this "python" which is located in /bin directory of your virtual env path. I solved this by mentioning the virtualenv's python path just like this:
<env path>/bin/python manage.py startapp myapp
Solved it!!!
After searching for ages and trying all these other suggestions which didn't work, I finally found the solution for my setup.
My setup/scenario:
Windows, Python27
My django project is checked out via svn
when running python manage.py runserver in the new folder, I got the import error
python manage.py runserver used to work in the original folder (which I would commit changes from) until I deleted it
Solution
Remove any the folder named django in the same directory of manage.py
Thats right...as soon as I removed the folder "django" which only contained a __init__.py file...I could run the server again!
Have no idea why though
Try change your first line of manage.py.
Change
#!/usr/bin/python
by
#!/usr/bin/env python
For those of you using Django 1.6 or newer, note that execute_manager was removed. There is a solution posted in the second SO answer here.
Store the python python path in a variable and execute.This would include the otherwise missing packages.
python_path= `which python`
$python_path manage.py runserver
==================================SOLUTION=========================================
First goto: virtualenv
by running the command:
source bin/activate
and install django because you are getting the error related to 'import django':
pip install django
Then run:
python manage.py runserver
(Note: please change 'runserver' to the program name you want to run)
For the same issue, it worked in my case.
==================================Synopsis=========================================
ERROR:
(Development) Rakeshs-MacBook-Pro:src rakesh$ python manage.py runserver
Traceback (most recent call last):
File "manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 14, in <module>
import django
ModuleNotFoundError: No module named 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 17, in <module>
"Couldn't import Django. Are you sure it's installed and "
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
(Development) Rakeshs-MacBook-Pro:src rakesh$
(Development) Rakeshs-MacBook-Pro:src rakesh$
(Development) Rakeshs-MacBook-Pro:src rakesh$ python -Wall manage.py test
Traceback (most recent call last):
File "manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ModuleNotFoundError: No module named 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 14, in <module>
import django
ModuleNotFoundError: No module named 'django'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "manage.py", line 17, in <module>
"Couldn't import Django. Are you sure it's installed and "
ImportError: Couldn't import Django. Are you sure it's installed and available on your PYTHONPATH environment variable? Did you forget to activate a virtual environment?
AFTER INSTALLATION of django:
(Development) MacBook-Pro:src rakesh$ pip install django
Collecting django
Downloading https://files.pythonhosted.org/packages/51/1a/e0ac7886c7123a03814178d7517dc822af0fe51a72e1a6bff26153103322/Django-2.1-py3-none-any.whl (7.3MB)
100% |████████████████████████████████| 7.3MB 1.1MB/s
Collecting pytz (from django)
Downloading https://files.pythonhosted.org/packages/30/4e/27c34b62430286c6d59177a0842ed90dc789ce5d1ed740887653b898779a/pytz-2018.5-py2.py3-none-any.whl (510kB)
100% |████████████████████████████████| 512kB 4.7MB/s
Installing collected packages: pytz, django
AFTER RESOLVING:
(Development) MacBook-Pro:src rakesh$ python manage.py runserver
Performing system checks...
System check identified no issues (0 silenced).
You have 15 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
August 05, 2018 - 04:39:02
Django version 2.1, using settings 'trydjango.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
[05/Aug/2018 04:39:15] "GET / HTTP/1.1" 200 16348
[05/Aug/2018 04:39:15] "GET /static/admin/css/fonts.css HTTP/1.1" 200 423
[05/Aug/2018 04:39:15] "GET /static/admin/fonts/Roboto-Bold-webfont.woff HTTP/1.1" 200 82564
[05/Aug/2018 04:39:15] "GET /static/admin/fonts/Roboto-Light-webfont.woff HTTP/1.1" 200 81348
[05/Aug/2018 04:39:15] "GET /static/admin/fonts/Roboto-Regular-webfont.woff HTTP/1.1" 200 80304
Not Found: /favicon.ico
[05/Aug/2018 04:39:16] "GET /favicon.ico HTTP/1.1" 404 1976
Good luck!!
environment: Python 3.9.6, Django 3.2.6, VS Code
Just disable Pylance extension and reload your VS Code.
I had a similar problem. PyCharm couldn't run the server but I could run it from the command line. I tried which python and then made sure that PyCharm was same interpreter and then everything worked OK.
This error usually occurs when django is not installed.
If you have already installed django but still getting the same error, then you must be working in separate virtual environment. You need to install django in your virtual environmnent as well. When you are in shell of virtual machine simply do this:
pip install django
It is because virtual machine has separate file system, it doesn't recognize django even if it is installed on your system.
I fixed this problem by changing #PATH="$VIRTUAL_ENV/bin:$PATH" to PATH="$PATH:$VIRTUAL_ENV/bin"
For reasons not obvious to me the python executable in the virtualenv dir does not see django but the normally installed python does.
Just a single mistake
This happened with me as well. My mistake was that I've created Django project before creating virtual env, later activated the env and was trying to start the server. Just install Django after activating env it will work
your project is created using an old version of django-admin.py, older than django1.3
to fix this create another django project and copy its manage.py and paste it in the old one
Agreed completely that this is a path issue but fwiw, I had this same error. It was due to the mistake of using a relative path for my Python executable when setting up my virtual environment. I had done this:
virtualenv -p ~/python_runtimes/2.7.3/bin/python venv2.7.3 --distribute
Instead I had to give the full path to the Python executable.
HTH,
Harlin
source ~/blog-venv/bin/activate
pick your virtualenv to replace "blog-venv" here.
Be sure you're running the right instance of Python with the right directories on the path. In my case, this error resulted from running the python executable by accident - I had actually installed Django under the python2.7 framework & libraries. The same could happen as a result of virtualenv as well.
Okay so it goes like this:
You have created a virtual environment and django module belongs to that environment only.Since virtualenv isolates itself from everything else,hence you are seeing this.
go through this for further assistance:
http://www.swegler.com/becky/blog/2011/08/27/python-django-mysql-on-windows-7-part-i-getting-started/
1.You can switch to the directory where your virtual environment is stored and then run the django module.
2.Alternatively you can install django globally to your python->site-packages by either running pip or easy_install
Command using pip: pip install django
then do this:
import django
print (django.get_version()) (depending on which version of python you use.This for python 3+ series)
and then you can run this:
python manage.py runserver
and check on your web browser by typing :localhost:8000
and you should see django powered page.
Hope this helps.
I included the name of the new App to the INSTALLED_APPS list in the settings.py "before" I issued the startapp command. Once I removed the list entry, I could create the app.
I solved this problem by using 'django-admin' command as following instead:
django-admin startproject _project_name
just remove the ".py" attached to "django-admin"
Having an application called site can reproduce this issue either.
I got this due to forgetting that I installed Django using pip -U, so it was only available to the user running my Django app. To run manage.py I had to do
sudo su -s /bin/bash MY_DJANGO_USER
/PATH/TO/MY/APP/manage.py

Categories

Resources