Serving django app with apache2+mod_wsgi - python

I have configured a linode server with apache2 and mod_wsgi..
the server is running
wsgi is running
postgre is running and syncdb was successful
what I am having trouble finishing is actually serving the app..
my file structure:
.
├── logfile
└── srv
├── logfile
└── www
└── quickerhub.com
├── admin
│   ├── css
│   │   ├── base.css
│   │   ├── changelists.css
│   │   ├── dashboard.css
│   │   ├── forms.css
│   │   ├── ie.css
│   │   ├── login.css
│   │   ├── rtl.css
│   │   └── widgets.css
│   ├── img
│   │   ├── changelist-bg.gif
│   │   ├── changelist-bg_rtl.gif
│   │   ├── chooser-bg.gif
│   │   ├── chooser_stacked-bg.gif
│   │   ├── default-bg.gif
│   │   ├── default-bg-reverse.gif
│   │   ├── deleted-overlay.gif
│   │   ├── gis
│   │   │   ├── move_vertex_off.png
│   │   │   └── move_vertex_on.png
│   │   ├── icon_addlink.gif
│   │   ├── icon_alert.gif
│   │   ├── icon_calendar.gif
│   │   ├── icon_changelink.gif
│   │   ├── icon_clock.gif
│   │   ├── icon_deletelink.gif
│   │   ├── icon_error.gif
│   │   ├── icon-no.gif
│   │   ├── icon_searchbox.png
│   │   ├── icon_success.gif
│   │   ├── icon-unknown.gif
│   │   ├── icon-yes.gif
│   │   ├── inline-delete-8bit.png
│   │   ├── inline-delete.png
│   │   ├── inline-restore-8bit.png
│   │   ├── inline-restore.png
│   │   ├── inline-splitter-bg.gif
│   │   ├── nav-bg.gif
│   │   ├── nav-bg-grabber.gif
│   │   ├── nav-bg-reverse.gif
│   │   ├── nav-bg-selected.gif
│   │   ├── selector-icons.gif
│   │   ├── selector-search.gif
│   │   ├── sorting-icons.gif
│   │   ├── tool-left.gif
│   │   ├── tool-left_over.gif
│   │   ├── tool-right.gif
│   │   ├── tool-right_over.gif
│   │   ├── tooltag-add.gif
│   │   ├── tooltag-add_over.gif
│   │   ├── tooltag-arrowright.gif
│   │   └── tooltag-arrowright_over.gif
│   └── js
│   ├── actions.js
│   ├── actions.min.js
│   ├── admin
│   │   ├── DateTimeShortcuts.js
│   │   ├── ordering.js
│   │   └── RelatedObjectLookups.js
│   ├── calendar.js
│   ├── collapse.js
│   ├── collapse.min.js
│   ├── core.js
│   ├── getElementsBySelector.js
│   ├── inlines.js
│   ├── inlines.min.js
│   ├── jquery.init.js
│   ├── jquery.js
│   ├── jquery.min.js
│   ├── LICENSE-JQUERY.txt
│   ├── prepopulate.js
│   ├── prepopulate.min.js
│   ├── SelectBox.js
│   ├── SelectFilter2.js
│   ├── timeparse.js
│   └── urlify.js
├── interest
│   ├── django.wsgi
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── settings.py
│   ├── settings.pyc
│   ├── urls.py
│   └── wsgi.py
├── js
│   └── jquery-1.10.1.min.js
├── logfile
├── manage.py
├── README
├── reoccurring
│   ├── admin.py
│   ├── forms.py
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── usagelib.py
│   └── views.py
├── schedule
│   ├── admin.py
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── models.py
│   ├── models.pyc
│   ├── tests.py
│   ├── usagelib.py
│   └── views.py
├── src
│   ├── facebooksdk
│   │   ├── examples
│   │   │   ├── appengine
│   │   │   │   ├── app.yaml
│   │   │   │   ├── example.html
│   │   │   │   └── example.py
│   │   │   ├── newsfeed
│   │   │   │   ├── app.yaml
│   │   │   │   ├── facebookclient.py
│   │   │   │   ├── static
│   │   │   │   │   ├── base.css
│   │   │   │   │   ├── favicon.ico
│   │   │   │   │   └── robots.txt
│   │   │   │   └── templates
│   │   │   │   ├── base.html
│   │   │   │   ├── home.html
│   │   │   │   └── index.html
│   │   │   ├── oauth
│   │   │   │   ├── app.yaml
│   │   │   │   ├── facebookoauth.py
│   │   │   │   └── oauth.html
│   │   │   └── tornado
│   │   │   ├── example.html
│   │   │   ├── example.py
│   │   │   └── schema.sql
│   │   ├── facebook.py
│   │   ├── facebook_sdk.egg-info
│   │   │   ├── dependency_links.txt
│   │   │   ├── PKG-INFO
│   │   │   ├── SOURCES.txt
│   │   │   └── top_level.txt
│   │   ├── MANIFEST.in
│   │   ├── README.rst
│   │   └── setup.py
│   └── pip-delete-this-directory.txt
├── static
│   └── js
│   └── jquery-1.10.1.min.js
├── templates
│   ├── 404.html
│   ├── 500.html
│   ├── Base.html
│   ├── Home.html
│   ├── Reoccurring.html
│   └── Usersettings.html
└── usersetting
├── admin.py
├── __init__.py
├── __init__.pyc
├── models.py
├── models.pyc
└── views.py
my django.wsgi:
import os
import sys
sys.path.append('/srv/www/quickerhub.com/')
os.environ['PYTHON_EGG_CACHE'] = '/srv/www/quickerhub.com.python-egg'
os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'
import django.core.handlers.wsgi
application = django.core.handlers.wsgi.WSGIHandler()
my httpd.conf:
my sites-enabled/quickerhub.com:
WSGIPythonPath /srv/www/quickerhub.com
<VirtualHost *:80>
ServerName quickerhub.com
Alias /static/ /srv/www/quickerhub.com/interest/static/
WSGIScriptAlias / /srv/www/quickerhub.com/interest/django.wsgi
<Directory />
AllowOverride None
Options -Indexes
</Directory>
</VirtualHost>
I feel like everything is pointing to the correct stuff...
EDIT:
Now just getting a 404 file not found
Please help!
Thanks!

I just setup my site www.noobniche.com on Linode using WSGI. Unfortunately I'm not at home to compare my setup, if you're still having problems tonight I can look into it. From memory in my case, I had to enable by adding to sites-enabled and sites-available.
the structure should look something similar to this;
webapps(root directory)
nichesite
static
django_project_name
--> myproject.wsgi
--> settings.py
--> urls.py
--> views.py
manage.py
it seems your .wsgi file is located in your root directory, when it should be located within your django project folder.

So it looks as if you have conflicting settings for WSGIScriptAlias - one in httpd.conf, pointing to the right path for your wsgi file, and one in your sites-enabled file, pointing to the wrong path (/var/ instead of /srv/).
You should remove the one in httpd.conf and move it to sites-enabled/quickerhub.com so that the version there is correctly pointing to /srv/.

Related

Gunicorn giving me Module Not Found Error

I'm attempting to build a Django blog.
I have a couple of apps in there, but for some reason Gunicorn is giving me trouble with the error - ModuleNotFoundError: No module named 'blog.wsgi'
├── Procfile
├── blog
│   ├── blog
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   │   ├── __init__.cpython-36.pyc
│   │   │   ├── settings.cpython-36.pyc
│   │   │   ├── urls.cpython-36.pyc
│   │   │   └── wsgi.cpython-36.pyc
│   │   ├── settings.py
│   │   ├── urls.py
│   │   └── wsgi.py
│   ├── db.sqlite3
│   ├── manage.py
│   ├── media
│   │   ├── 1820-3-large.jpg
│   │   ├── 1820-3-large_XGHcfcZ.jpg
│   │   ├── 1820-3-large_ZTmLkYt.jpg
│   │   ├── 1820-3-large_dPbPsPW.jpg
│   │   └── paul-morris-144777.jpg
│   ├── posts
│   │   ├── __init__.py
│   │   ├── __pycache__
│   │   │   ├── __init__.cpython-36.pyc
│   │   │   ├── admin.cpython-36.pyc
│   │   │   ├── models.cpython-36.pyc
│   │   │   └── views.cpython-36.pyc
│   │   ├── admin.py
│   │   ├── apps.py
│   │   ├── migrations
│   │   │   ├── 0001_initial.py
│   │   │   ├── 0002_auto_20171217_0000.py
│   │   │   ├── __init__.py
│   │   │   └── __pycache__
│   │   │   ├── 0001_initial.cpython-36.pyc
│   │   │   ├── 0002_auto_20171217_0000.cpython-36.pyc
│   │   │   └── __init__.cpython-36.pyc
│   │   ├── models.py
│   │   ├── static
│   │   │   └── posts
│   │   │   ├── css
│   │   │   ├── img
│   │   │   │   └── home.jpg
│   │   │   └── js
│   │   ├── templates
│   │   │   └── posts
│   │   │   ├── home.html
│   │   │   └── post_details.html
│   │   ├── tests.py
│   │   └── views.py
│   └── sitepages
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── __init__.cpython-36.pyc
│   │   ├── admin.cpython-36.pyc
│   │   ├── models.cpython-36.pyc
│   │   └── views.cpython-36.pyc
│   ├── admin.py
│   ├── apps.py
│   ├── migrations
│   │   ├── __init__.py
│   │   └── __pycache__
│   │   └── __init__.cpython-36.pyc
│   ├── models.py
│   ├── static
│   │   └── sitepages
│   │   ├── css
│   │   │   └── about.css
│   │   ├── img
│   │   └── js
│   ├── templates
│   │   └── sitepages
│   │   └── about.html
│   ├── tests.py
│   └── views.py
├── requirements.txt
└── venv
`
This is the bulk of my file tree as I see it. I don't see any issues, but I'm definitely overlooking something. Let me know if you'd like to see any other pieces of code.
My Procfile says -
web: gunicorn blog.wsgi --log-file -
This is my wsgi.py file -
This is wsgi.py -
import os
from django.core.wsgi import get_wsgi_application
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "blog.settings")
application = get_wsgi_application()

How set setting environment with DJANGO_SETTINGS_MODULE

I am new in Python \ Django and I have some question about settings environment. I use git for edit my project and on server I have production project. But when I edit project on local machine I have my own settings(DATABASE) so if I pull edited project on server I need to change my settings, so I want to set DJANGO_SETTINGS_MODULE, but I don't know what path I must set and where I must allocate my default setting.
This is my hierarchy:
wow
├── core
│   ├── admin.py
│   ├── api_views.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── 0001_initial.py
│   │   ├── __init__.py
│   │   └── __pycache__
│   ├── models.py
│   ├── __pycache__
│   │   ├── admin.cpython-35.pyc
│   │   ├── api_views.cpython-35.pyc
│   │   ├── __init__.cpython-35.pyc
│   │   ├── models.cpython-35.pyc
│   │   ├── sitemap.cpython-35.pyc
│   │   ├── urls.cpython-35.pyc
│   │   └── views.cpython-35.pyc
│   ├── sitemap.py
│   ├── tests.py
│   ├── urls.py
│   └── views.py
├── manage.py
├── media
│   └── core
│   ├── iUcZCekdW68_FrqsEHv.jpg
│   ├── iUcZCekdW68.jpg
│   ├── qWI5I5NuIeg.jpg
│   ├── server1.png
│   └── server2.png
├── not_found
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   │   ├── __init__.py
│   │   └── __pycache__
│   ├── models.py
│   ├── __pycache__
│   │   ├── admin.cpython-35.pyc
│   │   ├── __init__.cpython-35.pyc
│   │   ├── models.cpython-35.pyc
│   │   ├── urls.cpython-35.pyc
│   │   └── views.cpython-35.pyc
│   ├── tests.py
│   ├── urls.py
│   └── views.py
├── requirements.txt
├── settings.py
├── static
│   ├── admin
│   │   ├── css
│   │   ├── fonts
│   │   ├── img
│   │   └── js
│   └── core
│   ├── css
│   ├── img
│   └── js
├── staticfiles
│   ├── admin
│   │   ├── css
│   │   ├── fonts
│   │   ├── img
│   │   └── js
│   └── core
│   ├── css
│   ├── img
│   └── js
├── templates
│   ├── 404.html
│   ├── admin
│   │   ├── base.html
│   │   ├── base_site.html
│   │   └── index.html
│   ├── character.html
│   ├── characters.html
│   ├── checkout.html
│   ├── index.html
│   └── server.html
├── url.xml
└── wow
├── gunicorn.conf.py
├── __init__.py
├── __pycache__
│   ├── __init__.cpython-35.pyc
│   ├── settings.cpython-35.pyc
│   ├── urls.cpython-35.pyc
│   └── wsgi.cpython-35.pyc
├── settings.py
├── urls.py
├── views.py
└── wsgi.py
What i must do, end where i must write os.environ["DJANGO_SETTINGS_MODULE"] = wow.settings, or something like this?

Including markdown converter dependency in python AWS Lambda function

I am building a package to upload as an AWS Lambda function. In trying to include a dependency to Markdown 2.6.8, a python implementation of Markdown. When trying to test the function, I get the error Unable to import module 'markdown-convert': No module named markdown. I see from other similar questions that I may need to compile the dependency files inside Amazon Linux to make them usable with Lambda. I am not familiar with python and wonder if I should also be exploring the way that this code is packaged within an egg, and whether that influences how the dependency is imported. What would you suggest I pursue to try to resolve the error?
Here is the file structure of my package:
.
├── markdown
│   ├── blockparser.py
│   ├── blockprocessors.py
│   ├── extensions
│   │   ├── abbr.py
│   │   ├── admonition.py
│   │   ├── attr_list.py
│   │   ├── codehilite.py
│   │   ├── def_list.py
│   │   ├── extra.py
│   │   ├── fenced_code.py
│   │   ├── footnotes.py
│   │   ├── headerid.py
│   │   ├── __init__.py
│   │   ├── meta.py
│   │   ├── nl2br.py
│   │   ├── __pycache__
│   │   │   ├── abbr.cpython-34.pyc
│   │   │   ├── admonition.cpython-34.pyc
│   │   │   ├── attr_list.cpython-34.pyc
│   │   │   ├── codehilite.cpython-34.pyc
│   │   │   ├── def_list.cpython-34.pyc
│   │   │   ├── extra.cpython-34.pyc
│   │   │   ├── fenced_code.cpython-34.pyc
│   │   │   ├── footnotes.cpython-34.pyc
│   │   │   ├── headerid.cpython-34.pyc
│   │   │   ├── __init__.cpython-34.pyc
│   │   │   ├── meta.cpython-34.pyc
│   │   │   ├── nl2br.cpython-34.pyc
│   │   │   ├── sane_lists.cpython-34.pyc
│   │   │   ├── smart_strong.cpython-34.pyc
│   │   │   ├── smarty.cpython-34.pyc
│   │   │   ├── tables.cpython-34.pyc
│   │   │   ├── toc.cpython-34.pyc
│   │   │   └── wikilinks.cpython-34.pyc
│   │   ├── sane_lists.py
│   │   ├── smart_strong.py
│   │   ├── smarty.py
│   │   ├── tables.py
│   │   ├── toc.py
│   │   └── wikilinks.py
│   ├── __init__.py
│   ├── inlinepatterns.py
│   ├── __main__.py
│   ├── odict.py
│   ├── postprocessors.py
│   ├── preprocessors.py
│   ├── __pycache__
│   │   ├── blockparser.cpython-34.pyc
│   │   ├── blockprocessors.cpython-34.pyc
│   │   ├── __init__.cpython-34.pyc
│   │   ├── inlinepatterns.cpython-34.pyc
│   │   ├── __main__.cpython-34.pyc
│   │   ├── odict.cpython-34.pyc
│   │   ├── postprocessors.cpython-34.pyc
│   │   ├── preprocessors.cpython-34.pyc
│   │   ├── serializers.cpython-34.pyc
│   │   ├── treeprocessors.cpython-34.pyc
│   │   ├── util.cpython-34.pyc
│   │   └── __version__.cpython-34.pyc
│   ├── serializers.py
│   ├── treeprocessors.py
│   ├── util.py
│   └── __version__.py
├── Markdown-2.6.8.egg-info
│   ├── dependency_links.txt
│   ├── installed-files.txt
│   ├── PKG-INFO
│   ├── SOURCES.txt
│   └── top_level.txt
└── markdown-convert.py
The contents of markdown-convert.py:
import markdown
def lambda_handler(event, context):
parsedHTML = {}
parsedHTML[u'text'] = markdown.markdown(event[u'text'])
return parsedHTML
Best advice here is to use virtualenv on your local development machine and install packages with pip.
Then when you are ready, make the lambda package by recursively copying the entire content of <your-virtual-env>/lib/python2.7/site-packages/*. Make sure the content is placed in the root of your zip package. This is very important!
Please, find more details here:
http://docs.aws.amazon.com/lambda/latest/dg/lambda-python-how-to-create-deployment-package.html

Django ImportError manage.py while deployment

I'm trying to deploy django app developed locally following the this guide, having server on debian not fedora.
After installing requirements.txt and setting up database when i try to execute second sanity check (or anything connected with manage.py) the following error occurs:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 327, in execute
django.setup()
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/apps/registry.py", line 115, in populate
app_config.ready()
File "/opt/anpene/anp/local/lib/python2.7/site-packages/debug_toolbar/apps.py", line 15, in ready
dt_settings.patch_all()
File "/opt/anpene/anp/local/lib/python2.7/site-packages/debug_toolbar/settings.py", line 243, in patch_all
patch_root_urlconf()
File "/opt/anpene/anp/local/lib/python2.7/site-packages/debug_toolbar/settings.py", line 231, in patch_root_urlconf
reverse('djdt:render_panel')
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 568, in reverse
app_list = resolver.app_dict[ns]
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 360, in app_dict
self._populate()
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 293, in _populate
for pattern in reversed(self.url_patterns):
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 417, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 410, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/opt/anpene/anpene/config/urls.py", line 26, in <module>
url(r'^contact/', include('contact_form.urls')),
File "/opt/anpene/anp/local/lib/python2.7/site-packages/django/conf/urls/__init__.py", line 52, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named contact_form.urls
project tree structure (mostly overwritten template from cookie-cutter-django):
anpene
│   ├── contrib
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   └── sites
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── media
│   │   └── uploads
│   ├── static
│   │   ├── audio
│   │   ├── css
│   │   ├── fonts
│   │   ├── images
│   │   ├── js
│   │   └── sass
│   ├── taskapp
│   │   ├── celery.py
│   │   └── __init__.py
│   ├── templates
│   │   ├── 404.html
│   │   ├── 500.html
│   │   ├── account
│   │   ├── base.html
│   │   ├── contact_form
│   │   ├── pages
│   │   ├── question.html
│   │   ├── quiz
│   │   ├── result.html
│   │   ├── result_testing.html
│   │   ├── users
│   │   └── zinnia
│   └── users
│   ├── adapters.py
│   ├── adapters.pyc
│   ├── admin.py
│   ├── admin.pyc
│   ├── apps.py
│   ├── apps.pyc
│   ├── context_processors.py
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── migrations
│   ├── models.py
│   ├── models.pyc
│   ├── templatetags
│   ├── tests
│   ├── urls.py
│   ├── urls.pyc
│   ├── views.py
│   └── views.pyc
├── compose
│   ├── django
│   │   ├── Dockerfile
│   │   ├── Dockerfile-dev
│   │   ├── entrypoint.sh
│   │   └── gunicorn.sh
│   ├── nginx
│   │   ├── dhparams.example.pem
│   │   ├── Dockerfile
│   │   ├── nginx.conf
│   │   ├── nginx-secure.conf
│   │   └── start.sh
│   └── postgres
│   ├── backup.sh
│   ├── Dockerfile
│   ├── list-backups.sh
│   └── restore.sh
├── config
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── settings
│   │   ├── common.py
│   │   ├── common.pyc
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── local.py
│   │   ├── local.pyc
│   │   ├── production.py
│   │   └── test.py
│   ├── urls.py
│   ├── urls.pyc
│   ├── wsgi.py
│   └── wsgi.pyc
├── dev.yml
├── django_quiz
│   ├── build
│   │   ├── bdist.linux-x86_64
│   │   └── lib.linux-x86_64-2.7
│   ├── dist
│   │   └── django_quiz_app-0.5.1-py2.7.egg
│   ├── django_quiz_app.egg-info
│   │   ├── dependency_links.txt
│   │   ├── not-zip-safe
│   │   ├── PKG-INFO
│   │   ├── requires.txt
│   │   ├── SOURCES.txt
│   │   └── top_level.txt
│   ├── essay
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── locale
│   │   ├── migrations
│   │   ├── models.py
│   │   ├── models.pyc
│   │   └── tests.py
│   ├── multichoice
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── locale
│   │   ├── migrations
│   │   ├── models.py
│   │   ├── models.pyc
│   │   └── tests.py
│   ├── quiz
│   │   ├── admin.py
│   │   ├── admin.pyc
│   │   ├── forms.py
│   │   ├── forms.pyc
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── locale
│   │   ├── migrations
│   │   ├── models.py
│   │   ├── models.pyc
│   │   ├── templates
│   │   ├── templatetags
│   │   ├── tests.py
│   │   ├── urls.py
│   │   ├── urls.pyc
│   │   └── views.py
│   ├── runtests.py
│   ├── setup.py
│   ├── test-settings.py
│   └── true_false
│   ├── __init__.py
│   ├── __init__.pyc
│   ├── locale
│   ├── migrations
│   ├── models.py
│   └── tests.py
├── docker-compose.yml
├── docs
│   ├── conf.py
│   ├── deploy.rst
│   ├── docker_ec2.rst
│   ├── index.rst
│   ├── __init__.py
│   ├── install.rst
│   ├── make.bat
│   └── Makefile
├── env
├── gulpfile.js
├── LICENSE
├── manage.py
├── package.json
├── pytest.ini
├── README.rst
├── requirements
│   ├── base.txt
│   ├── local.txt
│   ├── production.txt
│   └── test.txt
├── setup.cfg
├── staticfiles
│   ├── admin
│   │   ├── css
│   │   ├── fonts
│   │   ├── img
│   │   └── js
│   ├── admin_tools
│   │   ├── css
│   │   ├── images
│   │   └── js
│   ├── audio
│   │   └── juicy.mp3
│   ├── css
│   │   ├── animate.min.css
│   │   ├── bootstrap.min.css
│   │   ├── font-awesome.min.css
│   │   ├── lightbox.css
│   │   ├── main.css
│   │   ├── prettyPhoto.css
│   │   ├── progress-wizard.min.css
│   │   ├── project.css
│   │   └── responsive.css
│   ├── debug_toolbar
│   │   ├── css
│   │   ├── img
│   │   └── js
│   ├── django_extensions
│   │   ├── css
│   │   ├── img
│   │   └── js
│   ├── el-pagination
│   │   └── js
│   ├── fonts
│   │   ├── FontAwesome.otf
│   │   ├── fontawesome-webfont.eot
│   │   ├── fontawesome-webfont.svg
│   │   ├── fontawesome-webfont.ttf
│   │   ├── fontawesome-webfont.woff
│   │   ├── fontawesome-webfont.woff2
│   │   ├── glyphicons-halflings-regular.eot
│   │   ├── glyphicons-halflings-regular.svg
│   │   ├── glyphicons-halflings-regular.ttf
│   │   ├── glyphicons-halflings-regular.woff
│   │   └── glyphicons-halflings-regular.woff2
│   ├── images
│   │   ├── 404-bg.png
│   │   ├── 404.png
│   │   ├── aboutus
│   │   ├── blog
│   │   ├── blogdetails
│   │   ├── coming-soon1.png
│   │   ├── coming-soon2.png
│   │   ├── coming-soon3.png
│   │   ├── coming-soon4.png
│   │   ├── coming-soon-bg.png
│   │   ├── contact-bg.png
│   │   ├── favicon.ico
│   │   ├── home
│   │   ├── ico
│   │   ├── icon-map.png
│   │   ├── lightbox
│   │   ├── logo.png
│   │   ├── portfolio
│   │   ├── portfolio-details
│   │   └── services
│   ├── js
│   │   ├── audiojs.swf
│   │   ├── audio.min.js
│   │   ├── bootstrap.min.js
│   │   ├── coundown-timer.js
│   │   ├── gauge.js
│   │   ├── gmaps.js
│   │   ├── holder.js
│   │   ├── html5shiv.js
│   │   ├── jquery.countTo.js
│   │   ├── jquery.fitvids.js
│   │   ├── jquery.isotope.min.js
│   │   ├── jquery.js
│   │   ├── lightbox.min.js
│   │   ├── main.js
│   │   ├── masonry.min.js
│   │   ├── player-graphics.gif
│   │   ├── project.js
│   │   ├── respond.min.js
│   │   ├── w.gif
│   │   └── wow.min.js
│   ├── mptt
│   │   ├── arrow-move.png
│   │   ├── disclosure-down.png
│   │   ├── disclosure-right.png
│   │   ├── draggable-admin.css
│   │   └── draggable-admin.js
│   ├── sass
│   │   └── project.scss
│   ├── zinnia
│   │   ├── admin
│   │   └── theme
│   └── zinnia_bootstrap
│   ├── assets
│   ├── bootstrap
│   └── img
├── utility
│   ├── install_os_dependencies.sh
│   ├── install_python_dependencies.sh
│   ├── requirements-jessie.apt
│   ├── requirements-trusty.apt
│   └── requirements-xenial.apt
└── zinnia_customized
├── admin.py
├── admin.pyc
├── apps.py
├── __init__.py
├── __init__.pyc
├── migrations
│   ├── 0001_initial.py
│   ├── __init__.py
│   └── __init__.pyc
├── models.py
├── models.pyc
├── tests.py
├── urls.py
├── urls.pyc
├── views.py
└── views.pyc
I've installed django-contact-forms inside the virtual env.
Cheers

How to set scrapy IMAGES_STORE relative path

I am trying to set IMAGES_STORE as a relative path but i am getting error and if i am specifying IMAGES_STORE as a Full path it is working fine /home/vaibhav/scrapyprog/comparison/eScraperInterface/images
Error i am getting is at link
Actually it is giving me RuntimeError: OSError: [Errno 20] Not a directory: '/tmp/eScraper-1371463750-Lm8HLh.egg/images' error but if i set Full IMAGE_STORE path it is working fine can someone tell me how can i specify relative path...as i need to deploy this project at various system ...that's why i need relative path....
import os
#------------------------------------------------------------------------------
projectDirPath = os.path.abspath(os.path.dirname((os.path.dirname(__file__))))
imagesDIRPath = projectDirPath + "/images"
BOT_NAME = 'eScraper'
DOWNLOADER_DEBUG = True
CONCURRENT_REQUESTS = 200
AUTOTHROTTLE_DEBUG = True
AUTOTHROTTLE_ENABLED= True
DEPTH_STATS_VERBOSE = True
SPIDER_MODULES = ['eScraper.spiders']
NEWSPIDER_MODULE = 'eScraper.spiders'
COMMANDS_MODULE = 'eScraper.commands'
ITEM_PIPELINES = ['eScraper.pipelines.EscraperPipeline',
'eScraper.pipelines.MySQLStorePipeline']
IMAGES_STORE = imagesDIRPath
DOWNLOADER_MIDDLEWARES = {
'eScraper.rotate_useragent.RotateUserAgentMiddleware' :400,
'scrapy.contrib.downloadermiddleware.useragent.UserAgentMiddleware' : None
}
#------------------------------------------------------------------------------
My project structure:
├── eScraperInterface
│   ├── build
│   │   ├── bdist.linux-i686
│   │   └── lib.linux-i686-2.7
│   │   ├── eScraper
│   │   │   ├── commands
│   │   │   │   ├── __init__.py
│   │   │   │   └── runAllSpiders.py
│   │   │   ├── __init__.py
│   │   │   ├── items.py
│   │   │   ├── pipelines.py
│   │   │   ├── rotate_useragent.py
│   │   │   ├── settings.py
│   │   │   ├── spiders
│   │   │   └── userAgentList.py
│   │   ├── eScraperInterface
│   │   │   ├── __init__.py
│   │   │   ├── settings.py
│   │   │   ├── urls.py
│   │   │   └── wsgi.py
│   │   └── eScraperInterfaceApp
│   │   ├── __init__.py
│   │   ├── models.py
│   │   ├── tests.py
│   │   └── views.py
│   ├── checkImageExist.py
│   ├── eScraper
│   │   ├── commands
│   │   │   ├── __init__.py
│   │   │   ├── __init__.pyc
│   │   │   ├── runAllSpiders.py
│   │   │   └── runAllSpiders.pyc
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── items.py
│   │   ├── items.pyc
│   │   ├── pipelines.py
│   │   ├── pipelines.pyc
│   │   ├── rotate_useragent.py
│   │   ├── rotate_useragent.pyc
│   │   ├── settings.py
│   │   ├── settings.py~
│   │   ├── settings.pyc
│   │   ├── spiders
│   │   ├── userAgentList.py
│   │   └── userAgentList.pyc
│   ├── eScraperInterface
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── settings.py
│   │   ├── settings.pyc
│   │   ├── urls.py
│   │   ├── urls.pyc
│   │   ├── wsgi.py
│   │   └── wsgi.pyc
│   ├── eScraperInterfaceApp
│   │   ├── __init__.py
│   │   ├── __init__.pyc
│   │   ├── models.py
│   │   ├── models.py~
│   │   ├── models.pyc
│   │   ├── tests.py
│   │   └── views.py
│   ├── images
│   ├── __init__.py
│   ├── manage.py
│   ├── project.egg-info
│   │   ├── dependency_links.txt
│   │   ├── entry_points.txt
│   │   ├── PKG-INFO
│   │   ├── SOURCES.txt
│   │   └── top_level.txt
│   ├── scrapy.cfg
│   └── setup.py
├── README.txt
└── README.txt~
Assuming you've provided settings.py from eScraperInterface/eScraper/settings.py:
CUR_DIR = os.path.dirname(os.path.realpath(__file__))
IMAGES_STORE = os.path.join(CUR_DIR, '..', 'images')
Hope that helps.

Categories

Resources