Develop Reference

CNTK on Data Science VM (FasterRCNN): ImportError: No module called 'utils.rpn'

I have just set up a new Data Science Virtual Machine on Azure. I wanted to run a script FasterRCNN.py from
https://github.com/karolzak/CNTK-Hotel-pictures-classificator
but received an error:
from utils.rpn.rpn_helpers import create_rpn, create_proposal_target_layer
ImportError: No module named 'utils.rpn'
The environment I am running the script on has Python 3.4 and CNTK 2.1 installed.
I tried to add a Detection or Detection/utils directories (by editing ~./bashrc) to PYTHONPATH:
export PYTHONPATH="$PYTHONPATH:/home/username/CNTK-Hotel-pictures-classificator/Detection"
as described in
https://github.com/Microsoft/CNTK/issues/2705
but the error persists.
Update:
Screenshot

If anyone is struggling:
the solution to that problem is creating an empty __init__.py file in utils directory.

six: cannot import name python_2_unicode_compatible

With six 1.10.0 installed under Python and pip 2.6, an old Django 1.0.4 app is not able to import python_2_unicode_compatible even though it finds six 1.10.0 just fine:
>>> import six
>>> six.__version__
'1.10.0'
>>> from six import python_2_unicode_compatible
>>>
I've confirmed with python code within the app that it does indeed have access to six:
['appdirs==1.4.3', 'argparse==1.4.0', 'astkit==0.5.4', 'beautifulsoup==3.2.1',
'coverage==4.3.4', 'django-cms==2.0.0a0', 'django==1.0.4', 'dnspython==1.12.0',
'flup==1.0.2', 'importlib==1.0.4', 'iniparse==0.3.1', 'instrumental==0.5.3',
'mako==1.0.6', 'markupsafe==1.0', 'minimock==1.2.8', 'mysql-python==1.2.5',
'nose==1.3.7', 'packaging==16.8', 'pillow==3.4.2', 'pip==9.0.1', 'pluggy==0.4.0',
'py==1.4.33', 'pyparsing==2.2.0', 'python-dateutil==2.6.0', 'pyzor==1.0.0',
'setuptools==35.0.1', 'six==1.10.0', 'sorl-thumbnail==12.3', 'tox==2.7.0',
'uwsgi==2.0.15','virtualenv==15.1.0', 'wheel==0.29.0']
I am tasked to move a very old site that was running django 1.0.4 (you read that right, 1.0.4) and django_cms 2.0.0 Alpha to a new server. The old server croaked, so all I have is the backup of the main website files and dependencies that were installed long ago.
I am Dockerizing it to help document and deploy this in the future.
Ubuntu 14.04
Python 2.6 (same results with 2.7)
Django 1.0.4 (installed via local zip)
django_cms 2.0.0a0 (installed via local zip)
I have tried Apache mod_wsgi, gunicorn (pip2.6 installed) and currently using uwsgi (preferred, pip2.6 installed) to load the app.
Nginx is running in another Docker container with proxy_pass, and will the frontend proxy and TLS.
uwsgi starts the site up with the custom wsgi.
Upon loading the / index page, I had many import errors. Slowly, I am resolving each and every one of them (mostly related to the Django "MIDDLEWARE_CLASSES", which I have yet to find their definition).
I am currently stuck on the following error:
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/wsgi.py", line 230, in __call__
self.load_middleware()
File "/usr/local/lib/python2.6/dist-packages/django/core/handlers/base.py", line 41, in load_middleware
raise exceptions.ImproperlyConfigured, 'Error importing middleware %s: "%s"' % (mw_module, e)
django.core.exceptions.ImproperlyConfigured: Error importing middleware cms.middleware.user: "cannot import name python_2_unicode_compatible"
uwsgi starts up with the specified python2.6 just fine:
web_1 | [uWSGI] getting INI configuration from uwsgi.ini
web_1 | *** Starting uWSGI 2.0.15 (64bit) on [Wed Apr 26 16:27:43 2017] ***
web_1 | Python version: 2.6.9 (default, Oct 22 2014, 19:53:49) [GCC 4.8.2]
web_1 | Python main interpreter initialized at 0xef1050
web_1 | python threads support enabled
Also, python2.7 was originally configured and had the exact same error. I thought I read where python_2_unicode_compatible was deprecated in 2.7 or something, so I went back to the original version the site was running under.
Do I need to install virtualenv? I don't usually do it under Docker, and just install everything globally. I can't see how that would make a difference.
If six was not found, wouldn't I get an error that it could not import six, instead of python_2_unicode_compatible?

The python_2_unicode_compatible method was originally in Django, then added to six in 1.9.
One of your installed packages may be trying to import python_2_unicode_compatible from django.utils.encoding, rather than from the six package.

I'm trying to install skytools in order to migrate a DB to an Amazon RDS, but I'm getting an error

Setup Details:
1x EC2 running Ubuntu 14.04 with Postgres 9.3,
1x RDS running Postgres 9.3
I am trying to move a postgres database that I have on an EC2 onto an RDS. I am using Londiste, which is recommended by AWS for migrating databases with minimal downtime. Specifically, I am using Londiste/Skytools 2.1.13, because Londiste3 requires superuser privileges on both systems.
I have tried installing Skytools both via the zip from their website and via apt-get (both 2.1.13). I have installed what I believe to be all of the dependencies: postgres,libpq-dev, postgresql-server-dev-9.3, pip, and psycopg2.
Regardless, whenever I try to use any skytools commands, I get the following error:
Traceback (most recent call last):
File "/usr/local/bin/pgqadm.py", line 9, in
from pgq.ticker import SmartTicker File "/usr/local/lib/python2.7/dist-packages/pgq/init.py", line 6, in
import pgq.consumer File "/usr/local/lib/python2.7/dist-packages/pgq/consumer.py", line 6,
in
from pgq.baseconsumer import BaseConsumer, BaseBatchWalker File "/usr/local/lib/python2.7/dist-packages/pgq/baseconsumer.py", line 75,
in
class BaseConsumer(skytools.DBScript): File "/usr/local/lib/python2.7/dist-packages/skytools/apipkg.py", line 114,
in __makeattr
result = importobj(modpath, attrname) File "/usr/local/lib/python2.7/dist-packages/skytools/apipkg.py", line 37,
in importobj
module = import(modpath, None, None, ['doc']) File "/usr/local/lib/python2.7/dist-packages/skytools/scripting.py", line
23,
in
default_skylog = skytools.installer_config.skylog
AttributeError: 'module' object has no attribute 'skylog'
Am I missing a dependency? (Skylog seems to be a part of skytools, so I've concluded it's not that.) Has anyone else hit this issue? (There don't seem to be any google results with this error.) Thanks!

tl;dr:
make sure you're running pgqadm.py that is in your local skytools-2.x.y installation directory, eg: ./python/pgqadm.py ticker.ini install
This can be a problem if you have previously installed Skytools 3.x, which may come up first in your PATH.
Background
I just ran into this same issue. I have the same use case (migrating pg->rds), and followed the same recommendation from the AWS documentation.
I had initially installed Skytools 3.x before noticing that it requires superuser privileges on the would-be replica. which pgqadm.py returns /usr/bin/pgqadm.py, which is an artifact of my abandoned Skytools 3.x installation.
I became suspicious of a version conflict when I went to the Skytools source repository looking for instances of skylog. The first result was here:
https://github.com/pgq/skytools-legacy/blob/11afc5210a92d4a2ccf701e4f92138488c31a3cb/python/skytools/installer_config.py.in
However, I realized that this repository is now tracking the latest 3.x release. I switched to the apparent 2.1.x branch, and skylog was nowhere to be found, as expected:
https://github.com/pgq/skytools-legacy/blob/skytools_2_1_stable/python/skytools/installer_config.py.in
Fully-qualifying the path to pgqadm.py resolves this issue in this case.

How to install webpy framework on Windows

I have python 3.3.2 installed on windows 8 x64.
I am trying to install webpy framework. What I tried:
Downloaded latest webpy.
Extracted to python directroy.
in command promp I cd {python_dir/webpy_sub_dir}, then
python setup.py install
I get this error:
Traceback ....
File "setup.py", line 6 ....
from web import __version__
File ........ /web/__init__.py, line 14 in <module>
import utils,db,net,wcgi,http,webapi, .........
ImportError: No module named 'utils'
What could be the problem? I could not find any tutorials relating to windows installs.
P.S. I realise this is borderline "not a programming question" but looking at other stack forums there isn't a better place for this(IMO).

You might need to download the utils package from pypi
https://pypi.python.org/pypi/python-utils/

In the end the problem was that webpy is not compatible with python 3.x. I went with a similar alternative - bottle py.

Web.py is been ported to python 3.
You can join mail list and help

Django: manage.py ImportError because environment variable is undefined

I've just cloned a Django project using Mercurial to a Windows machine that I have set up Python 2.7 on. When I try to run manage.py (with or without a command), I get the following error:
Traceback (most recent call last):
File "C:\Users\jes.000\Documents\project\manage.py", line 30, in <module>
import django.core.servers.basehttp
File "C:\Python27\lib\site-packages\django\core\servers\basehttp.py", line 26, in <module>
from django.views import static
File "C:\Python27\lib\site-packages\django\views\static.py", line 95, in <module>
template_translatable = ugettext_noop(u"Index of %(directory)s")
File "C:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 75, in gettext_noop
return _trans.gettext_noop(message)
File "C:\Python27\lib\site-packages\django\utils\translation\__init__.py", line 48, in __getattr__
if settings.USE_I18N:
File "C:\Python27\lib\site-packages\django\utils\functional.py", line 184, in inner
self._setup()
File "C:\Python27\lib\site-packages\django\conf\__init__.py", line 40, in _setup
raise ImportError("Settings cannot be imported, because environment variable %s is undefined." %ENVIRONMENT_VARIABLE)
ImportError: Settings cannot be imported, because environment variable DJANGO_SETTINGS_MODULE is undefined.
I understand that manage.py is supposed to set that environment variable. I also understand that I might be getting this error because the project is not in the python path... but adding it manually seems like it shouldn't be necessary. Isn't manage.py supposed to resolve this issue as well?
Edit: I've just tested and found that it does the exact same thing under Linux.
Edit: Here's a useful discovery: I only get this error if I install Django via PIP, it works fine if I install the python-django package on Ubuntu. So there's something that the debian package is doing that PIP isn't. Maybe the debian package sets some environment variables? I need to figure that out, since I'm trying to develop on Windows.
Edit: I think I've found the problem, but not a solution. Working on an Ubuntu machine, when I install Django via PIP (pip install django), this project doesn't work with the error I gave. If I create a new project (django-admin startproject testproject) and try that it works, and it consists of manage.py in the created folder, with another folder that contains everything else. Like this:
+-testproject
+-manage.py
+-testproject
+-__init__.py
+-etc.
When I install the Debian package (apt-get install python-django) the project I'm trying to work on works, and if I create a test project (django-admin startproject testproject) it has a folder structure like this:
+-testproject
+-manage.py
+-__init__.py
+-etc.
Notice the difference: the PIP Django package seems to put the manage.py outside of the python module that is the app, while the Debian package puts manage.py inside the module. I assume that this is due to the two packages being of a different version of Django, in between which they changed the structure, because the two packages doing this differently doesn't make any sense.
So my problem is that the project I'm trying to develop was originally generated using the version that's been Debian packaged, while on my Windows machine I'm trying to use the version of Django from pip.
The trouble is that I'm not certain how to fix the problem. Naively moving the manage.py up a directory so it's above the project folder doesn't work, because this 'debian-package version' manage.py tries to just import settings, which fails if it's not in the same folder as settings.py. But somewhere else in Django, something seems to be expecting the project itself to be at //settings.py (as would be the case if the pip version of django-admin was used) rather than at /settings.py (as is the case if the debian-package version is used).
If I look it up, the version in the Ubuntu repositories is 1.3.1, while the version in PyPI is 1.4. Look at the release notes for 1.4, and hey:
An updated default project layout and manage.py that removes the
“magic” from prior versions. And for those who don’t like the new
layout, you can use custom project and app templates instead!
But... shouldn't projects generated with the 1.3 layout still work under 1.4..?

I'm not sure if this has changed in more recent version of django, but the default manage.py simply tries to import the settings.py file and start the manager. You would be expected to put the project in your PYTHONPATH, or to put the settings.py into your DJANGO_SETTINGS_MODULE variable, by default.
You can modify the manage.py to be a bit smarter with this snippet:
manage.py
#!/usr/bin/env python
from django.core.management import execute_manager
...
import settings
import os
project = os.path.basename(os.path.dirname(__file__))
os.environ['DJANGO_SETTINGS_MODULE'] = '%s.settings' % project
if __name__ == "__main__":
execute_manager(settings)
Now you can run manage.py from any location, since it will always explicitly set up the path to the settings.py file.

You may add the following code to the file manage.py ,like this:
from django.conf import settings
settings.configure()

This is indeed due to the difference in default layouts between 1.3 and 1.4. Normally Django 1.3 projects will work under 1.4, so something else is going on here that I can't quite figure out. I have established that the problem isn't platform-dependent.
I couldn't figure out the exact problem but I did figure out a fix. That was:
Create a new project under Django 1.4 and poach its manage.py file. After changing the project name, this file works as a manage.py. I named it 'manage14.py' so that the original manage.py would be left for my colleagues using Django 1.3.
Add the directory above the one that the project was in to the global PYTHONPATH. This is not an ideal solution, but it allowed an unmodified Django 1.4 manage.py to load the project without having to convert the project to Django 1.4's organization.
As soon a practical, I plan to convert the project to the Django 1.4 format since it seems to be better anyway, particularly from the perspective of avoiding this kind of problem. Unfortunately I'll have to wait until the Ubuntu repo's get 1.4 so that my colleagues will be up to the same version I am on Windows using PyPI.