Develop Reference

'Module' has no attritube 'Treestyle'

If found some hits on stackoverflow where people have issues using ete3 tools when PyQT4 is not installed, and this (used to) also apply for me. However, I've now tried to reinstall a piece of software and it keeps complaining about 'Module' has no attritube 'Treestyle'.
To test whether PyQT was installed, I ran this python script:
from PyQt4.Qt import PYQT_VERSION_STR
print("PyQt version:", PYQT_VERSION_STR)
Which prints:
('PyQt version:', '4.12.1')
I get this (common) error:
Traceback (most recent call last):
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 1820, in <module>
sys.exit(main())
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 1793, in main
args.start(args)
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 408, in start_evo_sim
result = init_and_simulate(simu, intermediate_load_file, _options)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/my_tools/utility.py", line 117, in wrapper
raise ex_type(message)
AttributeError: 'module' object has no attribute 'TreeStyle' (in subprocess)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/my_tools/utility.py", line 91, in process_func
ret = func(*args, **kwargs)
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 295, in init_and_simulate
sim = sim_mod.create_simulation(**options)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 1940, in create_simulation
sim = ODE_simulation(params_dict)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 1342, in __init__
super(ODE_simulation, self).__init__(params)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 86, in __init__
self.init_graphs()
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 795, in init_graphs
show=show, clean=clean, create=create)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1188, in __init__
self.init_phylo_tree_graph(clean=clean)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1219, in init_phylo_tree_graph
show=show, attribute_dict=self.attribute_mapper, create=clean, **kwargs)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 985, in __init__
self.init_tree_style_dict() # NOTE: unordered ok
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1110, in init_tree_style_dict
branch_vertical_margin=branch_vertical_margin)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1083, in make_tree_style
ts = ete3.TreeStyle()
Note: this time arround I am trying to install it on Ubuntu for Windows, but up till now almost everything that ran on Linux ran on this one too. It might be relevent though.
Any ideas?
PS To reproduce the problem, follow the following steps:
$ sudo apt-get install python-qt4
$ virtualenv ~/mypy --system-site-packages
$ source ~/mypy/bin/activate
$ (mypy) > pip install VirtualMicrobes
$ (mypy) > virtualmicrobes evo --name TestMicrobes >> error
PPS User eyllanesc confirmed this is an 'Ubuntu for Windows' problem. I'm still trying to fix it though, if anyone can help me figure out how to link the pyqt4 packages correctly to my virtualenv that would be great. I've tried linken the native one with ln -s but I'm not getting any further

I had similar issues. The problem might be that you don't have all the depencies installed and ete3 is not telling you all of them.
You can see which dependencies are missing with this code:
import ete3
ete3.__file__
Run it and it will tell you which dependencies are missing. Note that you might need to run it multiple times before you install them all!

Google Python cloud-dataflow instances broke without new deployment (failed pubsub import)

I have defined a few different Cloud Dataflow jobs for Python in the Google AppEngine Flex Environment. I have defined my requirements in a requirements.txt file, included my setup.py file, and everything was working just fine. My last deployment was on May 3rd, 2018. Looking through logs, I see that one of my jobs began failing on May 22nd, 2018. The job fails with a stack trace resulting from a bad import, seen below.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 582, in do_work
work_executor.execute()
File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", line 166, in execute
op.start()
File "apache_beam/runners/worker/operations.py", line 294, in apache_beam.runners.worker.operations.DoOperation.start (apache_beam/runners/worker/operations.c:10607)
def start(self):
File "apache_beam/runners/worker/operations.py", line 295, in apache_beam.runners.worker.operations.DoOperation.start (apache_beam/runners/worker/operations.c:10501)
with self.scoped_start_state:
File "apache_beam/runners/worker/operations.py", line 300, in apache_beam.runners.worker.operations.DoOperation.start (apache_beam/runners/worker/operations.c:9702)
pickler.loads(self.spec.serialized_fn))
File "/usr/local/lib/python2.7/dist-packages/apache_beam/internal/pickler.py", line 225, in loads
return dill.loads(s)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 277, in loads
return load(file)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 266, in load
obj = pik.load()
File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 1090, in load_global
klass = self.find_class(module, name)
File "/usr/local/lib/python2.7/dist-packages/dill/dill.py", line 423, in find_class
return StockUnpickler.find_class(self, module, name)
File "/usr/lib/python2.7/pickle.py", line 1124, in find_class
__import__(module)
File "/usr/local/lib/python2.7/dist-packages/dataflow_pipeline/tally_overages.py", line 27, in <module>
from google.cloud import pubsub
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub.py", line 17, in <module>
from google.cloud.pubsub_v1 import PublisherClient
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/__init__.py", line 17, in <module>
from google.cloud.pubsub_v1 import types
File "/usr/local/lib/python2.7/dist-packages/google/cloud/pubsub_v1/types.py", line 26, in <module>
from google.iam.v1.logging import audit_data_pb2
ImportError: No module named logging
So the main issue seems to come from the pubsub dependency relying on importing google.iam.v1.logging, which is installed from grpc-google-iam-v1.
Here is my requirements.txt file
Flask==0.12.2
apache-beam[gcp]==2.1.1
gunicorn==19.7.1
google-cloud-dataflow==2.1.1
google-cloud-datastore==1.3.0
pytz
google-cloud-pubsub
google-gax
grpc-google-iam-v1
googleapis-common-protos
google-cloud==0.32
six==1.10.0
protobuf
I am able to run everything locally just fine by doing the following from my project.
$ virtualenv --no-site-packages .
$ . bin/activate
$ pip install --ignore-installed -r requirements.txt
$ python main.py
No handlers could be found for logger "oauth2client.contrib.multistore_file"
INFO:werkzeug: * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
INFO:werkzeug: * Restarting with stat
No handlers could be found for logger "oauth2client.contrib.multistore_file"
WARNING:werkzeug: * Debugger is active!
INFO:werkzeug: * Debugger PIN: 317-820-645
specifically, I am able to do the following locally just fine
$ python
>>> from google.cloud import pubsub
>>> import google.iam.v1.logging
>>> google.iam.v1.logging.__file__
'/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/google/iam/v1/logging/__init__.pyc'
So I know that the installation of the grpc-google-iam-v1 package is working just fine locally.. the required files are there.
My questions are
Why is the install of grpc-google-iam-v1 on the Google AppEngine Flex Environment not installing all of the files correctly? I must be missing the /site-packages/google/iam/v1/logging directory.
Why would this randomly start failing? I didn't do any more deploys, the same code was running and working on the 21st and then it broke on the 22nd on May.
I was able to get the pipeline running again after changing the requirements.txt file to
Flask==0.12.2
apache-beam[gcp]
google-cloud-dataflow
gunicorn==19.7.1
google-cloud-datastore==1.3.0
pytz
google-cloud-pubsub
google-gax
grpc-google-iam-v1
googleapis-common-protos
google-cloud==0.32
six==1.10.0
protobuf
so simply removing the version requirements from apache-beam[gcp] and google-cloud-dataflow did the trick.

Building on the solution provided by John Allard, removing the version from the requirements.txt will automatically default to the latest version. Thus, with no version specified for apache-beam[gcp] , google-cloud-dataflow and google-cloud-pubsub they will all run on the latest version and solve the dependency issue. The requirements.txt will look like the following:
Flask==0.12.2
apache-beam[gcp]
gunicorn==19.7.1
google-cloud-dataflow
google-cloud-datastore==1.3.0
pytz
google-cloud-pubsub
google-gax
grpc-google-iam-v1
googleapis-common-protos
google-cloud==0.32
six==1.10.0
protobuf

Cloning a heroku hosted django project to your local machine

I'm new to python, Django, heroku and even PostgreSQL but am working on a Django project with a team of 3 devs (they set it up), so I'm trying to set it up on my local machine so I can start developing.
Firstly I cloned the repository to my local machine, there was a requirement.txt file, so I ran pip install -r ./requirements.txt to install all the python packages the project uses. Theres a runtime.txt file which specifies python 3.5 so I need to upgrade that. I see that gunicorn is one of the packages in requiments.txt, so I'm guessing I need to use that as the web server.
Heres the packages in requirements.txt that look important to me:
Django==1.9.6
gunicorn==19.5.0
psycopg2==2.6.1
boto3==1.3.1
boto==2.40.0
whitenoise==3.2
django-extensions==1.6.7
The site also has a .buildpacks file, which seems to be a heroku specific thing:
https://github.com/a2ikm/heroku-buildpack-libjpeg62
https://github.com/heroku/heroku-buildpack-python
https://github.com/rafaelp/heroku-buildpack-wkhtmltopdf
Theres a Procfile that contains this:
web: gunicorn myproject.wsgi --log-file -
worker: celery worker --app=genome.tasks.app -l info --beat
I'm wondering what the best way to do this is. Is there a way to automatically generate a virtual environment that has all the requirements and software specified in the project? I'm already using a virtual dev environment with created with Vagrant so I could provision it with all the dependencies, but I get the feeling theres a better way to do this.
UPDATE:
Heres what what happens when I try to run the server with python manage.py runserver:
$ python manage.py runserver
Current environment: P
Current environment: P
Performing system checks...
Unhandled exception in thread started by <function wrapper at 0x7fd0d08c71b8>
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/django/utils/autoreload.py", line 226, in wrapper
fn(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/runserver.py", line 116, in inner_run
self.check(display_num_errors=True)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 426, in check
include_deployment_checks=include_deployment_checks,
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/registry.py", line 75, in run_checks
new_errors = check(app_configs=app_configs)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 13, in check_url_config
return check_resolver(resolver)
File "/usr/local/lib/python2.7/dist-packages/django/core/checks/urls.py", line 23, in check_resolver
for pattern in resolver.url_patterns:
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 417, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/usr/local/lib/python2.7/dist-packages/django/utils/functional.py", line 33, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/usr/local/lib/python2.7/dist-packages/django/core/urlresolvers.py", line 410, in urlconf_module
return import_module(self.urlconf_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/var/www/sd2/selfdecode/decodify/urls.py", line 30, in <module>
url(r'', include("genome.urls")),
File "/usr/local/lib/python2.7/dist-packages/django/conf/urls/__init__.py", line 52, in include
urlconf_module = import_module(urlconf_module)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/var/www/sd2/selfdecode/genome/urls.py", line 7, in <module>
from . import views
File "/var/www/sd2/selfdecode/genome/views.py", line 20, in <module>
from decodify.aggregators import ArrayAgg, JsonAgg, ArrayaAggDelimited
File "/var/www/sd2/selfdecode/decodify/aggregators.py", line 29
def __init__(self, *expressions, distinct=False, **extra):
^
SyntaxError: invalid syntax
I'm guessing the parse error is due to using python 2.7 in a project that uses python 3.5. I installed python 3.5 with sudo apt-get install python3-all. Trying the same thing with python3:
$ python3 manage.py runserver
Traceback (most recent call last):
File "manage.py", line 8, in <module>
from django.core.management import execute_from_command_line
ImportError: No module named 'django'
> python3
The second error means the packages I installed with pip install requirements.txt, they were installed for python 2.7, not 3.5. So this is obviously where virtual environments can come in. I tried this sudo -H virtualenv -p /usr/bin/python3 py3env, both with and without sudo, and both times I got errors :
[06:08 PM]-[vagrant#machine1]-[/var/www/sd2/selfdecode]-[git master]
$ sudo -H virtualenv -p /usr/bin/python3 py3env
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /var/www/sd2/selfdecode/py3env/bin/python3
Also creating executable in /var/www/sd2/selfdecode/py3env/bin/python
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 2328, in <module>
main()
File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 713, in main
symlink=options.symlink)
File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 925, in create_environment
site_packages=site_packages, clear=clear, symlink=symlink))
File "/usr/local/lib/python2.7/dist-packages/virtualenv.py", line 1370, in install_python
os.symlink(py_executable_base, full_pth)
OSError: [Errno 71] Protocol error: 'python3' -> '/var/www/sd2/myproj/py3env/bin/python'
I came across this thread:
protocol error setting up virtualenvironment through vagrant on ubuntu
Where someone was having this problem using virtualenv inside a vagrant VM, and a possible solution is setting up the venv outside of the shared vagrant folder.

Unable to start App Engine application after updating it via Google Cloud SDK

Recently, I have updated Google App Engine from 1.9.17 to 1.9.18 via Google Cloud SDK by using command 'gcloud components update' in Windows 7 64 bit. After that I wasn't able to start any project using the App Engine launcher. Getting this error:
Traceback (most recent call last):
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\dev_appserver.py", line 83, in <module>
_run_file(__file__, globals())
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\dev_appserver.py", line 79, in _run_file
execfile(_PATHS.script_file(script_name), globals_)
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\devappserver2.py", line 36, in <module>
from google.appengine.tools.devappserver2 import dispatcher
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\dispatcher.py", line 29, in <module>
from google.appengine.tools.devappserver2 import module
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\module.py", line 71, in <module>
from google.appengine.tools.devappserver2 import vm_runtime_factory
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\vm_runtime_factory.py", line 25, in <module>
from google.appengine.tools.devappserver2 import vm_runtime_proxy
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\vm_runtime_proxy.py", line 29, in <module>
from google.appengine.tools.devappserver2 import log_manager
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\devappserver2\log_manager.py", line 34, in <module>
from google.appengine.tools.docker import containers
File "C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\docker\containers.py", line 47, in <module>
import docker
ImportError: No module named docker
2015-03-05 19:11:27 (Process exited with code 1)
I even installed the latest Google Cloud SDK, but I'm getting the same error.
I'm able to install the appengine SDK 1.9.18(without using Google Cloud SDK) and able to run the project successfully.
This error is happening only for the App Engine launcher installed via Google Cloud SDK in Windows 7.
This issue is raised in App Engine Issue Tracker: Issue 125. I recommend you to star this issue.

This has happened to me today to reinstall the app engine sdk. I could not run my code in the launcher.
I remember reading that is not used pip app engine, but now I have solved the problem.
In short what I did was:
Install pip the footsteps of https://pip.pypa.io/en/latest/installing.html (this also correctly install the setuptools)
Install docker-py by pip: pip install docker-py and ready, I can now run my code in the launcher
P.S.
Previously I tried to install the docker-py package, downloaded from https://github.com/docker/docker-py, but lacked setuptools, downloaded and installed the package did not work. So use this with pip.

This is currently an issue with the dev_appserver bundled in the Cloud SDK. A fix will be out soon. In the meanwhile, your options are:
1) Use gcloud preview app run to run your app when using the Cloud SDK
2) Install the standalone AppEngine SDK (which you already mentioned in your question)

If installing docker-py doesn't work and the stacktrace shows that the error line is:
from docker import docker
Change this line to:
import docker
Source

Jumping on the answer from #Tzach and adding some info.
The file to modify is containers.py
for me it is located here :
C:\Program Files\Google\Cloud SDK\google-cloud-sdk\platform\google_appengine\google\appengine\tools\docker
If you can't modify it because the file is open in an application, it is in fact that the folder is protected. Just copy/paste the file on your desktop and modify it from there. then copy it back in the original folder.

Django: Import error when trying to use django-audiofield from GitHub

I'm trying to use a django application on Github called django-audiofield in my django project. I'm using Aptana Studio IDE. I installed it in my virtual environment and I'm getting the following error:
Traceback (most recent call last):
File "/Users/home/Desktop/Web Development/Aptana Studio 3.0/musicproject/src/manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
utility.execute()
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
output = self.handle(*args, **options)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
return self.handle_noargs(**options)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/commands/validate.py", line 9, in handle_noargs
self.validate(display_num_errors=True)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/base.py", line 266, in validate
num_errors = get_validation_errors(s, app)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/core/management/validation.py", line 30, in get_validation_errors
for (app_name, error) in get_app_errors().items():
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/db/models/loading.py", line 158, in get_app_errors
self._populate()
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/db/models/loading.py", line 67, in _populate
self.load_app(app_name)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/db/models/loading.py", line 88, in load_app
models = import_module('.models', app_name)
File "/Users/home/virtualenv/venv/lib/python2.7/site-packages/django/utils/importlib.py", line 35, in import_module
__import__(name)
File "/Users/home/virtualenv/venv/src/audiofield/audiofield/models.py", line 19, in <module>
from audiofield.fields import AudioField
File "/Users/home/virtualenv/venv/src/audiofield/audiofield/fields.py", line 20, in <module>
from tasks import audio_convert_task
File "/Users/home/virtualenv/venv/src/audiofield/audiofield/tasks.py", line 14, in <module>
from celery.task import Task
ImportError: No module named celery.task
Finished "/Users/home/Desktop/Web Development/Aptana Studio 3.0/musicproject/src/manage.py validate" execution.
Please assist.

I'm the maintainer of the package, thanks to your post I realized that we were missing celery/django-celery in our requirements. It's now added in the latest version.
So "pip install django-audiofield" will install those dependencies for you.
If you have django-audiofield already installed you can run the following command to upgrade : "pip install django-audiofield --upgrade"

Have you installed celery? If no, install celery (pip install celery). If the github application in question (a link would have been nice) has a requirements.txt file anywhere, do pip install -r requirements.txt to install all the required dependencies of the application.
If yes, it looks like your import search path may not be set up correctly to find Celery - just a guess based on the import error.
Open up the django shell (type python manage.py shell when you're in your project's working directory) and type in import sys; print sys.path. Make sure that Celery is installed in one of those directories, since sys.path specifies the import search path for python modules.
If it isn't included there, you can temporarily append the directory celery is installed into by doing sys.path.append("<directory where celery is>") (mind the backslashes, since you're on Windows). If, after doing that, you can successfully from celery.tasks import Task, then you can set the PYTHONPATH environment variable appropriately to make it permanent.
Also, since you installed Celery into a virtualenv, it may be worth checking that both django and Celery are in the same virtualenv.
Finally, the latest commits to Celery indicate Celery now has native Django support, but since that hasn't been released as a stable version yet, you may want to look at django-celery.
Edit (in response to comment below):
Since you mentioned you're trying to get django-audiofield to work: it seems to be on pypi. This means you can use pip (or easy_install) to install it using pip install django-audiofield once you've installed all the required (non-python) dependencies, and stuff like celery should be downloaded for you.