Develop Reference

Python 3 flask install wkhtmltopdf on heroku

I have a problem to install the wkhtmltopdf binary on my heroku python app (flask).
A year ago (python 2) I already had an issue, but I was able to solve it by first adding the wkhtmltopdf-pack to the requirements and installing it on heroku and then setting the config var to WKHTMLTOPDF_BINARY=wkhtmltopdf-pack. Here is my old thread
Problem now:
I am trying to use the same approach for python 3, but no version of the wkhtmltopdf-pack works, every push gets rejected and I cant install it.
I tried these versions in the requirements:
wkhtmltopdf-pack==0.12.5
wkhtmltopdf-pack==0.12.4
wkhtmltopdf-pack==0.12.3
wkhtmltopdf-pack==0.12.3.0.post1
wkhtmltopdf-pack==0.12.2.4
I get these errors:
No matching distribution
or
error: can't copy 'bin/wkhtmltopdf-pack': doesn't exist or not a regular file
and I remember once it told me there was a SyntaxError and it could not decode something.
Alternative approach:
It seems it is also possible to use a buildpack, so I tried adding a buildpack:
heroku buildpacks:add https://github.com/dscout/wkhtmltopdf-buildpack.git
I see that the buildpack has been added, but there was no installation and there is also no config var for wkhtmltopdf. I dont understand how to trigger the installation, in all documantations for buildpacks its written "add the buildpack and you are ready to go".
Trying to create a PDF gives me a server error here:
OSError: No wkhtmltopdf executable found: "b''"
EDIT:
I managed to install the buildpack:
The push was successful, but no config var has been created and I have no clue what the path to the binary is.
EDIT
I was able to find the files through heroku bash:
app bin dev etc lib lib64 lost+found proc sbin sys tmp usr var
/ $ cd app
~ $ cd vendor
~/vendor $ dir
wkhtmltox
~/vendor $ cd wkhtmltox
~/vendor/wkhtmltox $ dir
lib
~/vendor/wkhtmltox $ cd lib
~/vendor/wkhtmltox/lib $ dir
libwkhtmltox.so libwkhtmltox.so.0 libwkhtmltox.so.0.12 libwkhtmltox.so.0.12.3
~/vendor/wkhtmltox/lib $ exit
Now I tried to all these files but all give an error:
OSError: wkhtmltopdf exited with non-zero code -11. error
Here is how I set the path:
# WKHTMLTOPDF config
if 'DYNO' in os.environ:
print ('loading wkhtmltopdf path on heroku')
MYDIR = os.path.dirname(__file__)
WKHTMLTOPDF_CMD = os.path.join(MYDIR + "/vendor/wkhtmltox/lib/", "libwkhtmltox.so")
else:
print ('loading wkhtmltopdf path on localhost')
MYDIR = os.path.dirname(__file__)
WKHTMLTOPDF_CMD = os.path.join(MYDIR + "/static/executables/bin/", "wkhtmltopdf.exe")

The best approach to get installed wkhtmltopdf on Heroku by getting the binary of wkhtmltopdf for python 3 instead of wkhtmltopdf-pack and you can achieve this by using pydf.
You can install it simply using pip like:
pip install python-pdf
or for Python 2:
pip install python-pdf==0.30.0
Unlike the buildpack based approach pydf installs with the wkhtmltopdf binary included making it very easy to use, and this is the right approach for Heroku.
But if you still want to stick with build-pack wkhtmltopdf, here's another solution you can give it a try:
Via: CLI Installation
$ heroku create --buildpack https://github.com/homelight/wkhtmltox-buildpack.git
Or Manually:
Add the following line to your .buildpacks file
https://github.com/homelight/wkhtmltox-buildpack.git
Please note that this buildpack is only compatible with the cedar-14 stack. You can use heroku stack:set cedar-14 to set the correct stack.

I was able to solve the problem on my own, following my first approach.
I found an other wkhtmltopdf-pack on pypi and added it to my requirements.txt:
wkhtmltopdf-pack-ng==0.12.3.0
Heroku was able to install this pack.
After that I added the config var for wkhtmltopdf:
heroku config:set WKHTMLTOPDF_BINARY=wkhtmltopdf-pack
The installation is now complete. I need to use the correct path now on my app:
if 'DYNO' in os.environ:
print ('loading wkhtmltopdf path on heroku')
WKHTMLTOPDF_CMD = subprocess.Popen(
['which', os.environ.get('WKHTMLTOPDF_BINARY', 'wkhtmltopdf-pack')], # Note we default to 'wkhtmltopdf' as the binary name
stdout=subprocess.PIPE).communicate()[0].strip()
else:
print ('loading wkhtmltopdf path on localhost')
MYDIR = os.path.dirname(__file__)
WKHTMLTOPDF_CMD = os.path.join(MYDIR + "/static/executables/bin/", "wkhtmltopdf.exe")
Thats it.

Provisioning CoreOS with Ansible pip error

I am trying to provision a coreOS box using Ansible. First a bootstapped the box using https://github.com/defunctzombie/ansible-coreos-bootstrap
This seems to work ad all but pip (located in /home/core/bin) is not added to the path. In a next step I am trying to run a task that installs docker-py:
- name: Install docker-py
pip: name=docker-py
As pip's folder is not in path I did it using ansible:
environment:
PATH: /home/core/bin:$PATH
If I am trying to execute this task I get the following error:
fatal: [192.168.0.160]: FAILED! => {"changed": false, "cmd": "/home/core/bin/pip install docker-py", "failed": true, "msg": "\n:stderr: /home/core/bin/pip: line 2: basename: command not found\n/home/core/bin/pip: line 2: /root/pypy/bin/: No such file or directory\n"}
what I ask is where does /root/pypy/bin/ come from it seems this is the problem. Any idea?

You can't use shell-style variable expansion when setting Ansible variables. In this statement...
environment:
PATH: /home/core/bin:$PATH
...you are setting your PATH environment variable to the literal value /home/core/bin:$PATH. In other words, you are blowing away any existing value of $PATH, which is why you're getting "command not found" errors for basic things like basename.
Consider installing pip somewhere in your existing $PATH, modifying $PATH before calling ansible, or calling pip from a shells cript:
- name: install something with pip
shell: |
PATH="/home/core/bin:$PATH"
pip install some_module

The problem lies in /home/core/bin/pip script which is literally:
#!/bin/bash
LD_LIBRARY_PATH=$HOME/pypy/lib:$LD_LIBRARY_PATH $HOME/pypy/bin/$(basename $0) $#
when run under root by ansible the $HOME variable is substituted with /root and not with /home/core.
Change $HOME with /home/core and it should work.

How to parse python module for openstack

Recently, when I was installing openstack on 3 vm on centos 7 using answer file I had the following error:
10.7.35.174_osclient.pp: [ ERROR ]
Applying Puppet manifests [ ERROR ]
ERROR : Error appeared during Puppet run: 10.7.35.174_osclient.pp
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list python-iso8601' returned 1: Error: No matching Packages to list
You will find full trace in log /var/tmp/packstack/20160318-124834-91QzZC/manifests/10.7.35.174_osclient.pp.log
Please check log file /var/tmp/packstack/20160318-124834-91QzZC/openstack-setup.log for more information
Additional information:
* Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
* File /root/keystonerc_admin has been created on OpenStack client host 10.7.35.174. To use the command line tools you need to source the file.
* To access the OpenStack Dashboard browse to http://10.7.35.174/dashboard .
Please, find your login credentials stored in the keystonerc_admin in your home directory.
I have already manually installed that module, but the same problem occures anyway.
That command only runs like that:
/usr/bin/yum -d 0 -e 0 -y list python2-iso8601
Is there any way to parse it to python?
Do you have any ideas how to solve it?

Found that kilo version works fine.

DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both

I've been usually installed python packages through pip.
For Google App Engine, I need to install packages to another target directory.
I've tried:
pip install -I flask-restful --target ./lib
but it fails with:
must supply either home or prefix/exec-prefix -- not both
How can I get this to work?

Are you using OS X and Homebrew? The Homebrew python page https://github.com/Homebrew/brew/blob/master/docs/Homebrew-and-Python.md calls out a known issue with pip and a work around.
Worked for me.
You can make this "empty prefix" the default by adding a
~/.pydistutils.cfg file with the following contents:
[install]
prefix=
Edit: The Homebrew page was later changed to recommend passing --prefix on the command line, as discussed in the comments below. Here is the last version which contained that text. Unfortunately this only works for sdists, not wheels.
The issue was reported to pip, which later fixed it for --user. That's probably why the section has now been removed from the Homebrew page. However, the problem still occurs when using --target as in the question above.

I believe there is a simpler solution to this problem (Homebrew's Python on macOS) that won't break your normal pip operations.
All you have to do is to create a setup.cfg file at the root directory of your project, usually where your main __init__.py or executable py file is. So if the root folder of your project is: /path/to/my/project/, create a setup.cfg file in there and put the magic words inside:
[install]
prefix=
OK, now you sould be able to run pip's commands for that folder:
pip install package -t /path/to/my/project/
This command will run gracefully for that folder only. Just copy setup.cfg to whatever other projects you might have. No need to write a .pydistutils.cfg on your home directory.
After you are done installing the modules, you may remove setup.cfg.

On OSX(mac), assuming a project folder called /var/myproject
cd /var/myproject
Create a file called setup.cfg and add
[install]
prefix=
Run pip install <packagename> -t .

Another solution* for Homebrew users is simply to use a virtualenv.
Of course, that may remove the need for the target directory anyway - but even if it doesn't, I've found --target works by default (as in, without creating/modifying a config file) when in a virtual environment.
*I say solution; perhaps it's just another motivation to meticulously use venvs...

I hit errors with the other recommendations around --install-option="--prefix=lib". The only thing I found that worked is using PYTHONUSERBASE as described here.
export PYTHONUSERBASE=lib
pip install -I flask-restful --user
this is not exactly the same as --target, but it does the trick for me in any case.

As other mentioned, this is known bug with pip & python installed with homebrew.
If you create ~/.pydistutils.cfg file with "empty prefix" instruction it will fix this problem but it will break normal pip operations.
Until this bug is officially addressed, one of the options would be to create your own bash script that would handle this case:
#!/bin/bash
name=''
target=''
while getopts 'n:t:' flag; do
case "${flag}" in
n) name="${OPTARG}" ;;
t) target="${OPTARG}" ;;
esac
done
if [ -z "$target" ];
then
echo "Target parameter must be provided"
exit 1
fi
if [ -z "$name" ];
then
echo "Name parameter must be provided"
exit 1
fi
# current workaround for homebrew bug
file=$HOME'/.pydistutils.cfg'
touch $file
/bin/cat <<EOM >$file
[install]
prefix=
EOM
# end of current workaround for homebrew bug
pip install -I $name --target $target
# current workaround for homebrew bug
rm -rf $file
# end of current workaround for homebrew bug
This script wraps your command and:
accepts name and target parameters
checks if those parameters are empty
creates ~/.pydistutils.cfg file with "empty prefix" instruction in it
executes your pip command with provided parameters
removes ~/.pydistutils.cfg file
This script can be changed and adapted to address your needs but you get idea. And it allows you to run your command without braking pip. Hope it helps :)

If you're using virtualenv*, it might be a good idea to double check which pip you're using.
If you see something like /usr/local/bin/pip you've broken out of your environment. Reactivating your virtualenv will fix this:
VirtualEnv: $ source bin/activate
VirtualFish: $ vf activate [environ]
*: I use virtualfish, but I assume this tip is relevant to both.

I have a similar issue.
I use the --system flag to avoid the error as I decribe here on other thread where I explain the specific case of my situation.
I post this here expecting that can help anyone facing the same problem.

Mininet Cannot find required executable controller

Whenever I want to run sshd.py example in mininet or some custome code I have written myself I get
*** Creating network
*** Adding controller
*** Adding hosts:
h1 h2 h3 h4 h5
*** Adding switches:
s1
*** Adding links:
(h1, s1) (h2, s1) (h3, s1) (h4, s1) (h5, s1)
*** Configuring hosts
h1 h2 h3 h4 h5
*** Starting controller
Cannot find required executable controller.
Please make sure that it is installed and available in your $PATH:
(/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin)
Though $ sudo mn --test pingall works fine!
I guess I should find the open flow controller executable and add it to the path but I don't know where is it located. but the test create a controller and works fine!
I have tried to reinstalling mininet with
$ ~/mininet/util/install.sh -a or [-fnv]

Stumbled upon the same issue with mininet on Ubuntu.
Try to explicitly specify the controller class when constructing a Mininet object, e.g. instead of
net = Mininet(topo)
do
from mininet.node import OVSController
net = Mininet(topo = topo, controller = OVSController)
That solved the problem in my case.

In the newst versions of OVS, the ovs-controller was renamed to test-controller.
First install the openvswitch-testcontroller if you haven't with the following command:
sudo apt-get install openvswitch-testcontroller
Second, create a symbolic link to the test-controller:
sudo ln /usr/bin/ovs-testcontroller /usr/bin/controller
That works for me.
You also, can review this link: http://installfights.blogspot.com.co/2016/09/cannot-find-required-executable.html

I wish the message was like
Cannot find required executable "controller".
Anyway, as long as you have ovs-controller installed e.g. provided by 'openvswitch-controller' package on debian like platform, all you have to do is
sudo ln /usr/bin/ovs-controller /usr/bin/controller

There is a problem with installation as far as I perceived. I tried the mininet VM and everything works fine in it.

I had the same problem and I solved it by installing the new version of mininet (2.1.0). With the 2.0.0 version I couldn't do anything.

ubuntu#ubuntu:~$ cd mininet/examples
ubuntu#ubuntu:~/mininet/examples$ dir
baresshd.py hwintf.py multipoll.py scratchnetuser.py
consoles.py limit.py multitest.py simpleperf.py
controllers2.py linearbandwidth.py popenpoll.py sshd.py
controllers.py milsontest.py popen.py tree1024.py
cpu.py miniedit.py README treeping64.py
emptynet.py multiping.py scratchnet.py
ubuntu#ubuntu:~/mininet/examples$ nano sshd.py
ubuntu#ubuntu:~/mininet/examples$ sudo python sshd.py
Have you tried this?
OR, if you using sudo mn command then try :
sudo mn --controller=remote

I had the same problem, so I removed mininet and instead installing again with "apt-get install mininet", I downloaded the source and installed everything that is included in the Mininet VM. This solved the problem.
Just check out the "Option 2: Native Installation from Source" on the website:
http://mininet.org/download/#option-1-mininet-vm-installation-easy-recommended

Just ran into this issue on the "SDN Hub tutorial VM 64-bit with Docker" (Ubuntu 14.04) when running MiniEdit. Curiously, when I ran my first MiniEdit topology from this great tutorial there were no problems.
But when I closed MiniEdit, closed the Mininet client, and moved on to run another topology, I got "Mininet Cannot find required executable controller" error.
I'll note that, when working with MiniEdit, the Mininet Client explicitly states:
NOTE: PLEASE REMEMBER TO EXIT THE CLI BEFORE YOU PRESS THE STOP BUTTON. Not exiting will prevent MiniEdit from quitting and will prevent you from starting the network again during this sessoin.
In this SDN Hub tutorial VM that I'm working with the controller file is /usr/bin/ovs-vsctl. I needed to create a softlink ln to a file called /usr/bin/controller for MiniNet to find the controller configuration. I did this with the command:
sudo ln /usr/bin/ovs-vsctl /usr/bin/controller
Finally, Mininet cleanup is a useful command if there are errors in due to existing/leftover topology configurations.
sudo mn -c