I want to use the ffmpeg buildpack in my Python app on Heroku.
I am using the ffmpeg buildpack from https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.
How can I use buildpack? subprocess? os? How to call the ffmpeg?
Anybody can teach me?
This is my code and I want to convert mp4 file to mp3 file.
Actually,I don`t know about the detect/compile/release file.
subprocess.call(['ffmpeg', '-i', 'xxx.mp4','-vn','-f mp3', 'xxx.mp3'])
subprocess.call(['ffmpeg', '-i', 'xxx.mp4','-vn','-f mp3', 'xxx.mp3'])
First of, I assume you already know how to deploy a Python app to Heroku and you already have a working app accessible from Heroku, as this answer is specific to how to use the ffmpeg buildpack. (If you don't yet, check Getting Started on Heroku with Python first).
Step 1: Adding the ffmpeg buildpack
Buildpacks basically tell Heroku how to setup the environment for your app (which dependencies to install, which scripts to run, etc.). For Python apps, you need to have the official heroku/python buildpack, and you can check this by:
$ heroku buildpacks
=== ginomempin-ffmpeg-app Buildpack URL
heroku/python
To add other dependencies (ffmpeg), you need to install the buildpack for it on your Heroku app (ex. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest). From the Heroku docs on Adding a buildpack, this is done by heroku buildpacks:add <buildpack>:
$ heroku buildpacks:add --index 2 https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
Buildpack added. Next release on ginomempin-ffmpeg-app will use:
1. heroku/python
2. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
Run git push heroku master to create a new release using these buildpacks.
$ heroku buildpacks
=== ginomempin-ffmpeg-app Buildpack URLs
1. heroku/python
2. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
Note the --index 2 there in my example. This is just to order the buildpacks, Python first since it's the main buildpack, then ffmpeg second. It depends on your app.
Now, test it by making changes to your code then deploy (i.e. git push heroku master). The Heroku logs should display that the buildpack is now added:
remote: -----> Python app detected
remote: -----> Installing requirements with pip
remote:
remote: -----> ffmpeg app detected
remote: -----> Install ffmpeg
remote: DOWNLOAD_URL = https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz
remote: exporting PATH
Step 2: Checking the ffmpeg buildpack
Use the heroku run command to check how to use the installed ffmpeg. For this sample app, I pushed a assets/sample.mp4 test file on the root directory of my app.
├── app.py
├── assets
│ └── sample.mp4
├── ...
└── runtime.txt
$ heroku run "which ffmpeg"
Running which ffmpeg on ⬢ ginomempin-ffmpeg-app... up, run.7460 (Free)
/app/vendor/ffmpeg/ffmpeg
$ heroku run "ffmpeg -i assets/sample.mp4 -vn -f mp3 assets/sample.mp3"
...
Output #0, mp3, to 'assets/sample.mp3':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
...
Once you now know how to run your ffmpeg commands (and that it works), all you have to do is call the same set of commands from your app. Note that, you don't need to change directories (as you did with your initial code) or specify the path to ffmpeg.
Step 3: Calling ffmpeg from Python app
Using Python's subprocess to call the same commands:
cmd = ['ffmpeg', '-i', './assets/sample.mp4', '-vn', '-f', 'mp3', './assets/sample.mp3']
out = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(out.stdout)
print(out.stderr)
for f in os.listdir("./assets"):
print(f)
Make sure that you separate all the space-separated parts of the command into a separate element in the list. You can then check the output using heroku logs --tail (for some reason, the ffmpeg output is stored in stderr instead of stdout):
2019-09-29T11:54:57.050692+00:00 app[web.1]: b''
2019-09-29T11:54:57.050736+00:00 app[web.1]: b"ffmpeg version N-50091-gfc20ba9e04-static https://johnvansickle.com/ffmpeg/
...
Output #0, mp3, to './assets/sample.mp3':\n
Metadata:\n
major_brand : isom\n
minor_version : 512\n
compatible_brands: isomiso2avc1mp41\n
TSSE : Lavf58.33.100\n
...
2019-09-29T11:54:57.050809+00:00 app[web.1]: sample.mp4
2019-09-29T11:54:57.050815+00:00 app[web.1]: sample.mp3
You should be getting the same subprocess.run output as the output you get when you use heroku run.
Related
I know there are a number of these questions, but none seem to solve my issue.
I'm running ruby SASS via django-pipelines.
During deployment, pipelines compiles some SASS files through the command python manage.py collectstatic --noinput which is done by Heroku automatically.
Previously I had no problems deploying this. I've recently come back to the code and now when I deploy to Heroku, I receive the following errors (full error log can be found here):
File "/app/.heroku/python/lib/python2.7/site-packages/pipeline/compilers/__init__.py", line 126, in execute_command
"{0!r} exit code {1}\n{2}".format(argument_list, compiling.returncode, stderr))
pipeline.exceptions.CompilerError: ['/usr/bin/env', 'sass', '--load-path', '/app/app/static', '--load-path', '/app/app2/static', u'/app/staticfiles/app2/stylesheets/app2.scss', u'/app/staticfiles/app2/stylesheets/app2.css'] exit code 1
/tmp/build_8d2e04464970e15667c8f825fc387c8a/myapp-8b763976868799a3f147ac8cedbd82421efa7525/vendor/ruby-2.2.2/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require': cannot load such file -- bundler/setup (LoadError)
from /tmp/build_8d2e04464970e15667c8f825fc387c8a/myapp-8b763976868799a3f147ac8cedbd82421efa7525/vendor/ruby-2.2.2/lib/ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in `require'
from /tmp/build_8d2e04464970e15667c8f825fc387c8a/myapp-8b763976868799a3f147ac8cedbd82421efa7525/vendor/bundle/bin/sass:14:in `<main>'
! Error while running '$ python manage.py collectstatic --noinput'.
As you can see, I've deployed using Ruby 2.2.2 previously, but now bundler has been updated on Heroku from 1.9.7 to 1.11.2.
Sass is the same version (3.4.19).
Line 14 of bin/sass from the last successful deployment on heroku.
$ sass --version
Sass 3.4.19 (Selective Steve)
$ cat ./vendor/bundle/bin/sass | sed '14!d'
require 'bundler/setup'
$ bundle --version
Bundler version 1.9.7
Curiously, the contents of my local version of bin/sass look nothing like the one on Heroku.
$ cat ~/.rvm/gems/ruby-2.2.2#app/bin/sass | sed '14!d'
str = ARGV.first
Locally, I'm running Ruby 2.2.2 and have tried both bundler 1.10.6 and 1.11.2.
Here is my Gemfile:
source 'https://rubygems.org'
ruby '2.2.2'
gem 'sass', '3.4.19'
I had the same problem as yours and found that the only solution was to switch out the SASS Compiler with Compass Compiler (installed with the django-pipeline-compass package). See Andrew's answer here: https://stackoverflow.com/a/31420009/6080975
I am building a web application that uses OpenCV in its back-end. I have built the application on Ubuntu (and I tried it on Windows, too) and it works fine. Currently, I am trying to configure OpenCV to work on Heroku. As OpenCV is not possible to be loaded using pip, I read about using heroku buildpacks which provide customization for the server environment.
The following is my attempt to test two of OpenCV buildpacks:
I build simple web server with Flask that tries to import OpenCV:
#hello.py
import os
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
text = ''
try:
import cv2
text = 'success'
except:
text = 'fail'
pass
return text + ' to load openCV'
if __name__ == "__main__":
port = int(os.environ.get("PORT", 5000))
app.run(host='0.0.0.0', port=port)
The above code should return either success or fail in loading OpenCV.
Then I configured Heroku to use (heroku multi buildpack) by running the following command:
heroku buildpacks:set https://github.com/ddollar/heroku-buildpack-multi
In the .buildpacks file (that is required by multi buildpack) I put the https://github.com/heroku/heroku-buildpack-python and https://github.com/slobdell/heroku-buildpack-python-opencv-scipy buildpacks.
The first one is for compiling a python application and for installing other modules (e.g., Flask) through pip. The second buildpack is the one that is supposed to load OpenCV.
After all, the whole application did not work!
I got (Application Error) page in Heroku as following screenshot:
I tried to use other buildpack (https://github.com/diogojc/heroku-buildpack-python-opencv-scipy) but I got the same result.
My questions are:
What is wrong in the steps I did?
How should I call (or use) OpenCV within my application in heroku?Should I use import statement or some other commands?
I could install by doing as follows:
cd /path/to/your/dir && git init
heroku create MYAPP (start from scratch)
heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git --app MYAPP
create .buildpacks as follows:
https://github.com/heroku/heroku-buildpack-python
https://github.com/diogojc/heroku-buildpack-python-opencv-scipy#cedar14
git add . && git commit -m 'MESSAGE' && git push heroku master
For anyone seeing this today and having the same issue, switch opencv-python in your requirements.txt to opencv-python-headless. This sidesteps the problem with the problematic library file.
The following steps should solve the problem of openCV which you are facing -
Add the heroku-buildpack-apt to the BuildPack by pasting - https://github.com/heroku/heroku-buildpack-apt to add buildpack in dasboard.
ScreenShot -
Adding through Dashboard -> Settings -> Add BuildPacks
Then add the Aptfile in your Github base directory which contains -
libsm6
libxrender1
libfontconfig1
libice6
- one library in each line. See Example Github Link
Now build and deploy and you are ready to go!
I am trying to deploy my first example app with Django/Heroku using the Django/Heroku Getting Started Tutorial.
My tools: Python 3.4 and Windows 7 PowerShell.
My challenge: deploying to Heroku fails and I am not sure why. Upon my first "git push" I saw that python-2.7.0 was used by default. I then added a runtime.txt (python-3.4.0) file in the app root.
Here is what happens when I run git push heroku master
-----> Python app detected
-----> Preparing Python runtime (python-3.4.0)
-----> Installing Setuptools (2.1)
-----> Installing Pip (1.5.4)
-----> Installing dependencies using Pip (1.5.4)
Exception:
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/commands/install.py", line 262, in run
for req in parse_requirements(filename, finder=finder, options=options, session=session):
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/req.py", line 1546, in parse_requirements
session=session,
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/download.py", line 275, in get_file_content
content = f.read()
File "/app/.heroku/python/lib/python3.4/codecs.py", line 313, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
Storing debug log for failure in /app/.pip/pip.log
! Push rejected, failed to compile Python app
Here the content of my requirements.txt file (created with pip freeze > requirements.txt)
Django==1.6.2
dj-database-url==0.3.0
dj-static==0.0.5
django-toolbelt==0.0.1
gunicorn==18.0
psycopg2==2.5.2
pystache==0.5.3
static==1.0.2
Here my Procfile (btw: gunicorn seems to be a Unix "only" command and does not work for Windows; read here):
web: gunicorn mytodo.wsgi
The Heroku tutorial does not mention a setup.py file, but it seems that one is necessary, so I simply copied a template.... not my preferred solution, but I did not know what else to do.
setup(
name='mysite',
version='0.1.0',
install_requires=[], # Don't put anything here, just use requirements.txt
packages=['mysite'],
package_dir={'mysite': 'src/mysite'},
)
What could be going on:
- The unicode error message could stem from the Procfile. Somewhere online I read that it has to be ASCII file, but I am not sure how to declare that as the Procfile has no file ending.
- The setup.py file is wrong.
Any help is appreciated. Thanks!
I encountered this exact problem during my own attempt to deploy a Django app to Heroku on Windows 7. The cause turned out to be this: The command pip freeze >requirements.txt encodes the file in UTF-16 format. Heroku expects requirements.txt to be ansi-encoded.
To fix it, I opened requirements.txt in Notepad, went to File->Save As, and set the Encoding to ANSI before saving again. After git-committing the new requirements.txt, I was able to run git push heroku master and it worked as expected.
Try removing static==1.0.2 from requirements.txt. It doesn't play nice with python 3.4. However, it will be installed properly through dj-static. This worked for me:
Django==1.5.1
dj-database-url==0.2.2
dj-static==0.0.5
gunicorn==18.0
psycopg2==2.5.1
I'm no expert, but take a look at this blog post about deploying to Heroku from a Windows machine. Hope it helps. http://www.swarley.me.uk/blog/2014/02/24/create-a-django-development-environment-on-64-bit-windows-for-heroku-deployment/
update: ok, I think I have a better answer. First, Heroku flat out says Windows users will have problems that Linux and iOS users will not: https://devcenter.heroku.com/articles/bundler-windows-gemfile The article is about Ruby but the same problems will apply to other languages because they are talking about the OS you are coming from.
However, this solution worked for me: Use Bitbucket as your remote repository, and upload to it from your Windows machine. Then from Bitbucket upload to Heroku. Here is a very similar question and answer here on SO: Deploying to Heroku using git on bitbucket
I would like to add OCR capabilities to my Django app running on Heroku. I suspect the easiest way is by using Tesseract. I've noticed that there are a number of python wrappers for Tesseract's API, but what is the best way to get Tesseract installed and running on Heroku? Via a custom buildpack like heroku-buildpack-tesseract maybe?
I'll try to capture some notes on the solution I arrived at here.
My .buildpacks file:
https://github.com/heroku/heroku-buildpack-python
https://github.com/clearideas/heroku-buildpack-ghostscript
https://github.com/marcolinux/heroku-buildpack-libraries
My .buildpacks_bin_download file:
tesseract-ocr https://s3.amazonaws.com/tesseract-ocr/heroku/tesseract-ocr-3.02.02.tar.gz 3.02 eng,spa
Here is the key piece of python that does the OCRing of pdf files:
# Additional processing
document_path = Path(str(document.attachment_file))
if document_path.ext == '.pdf':
working_path = Path('temp', document.directory)
working_path.mkdir(parents=True)
input_path = Path(working_path, name)
input_path.write_file(document.attachment_file.read(), 'w')
rb = ReadBot()
args = [
'VBEZ',
# '-sDEVICE=tiffg4',
'-sDEVICE=pnggray',
'-dNOPAUSE',
'-r600x600',
'-sOutputFile=' + str(working_path) + '/page-%00d.png',
str(input_path)
]
ghostscript.Ghostscript(*args)
image_paths = working_path.listdir(pattern='*.png')
txt = ''
for image_path in image_paths:
ocrtext = rb.interpret(str(image_path))
txt = txt + ocrtext
document.notes = txt
document.save()
working_path.rmtree()
Heroku, Django and tesseract
This doc will walk you through setting up tesseract on Heroku (i'm using django)
Steps
1) Add heroku-apt-buildpack using the command:
This is the stable version. See the source repository
$ heroku buildpacks:add --index 1 heroku-community/apt
2) Add Aptfile to project directory
`
$ touch Aptfile
3) Add the folowing to the Aptfile
tesseract-ocr-eng is the English language file for tesseract.
tesseract-ocr
tesseract-ocr-eng
4) Get path to the data downloaded by the tesseract-ocr-eng package
We will use this path for the next step
$ heroku run bash
$ find -iname tessdata # this will give us the path we need
You can exit heroku shell now exit
5) Now set a heroku config variable named TESSDATA_PREFIX to path
Set a heroku config variable named TESSDATA_PREFIX to the path returned from find -iname tessdata cmnd above
$ heroku config:set TESSDATA_PREFIX=./.apt/usr/share/tesseract-ocr/4.00/tessdata
Now set heroku set a heroku config variable named TESSDATA_PREFIX to the path returned from find -iname tessdata
6) Push changes to heroku
Set a heroku config variable named TESSDATA_PREFIX to the path returned from find -iname tessdata cmnd above
$ git push heroku master
I hope this helps. Let me know if it works for you.
I "finished" a little python project and I want to deploy it on heroku
GitHub page.
I want to execute: python2 main.py -i json-rpc
in order to have the json-rpc server listening for connections
but I get the following error when pushing to heroku:
$ git push heroku master Counting objects: 153, done. Delta
compression using up to 8 threads. Compressing objects: 100% (87/87),
done. Writing objects: 100% (153/153), 43.42 KiB, done. Total 153
(delta 61), reused 153 (delta 61)
-----> Heroku receiving push ! Heroku push rejected, no Cedar-supported app detected
To git#heroku.com:panager.git ! [remote rejected] master -> master
(pre-receive hook declined) error: failed to push some refs to
'git#heroku.com:panager.git'
What you might want to try doing is creating a Procfile. The full filename is Procfile, no extension, and it goes in the main directory of your project folder.
The content of that file would be:
web: python main.py -i json-rpc
Give that a shot and see if it works.
Alternatively, you may have forgotten to create a virtualenv for your app.
You should follow the instructions in Heroku's guide Getting Started with Python on Heroku
Update:
Having finally tested this myself on a fresh Heroku app, what you're missing is a requirements.txt. Even though you don't have any dependencies, you still need it. Within your virtualenv in the main project folder, run pip freeze > requirements.txt, and then git add . then git commit -m "added requirements.txt", and then push to Heroku and it should work.
Also, make sure that requirements.txt is saved with ANSI encoding, not Unicode or UTF-8! If you're a total n00b like me you can simply open requirements.txt in Notepad, choose SAVE AS and change the "Encoding" from the drop down. I tried all of the recommendations above but my error was due to this simple encoding problem.