I "finished" a little python project and I want to deploy it on heroku
GitHub page.
I want to execute: python2 main.py -i json-rpc
in order to have the json-rpc server listening for connections
but I get the following error when pushing to heroku:
$ git push heroku master Counting objects: 153, done. Delta
compression using up to 8 threads. Compressing objects: 100% (87/87),
done. Writing objects: 100% (153/153), 43.42 KiB, done. Total 153
(delta 61), reused 153 (delta 61)
-----> Heroku receiving push ! Heroku push rejected, no Cedar-supported app detected
To git#heroku.com:panager.git ! [remote rejected] master -> master
(pre-receive hook declined) error: failed to push some refs to
'git#heroku.com:panager.git'
What you might want to try doing is creating a Procfile. The full filename is Procfile, no extension, and it goes in the main directory of your project folder.
The content of that file would be:
web: python main.py -i json-rpc
Give that a shot and see if it works.
Alternatively, you may have forgotten to create a virtualenv for your app.
You should follow the instructions in Heroku's guide Getting Started with Python on Heroku
Update:
Having finally tested this myself on a fresh Heroku app, what you're missing is a requirements.txt. Even though you don't have any dependencies, you still need it. Within your virtualenv in the main project folder, run pip freeze > requirements.txt, and then git add . then git commit -m "added requirements.txt", and then push to Heroku and it should work.
Also, make sure that requirements.txt is saved with ANSI encoding, not Unicode or UTF-8! If you're a total n00b like me you can simply open requirements.txt in Notepad, choose SAVE AS and change the "Encoding" from the drop down. I tried all of the recommendations above but my error was due to this simple encoding problem.
Related
***EDIT Issue resolved. Several things needed to be done to setup pre-commit in my environment prior to commit/push.
'git init'
'pre-commit install'
open new terminal, activate venv
'pre-commit run —all-files'
Hope this helps someone in the future.
Junior Web Developer making his first post!
I'm setting up a Python Django project using the Cookie-cutter template https://github.com/cookiecutter/cookiecutter-django
However, upon any commit to GitHub, the GitHub Actions test fails on linter at the Run pre-commit step. I receive the error message: "Error: The process '/opt/hostedtoolcache/Python/3.9.10/x64/bin/pre-commit' failed with exit code 1"
My research indicates this is a generic python linter failure, but it's not clear why it's failing. The slug repo doesn't have any issues and I've setup the project many times, on 3 different machines (2 Macs and 1 Windows), pushing to new repositories each time, slightly modifying the settings each time, not modifying any code after project initialization, sadly receiving the same results.
I'm completely stumped and my progress has come to a stop on this issue. Please note I don't have much experience with GitHub Actions. Part of using this project template is having these nice features work to support future development.
One thing I'm confused about is why python 3.9.10 is being used in the pre-commit step when command "python3 --version" on all my machines I've used to initialize the project is returning python 3.9.9. A simple search in the project for "3.9.10" returns no results.
Below are my relevant project build settings. I've attempted many different combinations, but this is the result I'd like to get to production. Everything builds and runs via Docker-compose locally without issues. Thanks massively for your help and guidance! Some seemingly non-relevant links removed in the code due to reputation limit.
https://github.com/TElphee01/django1
version: 0.1.0
open_source_license: 2 - BSD
timezone: EST
windows: n
use_pycharm: n
use_docker: y
postgresql_version: 14.1
js_task_runner: gulp
cloud_provider: AWS
Mail_service: Mailgun
Use_async:n
Use_drf: y
custom_bootstrap_compilation: n
use_compressor: y
use_celery: y
use_mailhog: y
use_sentry: y
Use_whitenoise: y
use_heroku: y
ci_tool: Github
Keep_local_envs_in_vcs: y
debug: n
Error message found at: https://github.com/TElphee01/django1/runs/5325567134?check_suite_focus=true under Run pre-commit step
Run pre-commit/action#v2.0.3
install pre-commit
/opt/hostedtoolcache/Python/3.9.10/x64/bin/pre-commit run --show-diff-on-failure --color=always --all-files
[INFO] Initializing environment for github.com/pre-commit/pre-commit-hooks.
[INFO] Initializing environment for github.com/psf/black.
[INFO] Initializing environment for github.com/PyCQA/isort.
[INFO] Initializing environment for github.com/PyCQA/flake8.
[INFO] Initializing environment for github.com/PyCQA/flake8:flake8-isort.
[INFO] Installing environment for github.com/pre-commit/pre-commit-hooks.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for github.com/psf/black.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for github.com/PyCQA/isort.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
[INFO] Installing environment for github.com/PyCQA/flake8.
[INFO] Once installed this environment will be reused.
[INFO] This may take a few minutes...
trim trailing whitespace.................................................Passed
fix end of files.........................................................Failed
- hook id: end-of-file-fixer
- exit code: 1
- files were modified by this hook
Fixing README.md
check yaml...............................................................Passed
black....................................................................Passed
isort....................................................................Failed
- hook id: isort
- files were modified by this hook
Fixing /home/runner/work/django1/django1/tefrontend/users/tests/test_views.py
flake8...................................................................Passed
pre-commit hook(s) made changes.
If you are seeing this message in CI, reproduce locally with: `pre-commit run --all-files`.
To run `pre-commit` as part of git workflow, use `pre-commit install`.
All changes made by hooks:
diff --git a/README.md b/README.md
index a64ec04..c4c8a44 100644
Binary files a/README.md and b/README.md differ
diff --git a/tefrontend/users/tests/test_views.py b/tefrontend/users/tests/test_views.py
index ebdc864..4fe526a 100644
--- a/tefrontend/users/tests/test_views.py
+++ b/tefrontend/users/tests/test_views.py
## -11,11 +11,7 ## from django.urls import reverse
from tefrontend.users.forms import UserAdminChangeForm
from tefrontend.users.models import User
from tefrontend.users.tests.factories import UserFactory
-from tefrontend.users.views import (
- UserRedirectView,
- UserUpdateView,
- user_detail_view,
-)
+from tefrontend.users.views import UserRedirectView, UserUpdateView, user_detail_view
pytestmark = pytest.mark.django_db
Error: The process '/opt/hostedtoolcache/Python/3.9.10/x64/bin/pre-commit' failed with exit code 1
It looks like the pre-commit checks have detected some specific issues in your project:
the "end of file" check, which ensures that a file is either empty, or ends with one newline, and
the "isort" check, which has to do with the ordering and formatting of import statements.
In both cases, the tooling is also correcting the problem.
If you were to install and run pre-commit locally, you could just commit the changes it makes and then push your code when the commits pass the checks locally.
If you install pre-commit and then run pre-commit install in your repository, it will be installed as a pre-commit hook so these checks will be run for you automatically when you commit changes.
I have been trying to push changes to my GitHub repo but been facing this problem:
Uploading LFS objects: 100% (164/164), 153 MB | 0 B/s, done.
Enumerating objects: 25977, done.
Counting objects: 100% (25968/25968), done.
Delta compression using up to 4 threads
Compressing objects: 100% (18133/18133), done.
Writing objects: 100% (25955/25955), 146.83 MiB | 232.00 KiB/s, done.
Total 25955 (delta 6561), reused 25816 (delta 6463)
remote: Resolving deltas: 100% (6561/6561), completed with 4 local objects.
remote: error: GH001: Large files detected. You may want to try Git Large File Storage - https://git-lfs.github.com.
remote: error: Trace: 9f797e9b5f7a1c01fca6706ad62e21ba
remote: error: See http://git.io/iEPt8g for more information.
remote: error: File venv/lib/python3.6/site-packages/tensorflow_core/python/_pywrap_tensorflow_internal.so is 225.31 MB; this exceeds GitHub's file size limit of 100.00 MB
To https://github.com/yovelcohen/nba-stats.git
! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'https://github.com/yovelcohen/nba-stats.git'
(base) yovel#the-beastpad:~$ cd venv/lib/python3.6/site-packages/tensorflow_core/python/
bash: cd: venv/lib/python3.6/site-packages/tensorflow_core/python/_pywrap_tensorflow_internal.so: No such file or directory
I imported TensorFlow to the project but then removed it, when trying to delete it from the terminal it's not there (and when looking for it manually).
any help would be appreactied!
Since you should not push any generated binary file anyway, start first by removing them locally, commit and push:
git rm -r '*.so' -f
echo '*.so' >> .gitignore
But if that still fails, it means you are trying to push past commits (not just the latest one) with *.so files, even though the last commit no longer includes any such files.
If that is the case, since git filter-branch is soon deprecated, consider newren/git-filter-repo to remove any large files like those *.so ones, before pushing (or force pushing of that rewrites the content of past commits previously pushed).
I want to use the ffmpeg buildpack in my Python app on Heroku.
I am using the ffmpeg buildpack from https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.
How can I use buildpack? subprocess? os? How to call the ffmpeg?
Anybody can teach me?
This is my code and I want to convert mp4 file to mp3 file.
Actually,I don`t know about the detect/compile/release file.
subprocess.call(['ffmpeg', '-i', 'xxx.mp4','-vn','-f mp3', 'xxx.mp3'])
subprocess.call(['ffmpeg', '-i', 'xxx.mp4','-vn','-f mp3', 'xxx.mp3'])
First of, I assume you already know how to deploy a Python app to Heroku and you already have a working app accessible from Heroku, as this answer is specific to how to use the ffmpeg buildpack. (If you don't yet, check Getting Started on Heroku with Python first).
Step 1: Adding the ffmpeg buildpack
Buildpacks basically tell Heroku how to setup the environment for your app (which dependencies to install, which scripts to run, etc.). For Python apps, you need to have the official heroku/python buildpack, and you can check this by:
$ heroku buildpacks
=== ginomempin-ffmpeg-app Buildpack URL
heroku/python
To add other dependencies (ffmpeg), you need to install the buildpack for it on your Heroku app (ex. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest). From the Heroku docs on Adding a buildpack, this is done by heroku buildpacks:add <buildpack>:
$ heroku buildpacks:add --index 2 https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
Buildpack added. Next release on ginomempin-ffmpeg-app will use:
1. heroku/python
2. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
Run git push heroku master to create a new release using these buildpacks.
$ heroku buildpacks
=== ginomempin-ffmpeg-app Buildpack URLs
1. heroku/python
2. https://github.com/jonathanong/heroku-buildpack-ffmpeg-latest.git
Note the --index 2 there in my example. This is just to order the buildpacks, Python first since it's the main buildpack, then ffmpeg second. It depends on your app.
Now, test it by making changes to your code then deploy (i.e. git push heroku master). The Heroku logs should display that the buildpack is now added:
remote: -----> Python app detected
remote: -----> Installing requirements with pip
remote:
remote: -----> ffmpeg app detected
remote: -----> Install ffmpeg
remote: DOWNLOAD_URL = https://johnvansickle.com/ffmpeg/builds/ffmpeg-git-amd64-static.tar.xz
remote: exporting PATH
Step 2: Checking the ffmpeg buildpack
Use the heroku run command to check how to use the installed ffmpeg. For this sample app, I pushed a assets/sample.mp4 test file on the root directory of my app.
├── app.py
├── assets
│ └── sample.mp4
├── ...
└── runtime.txt
$ heroku run "which ffmpeg"
Running which ffmpeg on ⬢ ginomempin-ffmpeg-app... up, run.7460 (Free)
/app/vendor/ffmpeg/ffmpeg
$ heroku run "ffmpeg -i assets/sample.mp4 -vn -f mp3 assets/sample.mp3"
...
Output #0, mp3, to 'assets/sample.mp3':
Metadata:
major_brand : isom
minor_version : 512
compatible_brands: isomiso2avc1mp41
...
Once you now know how to run your ffmpeg commands (and that it works), all you have to do is call the same set of commands from your app. Note that, you don't need to change directories (as you did with your initial code) or specify the path to ffmpeg.
Step 3: Calling ffmpeg from Python app
Using Python's subprocess to call the same commands:
cmd = ['ffmpeg', '-i', './assets/sample.mp4', '-vn', '-f', 'mp3', './assets/sample.mp3']
out = subprocess.run(cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(out.stdout)
print(out.stderr)
for f in os.listdir("./assets"):
print(f)
Make sure that you separate all the space-separated parts of the command into a separate element in the list. You can then check the output using heroku logs --tail (for some reason, the ffmpeg output is stored in stderr instead of stdout):
2019-09-29T11:54:57.050692+00:00 app[web.1]: b''
2019-09-29T11:54:57.050736+00:00 app[web.1]: b"ffmpeg version N-50091-gfc20ba9e04-static https://johnvansickle.com/ffmpeg/
...
Output #0, mp3, to './assets/sample.mp3':\n
Metadata:\n
major_brand : isom\n
minor_version : 512\n
compatible_brands: isomiso2avc1mp41\n
TSSE : Lavf58.33.100\n
...
2019-09-29T11:54:57.050809+00:00 app[web.1]: sample.mp4
2019-09-29T11:54:57.050815+00:00 app[web.1]: sample.mp3
You should be getting the same subprocess.run output as the output you get when you use heroku run.
I am building a web application that uses OpenCV in its back-end. I have built the application on Ubuntu (and I tried it on Windows, too) and it works fine. Currently, I am trying to configure OpenCV to work on Heroku. As OpenCV is not possible to be loaded using pip, I read about using heroku buildpacks which provide customization for the server environment.
The following is my attempt to test two of OpenCV buildpacks:
I build simple web server with Flask that tries to import OpenCV:
#hello.py
import os
from flask import Flask
app = Flask(__name__)
#app.route("/")
def hello():
text = ''
try:
import cv2
text = 'success'
except:
text = 'fail'
pass
return text + ' to load openCV'
if __name__ == "__main__":
port = int(os.environ.get("PORT", 5000))
app.run(host='0.0.0.0', port=port)
The above code should return either success or fail in loading OpenCV.
Then I configured Heroku to use (heroku multi buildpack) by running the following command:
heroku buildpacks:set https://github.com/ddollar/heroku-buildpack-multi
In the .buildpacks file (that is required by multi buildpack) I put the https://github.com/heroku/heroku-buildpack-python and https://github.com/slobdell/heroku-buildpack-python-opencv-scipy buildpacks.
The first one is for compiling a python application and for installing other modules (e.g., Flask) through pip. The second buildpack is the one that is supposed to load OpenCV.
After all, the whole application did not work!
I got (Application Error) page in Heroku as following screenshot:
I tried to use other buildpack (https://github.com/diogojc/heroku-buildpack-python-opencv-scipy) but I got the same result.
My questions are:
What is wrong in the steps I did?
How should I call (or use) OpenCV within my application in heroku?Should I use import statement or some other commands?
I could install by doing as follows:
cd /path/to/your/dir && git init
heroku create MYAPP (start from scratch)
heroku config:add BUILDPACK_URL=https://github.com/ddollar/heroku-buildpack-multi.git --app MYAPP
create .buildpacks as follows:
https://github.com/heroku/heroku-buildpack-python
https://github.com/diogojc/heroku-buildpack-python-opencv-scipy#cedar14
git add . && git commit -m 'MESSAGE' && git push heroku master
For anyone seeing this today and having the same issue, switch opencv-python in your requirements.txt to opencv-python-headless. This sidesteps the problem with the problematic library file.
The following steps should solve the problem of openCV which you are facing -
Add the heroku-buildpack-apt to the BuildPack by pasting - https://github.com/heroku/heroku-buildpack-apt to add buildpack in dasboard.
ScreenShot -
Adding through Dashboard -> Settings -> Add BuildPacks
Then add the Aptfile in your Github base directory which contains -
libsm6
libxrender1
libfontconfig1
libice6
- one library in each line. See Example Github Link
Now build and deploy and you are ready to go!
I am trying to deploy my first example app with Django/Heroku using the Django/Heroku Getting Started Tutorial.
My tools: Python 3.4 and Windows 7 PowerShell.
My challenge: deploying to Heroku fails and I am not sure why. Upon my first "git push" I saw that python-2.7.0 was used by default. I then added a runtime.txt (python-3.4.0) file in the app root.
Here is what happens when I run git push heroku master
-----> Python app detected
-----> Preparing Python runtime (python-3.4.0)
-----> Installing Setuptools (2.1)
-----> Installing Pip (1.5.4)
-----> Installing dependencies using Pip (1.5.4)
Exception:
Traceback (most recent call last):
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/commands/install.py", line 262, in run
for req in parse_requirements(filename, finder=finder, options=options, session=session):
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/req.py", line 1546, in parse_requirements
session=session,
File "/app/.heroku/python/lib/python3.4/site-packages/pip-1.5.4-py3.4.egg/pip/download.py", line 275, in get_file_content
content = f.read()
File "/app/.heroku/python/lib/python3.4/codecs.py", line 313, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xff in position 0: invalid start byte
Storing debug log for failure in /app/.pip/pip.log
! Push rejected, failed to compile Python app
Here the content of my requirements.txt file (created with pip freeze > requirements.txt)
Django==1.6.2
dj-database-url==0.3.0
dj-static==0.0.5
django-toolbelt==0.0.1
gunicorn==18.0
psycopg2==2.5.2
pystache==0.5.3
static==1.0.2
Here my Procfile (btw: gunicorn seems to be a Unix "only" command and does not work for Windows; read here):
web: gunicorn mytodo.wsgi
The Heroku tutorial does not mention a setup.py file, but it seems that one is necessary, so I simply copied a template.... not my preferred solution, but I did not know what else to do.
setup(
name='mysite',
version='0.1.0',
install_requires=[], # Don't put anything here, just use requirements.txt
packages=['mysite'],
package_dir={'mysite': 'src/mysite'},
)
What could be going on:
- The unicode error message could stem from the Procfile. Somewhere online I read that it has to be ASCII file, but I am not sure how to declare that as the Procfile has no file ending.
- The setup.py file is wrong.
Any help is appreciated. Thanks!
I encountered this exact problem during my own attempt to deploy a Django app to Heroku on Windows 7. The cause turned out to be this: The command pip freeze >requirements.txt encodes the file in UTF-16 format. Heroku expects requirements.txt to be ansi-encoded.
To fix it, I opened requirements.txt in Notepad, went to File->Save As, and set the Encoding to ANSI before saving again. After git-committing the new requirements.txt, I was able to run git push heroku master and it worked as expected.
Try removing static==1.0.2 from requirements.txt. It doesn't play nice with python 3.4. However, it will be installed properly through dj-static. This worked for me:
Django==1.5.1
dj-database-url==0.2.2
dj-static==0.0.5
gunicorn==18.0
psycopg2==2.5.1
I'm no expert, but take a look at this blog post about deploying to Heroku from a Windows machine. Hope it helps. http://www.swarley.me.uk/blog/2014/02/24/create-a-django-development-environment-on-64-bit-windows-for-heroku-deployment/
update: ok, I think I have a better answer. First, Heroku flat out says Windows users will have problems that Linux and iOS users will not: https://devcenter.heroku.com/articles/bundler-windows-gemfile The article is about Ruby but the same problems will apply to other languages because they are talking about the OS you are coming from.
However, this solution worked for me: Use Bitbucket as your remote repository, and upload to it from your Windows machine. Then from Bitbucket upload to Heroku. Here is a very similar question and answer here on SO: Deploying to Heroku using git on bitbucket