I have a Pyramid webapp with a Postgres database, and I'm using git for version control.
This is how my files are structured:
myapp/
|----dotcloud.yml
|----env/ # virtualenv
|----MyProject/
|
|----production.ini
|----requirements.txt
|----myapp.sql
|----myapp.psql
|----wsgi.py
|----myproject
|
|----scripts/
|----static/
|----templates/
|----__init__.py
|----views.py
|----models.py
This is my dotcloud.yml:
www:
type: python
config:
python_version: v2.7
approot: home/home/myapp
db:
type: postgresql
This is my wsgi.py:
from pyramid.paster import get_app, setup_logging
ini.ath = '.../myproject/production.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')
This is my (simplified) __init__.py:
def main(global_config, **settings):
engine = engine_from_config(settings, 'sqlalchemy.')
DBSession.configure(bind=engine)
config = Configurator(...)
config.add_static_view('static', 'static', cache_max_age=3600)
config.add_route('index', '/')
# other routes...
config.scan()
return config.make_wsgi_app()
I've read the official documentation and the third-party documentation to get to this point but there must be something I'm not doing right. It's my first time deploying a webapp, and running my app locally still works.
In MyProject/ (where the dotcloud.yml file resides) I did dotcloud create mydomainname, dotcloud connect mydomainname and then dotcloud push. But I'm getting an internal server error. What am I doing wrong?
Also, the documentation says that if I'm using git, I must state that explicitly when I use dotcloud create or dotcloud connect, but what is the exact command?
EDIT: Before moving to DotCloud, I tried to use DigitalOcean, and had some problems when using pip to install the dependencies in requirements.txt. I had to do a git clone separately on the CLI so that I could enter my username and password, and I also had to install psycopg2 manually. Could this be one of the problems here too? If so, how can I fix it?
There are several things you should try changing. First, do not push your virtualenv directory (env). The dotCloud builder will create another virtualenv based on your requirements.txt. One way to avoid pushing your env directory would be to move dotcloud.yml to MyProject. You seem to think that is where it is ("In MyProject/ (where the dotcloud.yml file resides)" ) but that is not what your file tree says.
Then, do the dotcloud connect or dotcloud create in MyProject, as you stated.
You should remove the approot line from your dotcloud.yml. Allow approot to go to its default, the current directory.
Related
I am following the instructions here to deploy an app in Google App Engine. Everything works correctly.
Nevertheless, Google, by default, looks for the main folder (where app = Flask(__name__) is defined) in main.py. How could I redefine this? I would like to define this main folder as app.py.
Rename main.py to app.py
Add entrypoint: gunicorn -b :$PORT app:app to your app.yaml file. This is where you are telling Google to find the app object in a file called app
Add gunicorn to your requirements.txt file
Notes:
i. Because you're changing from main.py to app.py, you need to specify an entrypoint. GAE documentation says
If your app meets the following requirements, App Engine will start
your app with the gunicorn web server if you don't specify the
entrypoint field:
The root of your app directory contains a main.py file with a WSGI-compatible object called app.
Your app does not contain Pipfile or Pipfile.lock files.
ii. If you add an entrypoint, then you need to include gunicorn in your requirements.txt file
iii. I just tested the above configuration (the answer I gave) on a dev environment (Python 3.9 environment on Macbook using dev_appserver.py) and it works
I want to have different environment variables based on development and production
but i can't seem to find anything related to this topic for FastAPI.
Is it possible that i can have .env, .env.local, .env.prod to have different environment variables
I don't think you need multiple files. Usually how it's done is, have a single config file that has the default values, usually this is your "local" config file. For prod, staging and other environments, you can override these settings by setting environment variables, most hosts support it nowadays. It's more secure and you don't have to expose production secrets and keys in your repository.
This library is one example of what you code use:
https://github.com/theskumar/python-dotenv
EDIT
For example, if your application is hosted in Heroku, the heroku config commands of the Heroku CLI makes it easy to manage your app’s config vars:
heroku config:set SOME_CONFIG_I_NEED=value for production
You can also edit config vars from your app’s Settings tab in the Heroku Dashboard.
Please refer to the Heroku documentation for more information.
After you set the env vars in Heroku, this is how you would access them from your Python code (using python-dotenv):
First, install python-dotenv:
pip install python-dotenv
Now, create a file called .env in the root of your project with the following contents:
# Development settings
SOME_CONFIG_I_NEED=value for development
Now in your python file:
from dotenv import load_dotenv
load_dotenv() # take environment variables from .env.
SOME_CONFIG_I_NEED = os.environ.get("SOME_CONFIG_I_NEED")
print(SOME_CONFIG_I_NEED) # This will print "value for development" when running on local, and will print "value for production" when running in Heroku.
The .env file
ADMIN_EMAIL="deadpool#example.com"
APP_NAME="ChimichangApp"
Read settings from .env¶
And then update your config.py with:
from pydantic import BaseSettings
class Settings(BaseSettings):
app_name: str = "Awesome API"
admin_email: str
class Config:
env_file = ".env"
Here we create a class Config inside of your Pydantic Settings class, and set the env_file to the filename with the dotenv file we want to use.
Read variable
from config import Settings
app = FastAPI()
setting = Settings()
print(setting)
Learn more
An alternative approach could be to use the Pydantic Settings:
https://pydantic-docs.helpmanual.io/usage/settings/
There is also a bit about that in the FastAPI docs, but personally I choose not to 'integrate' the nice Pydantic Settings that way.
https://fastapi.tiangolo.com/advanced/settings/
Right now I have an app that write data for users to a database file called users.sql. However, I pushed this to heroku through github and the data doesn't stick. If a certain time has passed, heroku will sleep and all the files will have to be redeployed when another login is detected. I heard that I could add heroku postgres as an addon to heroku but I am not sure what to do to integrate this with my current code. What do I do to set it up?
I stumbled across your question while trying to connect my app to Postgres and getting very frustrated at the lack of clear instructions. So here's the step by step process with all the errors I struggled through (because that's where the tutorials leave you gasping for air while they thunder on obliviously...)
TL;DR
Install Postgres
Install the Heroku CLI
Install psycopg2 or psycopg2-binary and gunicorn
Create an app in Heroku with a Postgres database and save the database url as the SQLALCHEMY_DATABASE_URI
Create Procfile, requirements.txt and runtime.txt
Push to Github and connect repo to Heroku
Deploy branch
from app import db db.create_all()
I work with WSL so all command line syntax in my answer is Bash, and I'm working on Windows 10.
I'm working partly off a youtube tutorial by Traversy Media. Here's the Github repo he works with (I definitely recommend initially working with a basic app instead of struggling with the errors that crop up in a complex app with extra packages).
Before you start
Create an account in Heroku and install the Heroku CLI
Download and install PostgreSQL (this will also give you pgAdmin 4)
Fork and clone the Github repo, and check it runs ok (if you're not working with your own app)
Preliminaries
Create a virtual env and install all packages. Ensure you have psycopg2and gunicorn
ERROR: Failed building wheel for psycopg2-binary
For whatever reason, psycopg refuses to install on my system. You can get round this by using pip3 install psycopg2-binary instead - there's no difference as far as our requirements go.
Connect to Postgres
Open pgAdmin 4 from the start menu. This may prompt you for a password if it's the first time you're logging in or if you were logged out.
could not connect to server: Connection refused (0x0000274D/10061) Is the
server running on host "127.0.0.1" and accepting TCP/IP connections on port 5432?
Use sudo service postgresql status to check if the PostgreSQL server
is running. If it is, try stopping and starting it. If it isn't, start
it.
sudo service postgresql start
sudo service postgresql stop
In the left-hand panel, right-click on Databases and click Create > Database.... Enter your database name (the tutorial uses "lexus") and save.
In app.py (or wherever you do your database config) find the line
if ENV == 'dev':
...
app.config['SQLALCHEMY_DATABASE_URI'] = ''
and put the database connection string inside the quotes. The syntax is
postgresql://[username]:[password]#[host name]/[db name]
# [username] - Right-click on your server in pgAdmin (probably a variation of `PostgreSQL 13`). Go to the Connection tab and check the Username field.
# [password] - The root password you set the first time you logged in to pgAdmin.
# [host name] - In the same place as your username, in the Host name/address field.
# [db name] - The name you entered when creating your database.
In my case, the connection string is:
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://postgres:123456#localhost/lexus'
Important Disclaimer
This solution has you put the database password in code: app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://postgres:123456#localhost/lexus'
This saves the password in your version control history, where it can/will eventually allow anyone on the Internet to control your database. Yikes! Get DB user & password from env vars instead.
Thanks #Paul Cantrell!
Enter the python console and create the database:
>>> from app import db
>>> db.create_all()
If you didn't get an error on that, consider yourself the luckiest thing since sliced bread. In pgAdmin go to Databases > [your db name] > Schemas > Tables to check they've been created.
ImportError: cannot import name 'db' from 'app' (unknown location)
db is not located in app in your application. Find where it's
defined and import it from there.
(psycopg2.OperationalError) FATAL: password authentication failed for user "postgres"
You're using the wrong password. I found this confusing
because it's unclear whether you need to use the master password (the
one you set when logging in to pgAdmin for the first time) or the
password for the database you're connecting to. It's the master password that's needed here.
Enter psql -h 127.0.0.1 postgres (psql -h [ip address] [username]) in the command line - the password that authenticates is the one you need for the connection string.
(psycopg2.OperationalError) FATAL: database "lexus" does not exist
You might be connecting to the wrong database, using the wrong
username, or your database hasn't been created. Check your connection string and pgAdmin to confirm that details are correct.
db.create_all() seemed to work, but I have no tables in pgAdmin!
If your database models are stored in a separate file, import them
into app.py after db is initialised. It should look like this:
app = Flask(__name__)
...
db = SQLAlchemy(app)
...
from app.models import *
Send data to Postgres
Run the app and submit some data through the form (or create some data in your table). In pgAdmin, right-click the table you just created and refresh, then right-click and view/edit data.
Could not send data to server: Socket is not connected (0x00002749/10057) could not send SSL negotiation packet: Socket is not connected (0x00002749/10057)
Disconnect and reconnect server (right-click on server and click Disconnect Server, then Connect Server) and refresh table again.
If that didn't work, right-click on server and click Properties, go to the Connection tab and change the Host name/address from
localhost to 127.0.0.1. Then repeat step 1.
Create Heroku app
Since I prefer using Heroku directly with Github instead of the CLI, I'm not using git init here, but note that if your code is not already a Github repo you would need to use it.
In the command line:
heroku login # If you have the Heroku CLI installed correctly, this will open a tab in the browser and log you in to Heroku.
heroku create unique-app-name # (replace `unique-app-name` with your own name). This creates an app on the Heroku server, and it's the place your code is cloned to so that it can be run. Check that it has appeared in your [Heroku dashboard](https://dashboard.heroku.com/).
heroku addons:create heroku-postgresql:hobby-dev -a unique-app-name # This creates an empty postgres database for your app. Heroku uses its own database, not your pgAdmin database.
heroku config -a unique-app-name # The url you set earlier as the `SQLALCHEMY_DATABASE_URI` points to the database you created in pgAdmin, but since your app is now going to be using Heroku's database you need a new url. Copy the DATABASE_URL that is returned.
Go back to the code where you set app.config['SQLALCHEMY_DATABASE_URI'] earlier, and insert the new url inside the else block. It should look more or less like this:
if ENV == 'dev':
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://postgres:123456#localhost/lexus'
else:
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgres://urlyoujustgotfromherokuozftkndvkayeyc:691196bfb1b1ca8318b733935b10c97a19fd41#ec2-52-71-161-140.compute-1.amazonaws.com:5432/d3pofh2b55a2ct'
If you're working with the tutorial's github repo, set ENV = 'prod', otherwise set the config to use the heroku url however you want.
Add the following files to your root directory if they're not yet included in your project files:
# Procfile
web: gunicorn app:app # The first `app` refers to the app.py and the second `app` refers to the app object, ie `app = Flask(__name__)`. Heroku needs this to know how to run your code.
# runtime.txt
python-3.8.5 # Find your python version by entering `python --version` or `python3 --version` in the command line.
# requirements.txt
# Create this by running `pip3 freeze > requirements.txt`. According to the [Heroku docs](https://devcenter.heroku.com/articles/python-runtimes)
# this must be done after changing runtime versions, so it's best to do it last.
Push everything to Github (again, this is if you're already in a repo otherwise you'd be using the Heroku CLI process which I'm not following).
Deploy on Heroku
Click on the app you created previously in the Heroku dashboard and go to the Deploy tab. In the Deployment method options, click Github. Connect your Github account and repository.
Scroll down to the Manual deploy section and select the branch you want to deploy (or just leave it as master if that's the only one). Click Deploy Branch.
Requested runtime (python-3.7.2) is not available for this stack (heroku-20).
! Aborting. More info: https://devcenter.heroku.com/articles/python-support
! Push rejected, failed to compile Python app.
! Push failed
It looks like the python version is wrong. Check that runtime.txt is showing the right python version. If it
isn't, update it, delete and recreate requirements.txt, push to
Github and click Deploy Branch again.
Create Heroku database
Back in the command line:
heroku run python3
>>> from app import db
>>> db.create_all()
To check out your database, enter
heroku pg:psql --app unique-app-name
If you run select * from tablename; nothing will be returned because there isn't any data yet.
Click the View button in Heroku to open your app. Submit the form (or enter data however your app works), then run select * from tablename; again and you'll see the row of data.
In your init.py file you can set up your app.config['SQLALCHEMY_DATABASE_URI'] variable such that it will look for the one Heroku will give your application if it's running on heroku, OR use the configuration for your local database if it's running locally. You can achieve this as such:
app.config['SQLALCHEMY_DATABASE_URI'] = environ.get('DATABASE_URL') or "postgresql://postgres:password#localhost:5432/postgres"
replace what is after the OR w/ the connection details for your local connection
I would suggest considering setting up a local postgres database instead of sql if you're going to use heroku but you can technically use both. You just may run into minor issues, I have ran into issues w/ dates being handled slightly different, for example.
Firstly make heroku login from terminal:
$ heroku login
Now create the database for your app using following command:
$ heroku addons:create heroku-postgresql:hobby-dev --app app_name
Now after your Database has been created, get the URL of your Database using the following command:
$ heroku config --app app_name
Now that you have got your Database URL, replace the value of app.config[‘SQLALCHEMY_DATABASE_URI’] line in the “app.py” file with this Database URL:
app.config['SQLALCHEMY_DATABASE_URI'] = 'heroku_database_url'
And finally push code to heroku.
I'm trying to deploy a django web app to the Microsoft Azure and this is correctly deployed by the pipeline on DevOps Azure, but I get the error message (ModuleNotFoundError: No module named 'django) on portal Azure and cannot reach my app via the URL.
The app also works properly locally
Here is the whole error message: '''https://pastebin.com/mGHSS8kQ'''
How can I solve this error?
I understand you have tried the steps suggested in the SO thread Eyap shared, and few things here are already covers that. Kindly review these settings.
You can use this command instead - source /antenv3.6/bin/activate.
As a side note- The antenv will be available only after a deployment is initiated. Kindly check the “/” path from SSH and you should see a folder with name starting from antenv.
Browse to .python_packages/lib/python3.6/site-packages/ or .python_packages/lib/site-packages/. Kindly review the file path exists.
Review the Application logs as well (/home/LogFiles folder) from Kudu- https://<yourwebpp-name>.scm.azurewebsites.net/api/logs/docker
The App Service deployment engine automatically activates a virtual environment and runs
pip install -r requirements.txt
The requirements.txt file must be in the project root for dependencies to be installed.
For Django apps, App Service looks for a file named wsgi.py within your app code, and then runs Gunicorn using the following command:
is the name of the folder that contains wsgi.py
gunicorn --bind=0.0.0.0 --timeout 600 .wsgi
If you want more specific control over the startup command, use a custom startup command, replace with the name of folder that contains wsgi.py, and add a --chdir argument if that module is not in the project root.
For additional details, please checkout this document
Configure a Linux Python app for Azure App Service
Quickstart: Create a Python app in Azure App Service on Linux
I am trying to deploy my flask app. Usually I would have an app.py and put all code in it.
app.py
templates/
|
|--index.html
for my really small projects. But then I have a slightly larger app and follow the larger app guide by flask.
So I have this:
setup.py
app/
__init__.py
views.py
models.py
forms.py
templates/
| ---index.html
I now have all my routes and views in views.py and running the app in __init__.py:
from flask import Flask
app = Flask(__name__)
import app.views # Name in setup.py
if __name__ == "__main__":
app.run()
(This is just an example)
So now I follow the guide by running it with pip install -e . and running with:
>set FLASK_APP=app(name I set in setup.py) flask run and it works. Except I do not know how to run it with one command. Since there is no one file to run I can not use gunicorn or anything like that. I am not sure how to go about executing this app. How would I run pip install . on the cloud server heroku?
My problem is because I have to import the app from __init__.py and views using import blog.[insert import] (models, views etc.) Any help is appreciated. Thank you.
EDIT: I do not want to use blueprints though. That might be too much. My app is medium, not small but not large either
You absolutely can use Gunicorn to run this project. Gunicorn is not limited to a single file, it imports Python modules just the same as flask run can. Gunicorn just needs to know the module to import, an the WSGI object to call within that module.
When you use FLASK_APP, all that flask run does is look for module.app, module.application or instances of the Flask() class. It also supports a create_app() or make_app() app factory, but you are not using such a factory.
Gunicorn won't search, if you only give it a module, it'll expect the name application to be the WSGI callable. In your case, you are using app so all you have to do is explicitly tell it what name to use:
gunicorn app:app
The part before the : is the module to import (app in your case), the part after the colon is the callable object (also named app in your module).
If you have set FLASK_APP as a Heroku config var and want to re-use that, you can reference that on the command line for gunicorn:
gunicorn $FLASK_APP:app
As for heroku, it can handle requirement.txt or setup.py
c.f. https://devcenter.heroku.com/articles/python-pip#local-file-backed-distributions
If your Python application contains a setup.py file but excludes a requirements.txt file, python setup.py develop will be used to install your package and resolve your dependencies.
If you already have a requirements file, but would like to utilize this feature, you can add the following to your requirements file:
-e .
And about run command, i think you cat put Procfile like
web: FLASK_APP=app flask run
or
web: FLASK_APP=app python -m flask run