Develop Reference

Could you explain me how environment variables work here?

I have a project which I developed locally on my computer. Now I want to deploy it to Heroku.
For the local version I use SQLite and for Heroku version I'm going to use PostgreSQL.
I found a code, which easily lets me combine two databases depending on where I "reach" the program: locally or on Heroku.
if on_heroku:
app.config['SQLALCHEMY_DATABASE_URI'] = os.environ['DATABASE_URL']
else:
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgres://localhost:5432/myblog'
Looks interesting but I don't get the whole idea behind it. Could someone explain to me how it works or recommend me what to read?
Thank you!

From my understanding, the os.environ['DATABASE_URL'] connects to the postgres db url of the postgres db add-on provided in your application. However, if your application does not have a heroku connection (assuming that is the conditional provided in on_heroku) then it accesses a local postgres db.

A cleaner way of writing it is:
app.config['SQLALCHEMY_DATABASE_URI'] = os.environ.get('DATABASE_URL', 'postgres://localhost:5432/myblog')
because the other solution requires you to write logic to detect whether you are on Heroku on_heroku.
Basically if the environment variable DATABASE_URL exists, take the value of that. If not use the default value postgres://localhost:5432/myblog

Flask SQLAlchemy initial database connection extremely slow

When I make an initial database connection with FlaskSQLAlchemy it is extremely slow, sometimes taking a minute or more to successfully connect and execute a query. Any subsequent database calls after the initial connection are fast. From my understanding this has something to do with "lazy loading". How can I "force" this initial connection to occur earlier, or speed up the connection altogether?
I am using MS SQL Server for my database, and pyodbc for my DB API. The application is deployed on Windows IIS server, although this problem occurs even while connecting locally - it is exacerbated while deployed on IIS.

Use latest version of PYODBC driver. First install python package.
pip install pyodbc
If you use following code, then you will get fast Response from MSSQL database for sure.
import urllib
params = urllib.parse.quote_plus(
'DRIVER={ODBC Driver 17 for SQL Server};SERVER=server-name;DATABASE=database-name;UID=sa;PWD=****;Trusted_Connection=yes;'
)
class Config:
DEBUG = True
SECRET_KEY = "E9738F40-4210ACAD94B"
SQLALCHEMY_DATABASE_URI = "mssql+pyodbc:///?odbc_connect=%s" % params
Make sure your ODBC driver is accurate in your system. If you want to get knowledge about ODBC driver, then run following query in cmd command.
import pyodbc
pyodbc.drivers()
Then put following code in your init.py file
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(Config)
Above things will be giving your ORM (Object Relation Mapper) which will gives you a technique of mapping object parameters to the underlying RDBMS table structure. An ORM API provides methods to perform CRUD operations without having to write raw SQL statements.

Connecting Heroku Database to SQLAlchemy

so as part of a Web Programming MOOC, I have to connect to a PostgreSQL Database hosted on Heroku using Python and SQLalchemy. I have spent a lot of hours trying to do this to no avail. My main problem is that I'm not being able to connect to the Database, because every time a try to run a script to update or just check the Database I get "Is the server running on host "ec2-174-129-35-61.compute-1.amazonaws.com" (174.129.35.61) and accepting
TCP/IP connections on port 5432?" in the command prompt.
I've done a lot of research trying to figure it out and it seems that maybe the URI of my Database is not set to accept public connections. But I tried setting ssl = require and still haven't been able to solve the problem. Maybe I didn't do it right?
I have come to think that maybe I am doing something wrong with the code, or that I should try to connect through Heroku's CLI and not the command prompt but I am not sure. This is my simple test code:
import os
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
if not os.getenv("DATABASE_URL"):
raise RuntimeError("DATABASE_URL is not set")
engine = create_engine(os.getenv("DATABASE_URL"))
db = scoped_session(sessionmaker(bind=engine))
db.execute("INSERT INTO books VALUES (1, 1, 1, 1)")
db.commit()
I set the DATABASE_URL environment variable in the command prompt to the Database's URI but maybe I did it wrong? I also tried just using the URI in the code itself but still could not make it work.
This is my first time using Databases with Heroku so I am complete neewbie, maybe I am doing something else wrong?. I appreciate all the help that you guys can give. Thanks in advance

1) Check, did you copied all link, including postgres://
2) Maybe you had made a mistake during setting DATABASE_URL. So, try to add your URL directly to the script:
import os
from sqlalchemy import create_engine
from sqlalchemy.orm import scoped_session, sessionmaker
engine = "postgres://ennjrjkrfsb____all_code_of_key_.....___compute.amazonaws.com:5432/ddo7541ka3hio7"
db = scoped_session(sessionmaker(bind=engine))
db.execute("INSERT INTO books VALUES (1, 1, 1, 1)")
db.commit()
3) if you are using linux, try to open your database, using command psql postgres://ennjrjkrfsb____all_code_of_key_.....___compute.amazonaws.com:5432/ddo7541ka3hio7
4) Try to access to your database in any way via other internet provider (may be your provider or sys admin in your office restricted access to Heroku)
Update: Try to use free online IDE with interpreter (based on Linux), for example, https://cs50.io - from this site you can access to Heroku postgres

postgres dialect does not support any more in SQLAlchemy v1.4.x above, and it turns into postgresql, so the final connection string must be postgresql://username:password#host:port/database
Here is my working connection from SQLAlchemy to Heroku Postgresql
database_url = os.environ('DATABASE_URL')
if database_url.startswith('postgres://'):
database_url.replace('postgres://', 'postgresql://')
engine = create_engine(database_url)
# rest of your codes...
here is my reference Heroku Issue

How to configure MongoEngine for Heroku

I have postgresql as my main database and I am using mongodb for 'audit-trail' purposes.
I am using mongoengine for my django app.
In localhost, in order to connect to mongodb database I did this:
connect('dbname')
and that is it, as documentation suggested.
But this is not working in heroku.
I added MongoLab add-on to my heroku application and they gave me this connections string:
mongodb://<dbuser>:<dbpassword>#ds037622.mongolab.com:37622/heroku_app30998840
Then I tried this:
connect('heroku_app30998840', username='username', password='pwd', host='mongodb://<dbuser>:<dbpassword>#ds037622.mongolab.com:37622/heroku_app30998840')
This did not work either.
How can I fix this?
Or maybe I should use something else other than MongoEngine? some other better way?

That is what worked for me:
connect('heroku_app30998840', username='username', password='pwd', host='#ds037622.mongolab.com:37622')
i.e remove everything except host and port number in host part.

Python database WITHOUT using Django (for Heroku)

To my surprise, I haven't found this question asked elsewhere. Short version, I'm writing an app that I plan to deploy to the cloud (probably using Heroku), which will do various web scraping and data collection. The reason it'll be in the cloud is so that I can have it be set to run on its own every day and pull the data to its database without my computer being on, as well as so the rest of the team can access the data.
I used to use AWS's SimpleDB and DynamoDB, but I found SDB's storage limitations to be to small and DDB's poor querying ability to be a problem, so I'm looking for a database system (SQL or NoSQL) that can store arbitrary-length values (and ideally arbitrary data structures) and that can be queried on any field.
I've found many database solutions for Heroku, such as ClearDB, but all of the information I've seen has shown how to set up Django to access the database. Since this is intended to be script and not a site, I'd really prefer not to dive into Django if I don't have to.
Is there any kind of database that I can hook up to in Heroku with Python without using Django?

You can get a database provided from Heroku without requiring your app to use Django. To do so:
heroku addons:add heroku-postgresql:dev
If you need a larger more dedicated database, you can examine the plans at Heroku Postgres
Within your requirements.txt you'll want to add:
psycopg2
Then you can connect/interact with it similar to the following:
import psycopg2
import os
import urlparse
urlparse.uses_netloc.append('postgres')
url = urlparse.urlparse(os.environ['DATABASE_URL'])
conn = psycopg2.connect("dbname=%s user=%s password=%s host=%s " % (url.path[1:], url.username, url.password, url.hostname))
cur = conn.cursor()
query = "SELECT ...."
cur.execute(query)

I'd use MongoDB. Heroku has support for it, so I think it will be really easy to start and scale out: https://addons.heroku.com/mongohq
About Python: MongoDB is a really easy database. The schema is flexible and fits really well with Python dictionaries. That's something really good.
You can use PyMongo
from pymongo import Connection
connection = Connection()
# Get your DB
db = connection.my_database
# Get your collection
cars = db.cars
# Create some objects
import datetime
car = {"brand": "Ford",
"model": "Mustang",
"date": datetime.datetime.utcnow()}
# Insert it
cars.insert(car)
Pretty simple, uh?
Hope it helps.
EDIT:
As Endophage mentioned, another good option for interfacing with Mongo is mongoengine. If you have lots of data to store, you should take a look at that.

I did this recently with Flask. (https://github.com/HexIce/flask-heroku-sqlalchemy).
There are a couple of gotchas:
1. If you don't use Django you may have to set up your database yourself by doing:
heroku addons:add shared-database
(Or whichever database you want to use, the others cost money.)
2. The database URL is stored in Heroku in the "DATABASE_URL" environment variable.
In python you can get it by doing.
dburl = os.environ['DATABASE_URL']
What you do to connect to the database from there is up to you, one option is SQLAlchemy.

Create a standalone Heroku Postgres database. http://postgres.heroku.com