Develop Reference

Import python file to our unit test cases with popen variable

Objective : To Create UnitTestCases for main.py
How can we import another python file which will dynamically return result
File : main.py
import os
_ENV = os.popen("echo ${RPM_ENVIRONMENT}").read().split('\n')[0]
_HOST = os.popen("echo $(hostname)").read().split('\n')[0]
_JOB_TYPE=os.popen("echo ${JOB_TYPE}").read().split('\n')[0]
SERVER_URL = {
'DEV':{'ENV_URL':'https://dev.net'},
'UAT':{'ENV_URL':'https://uat.net'},
'PROD':{'ENV_URL':'https://prod.net'}
}[_ENV]
Import another python file to our testing script
when i import main.py , i will receive error on SERVER_URL = { KeyError '${RPM_ENVIRONMENT}'
I believe the only reason why its returning error is because it does not recognize RPM_ENVIRONMENT, how can we mock _ENV variables in our main file and print server url as https://dev.net
After i have succesfully import main.py in my test case file, then i will create my unitTest cases as the ENV variable is required for me.
Testing : test_main.py
import unittest, sys
from unittest import mock
# --> Error
sys.path.insert(1, 'C:/home/src')
import main.py as conf
# For an example : we should be able to print https://dev.net when we include this in our line
URL = conf.SERVER_URL.get('ENV_URL')
ENV = conf._ENV
#URL should return https://dev.net & ENV should return DEV
class test_tbrp_case(unittest.TestCase):
def test_port(self):
#function yet to be created
pass
if __name__=='__main__':
unittest.main()

There's little reason to shell out of Python. You can read an environment variable with os.environ. os.environ['RPM_ENVIRONMENT'].
import os
_ENV = os.environ['RPM_ENVIRONMENT']
SERVER_URL = {
'DEV':{'ENV_URL':'https://dev.net'},
'UAT':{'ENV_URL':'https://uat.net'},
'PROD':{'ENV_URL':'https://prod.net'}
}[_ENV]
Now your test can set RPM_ENVIRONMENT before importing main.py.
os.environ['RPM_ENVIRONMENT'] = 'UAT'
sys.path.insert(1, 'C:/home/src')
import main.py as conf

How to reference create_app from different sub-domain?

The relevant part of my app structure looks like the following:
hp/
|
|---app/
|---admin/
|---auth/
|---errors/
|---main/
|---__init__.py
|---email.py
|---models.py
|---search.py
|---config.py
|---quiz.py
I want to create a scripts/ domain in either hp/ or app/. In those scripts I need to be able to reference config values. I'm using dotenv to do that. In order to use dotenv, I need app to be available so that I can call app.config['CONFIG_NAME'].
Here's more or less what I'm trying to do:
import requests
from app import app
access_key = app.config['ACCESS_KEY']
secret_key = app.config['SECRET_KEY']
data = requests.get(f'https://api.foo.com/search?client_id={access_key}&page=1&query=foo').json()
If I try from app import app, as I have above, I get a ModuleNotFoundError: No module named 'app' error. If I try from .. import app I get a ValueError: attempted relative import beyond top-level package error.
Any guidance/advice is greatly appreciated!

I ended up solving this by changing the sys.path.
I added the following:
import sys
import os
import inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
appdir = os.path.dirname(currentdir)
hpdir = os.path.dirname(appdir)
sys.path.insert(0, hpdir)
from app import create_app
app = create_app()
After that I was able to successfully call app.config

How to get config params in another module in flask python?

In my main.py have the below code:
app.config.from_object('config.DevelopmentConfig')
In another module I used import main and then used main.app.config['KEY'] to get a parameter, but Python interpreter says that it couldn't load the module in main.py because of the import part. How can I access config parameters in another module in Flask?

Your structure is not really clear but by what I can get, import your configuration object and just pass it to app.config.from_object():
from flask import Flask
from <path_to_config_module>.config import DevelopmentConfig
app = Flask('Project')
app.config.from_object(DevelopmentConfig)
if __name__ == "__main__":
application.run(host="0.0.0.0")
if your your config module is in the same directory where your application module is, you can just use :
from .config import DevelopmentConfig

The solution was to put app initialization in another file (e.g: myapp_init_file.py) in the root:
from flask import Flask
app = Flask(__name__)
# Change this on production environment to: config.ProductionConfig
app.config.from_object('config.DevelopmentConfig')
Now to access config parameters I just need to import this module in different files:
from myapp_init_file import app
Now I have access to my config parameters as below:
app.config['url']
The problem was that I had an import loop an could not run my python app. With this solution everything works like a charm. ;-)

Python 2.7.3 Flask ImportError: No module named

I have a Flask application with this error:
ImportError: No module named annotaria, referer: http://ltw1413.web.cs.unibo.it/
So, my root in web server is:
/home/web/ltw1413/html
Inside html folder I have:
One folder named "annotaria
One file .wsgi named "wsgi.wsgi"
My file .wsgi is:
import sys
sys.path.insert(0, '/home/web/ltw1413/html')
from annotaria import app as application
Inside my folder "annotaria" I have:
"Static" folder : inside stylesheet and js
"Templates" folder : inside html
"run.py" : file python where I have my application
run.py is this:
from pydoc import html
from annotaria import app
from flask import Flask, render_template, redirect, request
import json
import urllib2
import urlparse
import re
import string
import os
from SPARQLWrapper import SPARQLWrapper, JSON
from rdflib import Graph, BNode, Literal, Namespace
from time import strftime
import urllib2
import BeautifulSoup
app = Flask(__name__)
#app.route('/')
def index():
return render_template('index.html')
#app.route('/index.html')
def reloader():
return render_template('index.html')
# other app.route()...
if __name__ == '__main__':
app.run(debug=True)
How can I find a solution? Where is my error?

Lots of errors here...
annotaria is not found on path (that is why it says... well.. exactly that).
In your code you also redefine app: you import it from annotaria and then redefine it in app = Flask(...

Others mentioned individual errors, but it would be helpful to understand the big picture. First of all, let's assume the following structure:
/home/web/ltw1413/html
- wsgi.wsgi
- annotaria/
- __init.py__
- run.py
- static/
- templates
Like Klaus mentioned, you need __init__.py to tell Python that annotaria is a valid package. But then your wsgi.wsgi file needs to import the app from the run module:
from annotaria.run import app as application
You also need to remove this unnecessary import from run.py, since you instantiate a new application:
from annotaria import app
Because there is no application to import, you instantiate a new Flask application.
Finally, make sure that the app runs manually before you start deploying it.

Using environment dependent django settings inside fabric task

I'm trying to use the database configuration set on settings files to make a database dump using fabric.
There's more than one settings file, so I'd like to be able to do so based on the environment I choose.
by now, my task is like this
def dump_database():
with cd('~/project_folder'), prefix(WORKON_VIRTUALENV):
django.settings_module(env.settings)
from django.conf import settings
dbname = settings.DATABASES['default']['NAME']
dbuser = settings.DATABASES['default']['USER']
dbpassword = settings.DATABASES['default']['PASSWORD']
fname = '/tmp/{0}-backup-{1}.sql.gz'.format(
dbname,
time.strftime('%Y%m%d%H%M%S')
)
run('mysqldump -u %s -p=%s %s | gzip -9 /tmp/backup-%s.sql.gz' % (
dbuser,
dbpassword,
dbname,
fname))
But I'm getting an ImportError:
ImportError: Could not import settings 'project.settings.production'
I've tried to use shell_env() to set the DJANGO_SETTINGS_MODULE instead of django.settings_module(env.settings), with the same result.
I use a task to change the environment based on a environment dict:
def environment(name):
env.update(environments[name])
env.environment = name
This way, I want to be able to create a dump from multiple hosts like:
fab environment:live dump_database
fab environment:otherhost dump_database
Without having to reproduce database settings from all hosts on fabfile.

Importing your Django settings file in fabric is explained here.
http://fabric.readthedocs.org/en/1.3.3/api/contrib/django.html
Quoting from the above link:
from fabric.api import run
from fabric.contrib import django
django.settings_module('myproject.settings')
from django.conf import settings
def dump_production_database():
run('mysqldump -u %s -p=%s %s > /tmp/prod-db.sql' % (
settings.DATABASE_USER,
settings.DATABASE_PASSWORD,
settings.DATABASE_NAME
))

NOTE: I don't answer the question but offer a different solution
I had the same problem.. so I did custom .py script like that:
I created a file named dump_db.py (placed next to fabfile.py for example, that is on the remote machine)
import os
import sys
from datetime import datetime
from django.conf import settings
def dump_mysql():
os.environ.setdefault("DJANGO_SETTINGS_MODULE", SETTINGS_MODULE)
DB_NAME = settings.DATABASES['default']['NAME']
DB_USER = settings.DATABASES['default']['USER']
DB_PASSWORD = settings.DATABASES['default']['PASSWORD']
dump_file_name = '{time}_{db_name}.sql'.format(
time=datetime.now().strftime('%Y_%m_%d'),
db_name=DB_NAME,
)
os.system('mysqldump -u {db_user} -p{db_password} {db_name} > {to_file}'.format(
db_user=DB_USER,
db_password=DB_PASSWORD,
db_name=DB_NAME,
to_file=dump_file_name,
))
return dump_file_name
if __name__ == '__main__':
try:
SETTINGS_MODULE = sys.argv[1:].pop()
except IndexError:
SETTINGS_MODULE = 'project_name.settings'
print dump_mysql()
As you see sys.argv[1:].pop() tries to take optional argument (the setting module in this case).
So in my fabfile:
import os
from fabric.api import env, local, run, prefix, cd
.....
def dump():
current_dir = os.getcwd()
with prefix('source {}bin/activate'.format(env.venv)), cd('{}'.format(env.home)):
dumped_file = run('python dump_db.py {}'.format(env.environment)) # the optional argument given
file_path = os.path.join(env.home, dumped_file)
copy_to = os.path.join(current_dir, dumped_file)
scp(file_path, copy_to)
def scp(file_path, copy_to):
local('scp {}:{} {}'.format(env.host, file_path, copy_to))
where env.environment = 'project_name.settings.env_module'
And this is how I dump my DB and copy it back to me.
Hope it comes handy to someone! :)