Develop Reference

How to run a specific function upon module selection in flask API?

I'm building a python visualization application. I'm using 2 modules for this:
LUX
Pandas Profiling
The above 2 are auto visualization libraries that take in a csv/xlsx file & give descriptive statistics of the data which includes visualizations.
LUX has a customizable function called 'Intent' which analyzes the relationship among specific variables upon selection.
The visualizations get stored in a HTML file on my local system.
What I have tried:
This is my Python code for the visualization:
# Importing libraries
import csv
from mimetypes import init
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
import os
import csv
import json
from pandas_profiling import ProfileReport
import pandas_profiling as pp
import lux
# # Pandas profiling
def pand_prof_fun(eda_file, savepath):
profile = ProfileReport(eda_file)
# Activating the widgets
profile.to_widgets()
profile.to_notebook_iframe()
# Saving & generating the report
pp.ProfileReport(eda_file)
profile.to_file(savepath)
profile = ProfileReport(eda_file, title="My Data Profile Report")
return profile
def lux_vis(eda_file, vis_file_path, vis_file_path_intent, intent_list):
eda_file.save_as_html(vis_file_path)
# Visualizing selected variables
eda_file.intent = intent_list
eda_file.save_as_html(vis_file_path_intent)
def run(csv_filepath, int_list):
csv_dataframe = pd.read_csv(csv_filepath)
os.makedirs(r'templates', exist_ok=True)
pand_prof_path = r'templates\pandas_prof.html'
lux_save_path = r'templates\lux_complete.html'
lux_intent_save_path = r'templates\lux_intent.html'
pand_prof_fun(csv_dataframe, pand_prof_path)
lux_vis(csv_dataframe, lux_save_path, lux_intent_save_path, int_list)
if __name__ == '__main__':
csv_filepath = r'C:\Users\projects\Churn_bank.csv'
I now want to build a Flask API such that the user can choose what they want to display, i.e., -->
Lux - displays the pairwise relationship between all variables. (or)
Lux using intent (Customization)- displays the relationship between specified variables. (or)
Pandas Profiling - displays the descriptive statistics of the entire data.
This is my API code for the 2nd module (Lux using intent):
from flask import Flask, render_template
import requests
from flask import Flask, request, jsonify
import json
import traceback
from visualization_function import run
from visualization_function import lux_vis
from visualization_function import pand_prof_fun
app = Flask(__name__)
#app.route('/visualization_with_us', methods = ['POST'])
def home1():
allowed_extensions = ['csv', 'xlsx']
file = request.json.get('filepath')
filename=file.split("/")[-1]
intents = request.json.get('intents')
file_extension = filename.split('.', 1)[1].lower()
try:
if file_extension in allowed_extensions:
run(file, intents)
return "Successful"
else:
return "File not supported"
except:
return "Exception in file"
if __name__ == '__main__':
app.run()
Entering input via 'Postman'
Entering the 'intents' via postman returns "successful" message & gives the right output. The HTML file gets created & saved in my "templates" folder. But when I try to include the other two modules in my API code - 'Module 1' & 'Module 3', my code doesn't work.
Here's how I've included 'Module 1' & 'Module 3' in my API code. My idea is to create a list of the modules & depending on the module chosen, the respective module/function runs -->
from flask import Flask
from flask import Flask, request
from importlib.util import module_from_spec
from flask import Flask, render_template
import requests
from flask import Flask, request, jsonify
import json
import traceback
from visualization_function import run
from visualization_function import lux_vis
from visualization_function import pand_prof_fun
app = Flask(__name__)
#app.route('/visualization_with_apoorva', methods = ['POST'])
def home1():
allowed_extensions = ['csv', 'xlsx']
available_modules=['pandas_profiling', 'lux_overall', 'lux_intent']
file = request.json.get('filepath')
filename=file.split("/")[-1]
file_extension = filename.split('.', 1)[1].lower()
intents = request.json.get('intents')
module_selection= request.json.get('module')
try:
if file_extension in allowed_extensions:
return lux_vis()
if module_selection=='lux_intent':
intents = request.json.get('intents')
run(file, intents)
if module_selection=='lux_overall':
run(file, lux_vis)
if module_selection=='pandas_profiling':
run(file, pand_prof_fun)
return "Successful"
else:
return "File not supported"
except:
return "Exception in file"
if __name__ == '__main__':
app.run()
Error: When I run the route on postman it returns "Exception in file" message.
Postman output
What is the error in my code? How do I include 'module 1' & 'module 3' in the same Flask API code such that the user can choose any one among the 3 modules & get the desired output?

Python Flask Gunicorn Configuration says "from flask_ckeditor import CKEditor, CKEditorField ImportError: No module named flask_ckeditor"

I am using python 3.7 environments, flask framework and with the latest ckeditor 5 and I actually run it smoothly without any error in localhost with this command: python script.py. But when it comes to the gunicorn configuration, gunicorn --bind 0.0.0.0:5000 wsgi:app I'm having this error: "from flask_ckeditor import CKEditor, CKEditorField
ImportError: No module named flask_ckeditor" I really don't know what to do, I hope somebody will help me.
Here is the code:
import os
from flask_sqlalchemy import SQLAlchemy
from sqlalchemy import create_engine, Column, Integer, String, Sequence
from sqlalchemy.ext.declarative import declarative_base
from flask import Flask
import hashlib
from flask_wtf import FlaskForm
from flask_wtf import Form
from wtforms import StringField, PasswordField, BooleanField
from wtforms.validators import InputRequired, Email, Length
from flask_bootstrap import Bootstrap
from flask_jwt_extended import JWTManager
from flask_ckeditor import CKEditor, CKEditorField
db = SQLAlchemy()
jwt = JWTManager()
ckeditor = CKEditor()
def create_app(config_type): # dev, test, prod
# create flask instance
app = Flask(__name__)
app.config['CKEDITOR_SERVE_LOCAL'] = True
app.config['CKEDITOR_HEIGHT'] = '800px'
app.config['CKEDITOR_WIDTH'] = '800px'
ckeditor.init_app(app)
#app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://postgres:postgres#localhost/catalog_db'
# setup configuration
configuration = os.path.join(os.getcwd(), 'config', config_type + '.py')
# load configuration from file
app.config.from_pyfile(configuration)
# attach flask to db instance(SQLALCHEMY)
db.init_app(app)
#attach flask to jwt
jwt.init_app(app)
Bootstrap(app)
# import blueprints
# inside app thats why app.catalog
from sales_tracking.api import api
from sales_tracking.api.get_api import get_api
from sales_tracking.api.get_api.get_all import get_all
from sales_tracking.api.post_api import post_api
from sales_tracking.api.post_api.authentications import authentications
from sales_tracking.api.post_api.bad_orders import bad_orders
from sales_tracking.api.post_api.box import box
from sales_tracking.api.post_api.breaktime import breaktime
from sales_tracking.api.post_api.change_dayoff import change_dayoff
from sales_tracking.api.post_api.check_in import check_in
from sales_tracking.api.post_api.check_in_monitoring import check_in_monitoring
from sales_tracking.api.post_api.competitors_act import competitors_act
from sales_tracking.api.post_api.competitors_promotion import competitors_promotion
from sales_tracking.api.post_api.confirmation_promotion_store import confirmation_promotion_store
from sales_tracking.api.post_api.confirmation_change_dayoff import confirmation_change_dayoff
from sales_tracking.api.post_api.confirmation_leave import confirmation_leave
from sales_tracking.api.post_api.confirmation_overtime import confirmation_overtime
from sales_tracking.api.post_api.daily_stocks import daily_stocks
from sales_tracking.api.post_api.facings import facings
from sales_tracking.api.post_api.leave import leave
from sales_tracking.api.post_api.logs_folder import logs_folder
from sales_tracking.api.post_api.mcp import mcp
from sales_tracking.api.post_api.mcp_adjustment import mcp_adjustment
from sales_tracking.api.post_api.nex import nex
from sales_tracking.api.post_api.planograms import planograms
from sales_tracking.api.post_api.product_discrepancy import product_discrepancy
from sales_tracking.api.post_api.confirmation_product_discrepancy import confirmation_product_discrepancy
from sales_tracking.api.post_api.promotions import promotions
from sales_tracking.api.post_api.request_overtime import request_overtime
from sales_tracking.api.post_api.rtv_pullout_and_create import rtv_pullout_and_create
from sales_tracking.api.post_api.shelf_availability import shelf_availability
from sales_tracking.api.post_api.stock_transfer import stock_transfer
from sales_tracking.api.post_api.tc import tc
from sales_tracking.api.post_api.tcp import tcp
from sales_tracking.api.post_api.team_attendance import team_attendance
from sales_tracking.api.post_api.update_position import update_position
from sales_tracking.api.post_api.sync_all import sync_all
from sales_tracking.web import web_initials
#reegister blueprint
app.register_blueprint(api)
app.register_blueprint(get_api)
app.register_blueprint(get_all)
app.register_blueprint(post_api)
app.register_blueprint(bad_orders)
app.register_blueprint(authentications)
app.register_blueprint(box)
app.register_blueprint(breaktime)
app.register_blueprint(change_dayoff)
app.register_blueprint(check_in)
app.register_blueprint(check_in_monitoring)
app.register_blueprint(competitors_act)
app.register_blueprint(competitors_promotion)
app.register_blueprint(confirmation_promotion_store)
app.register_blueprint(confirmation_change_dayoff)
app.register_blueprint(confirmation_leave)
app.register_blueprint(confirmation_overtime)
app.register_blueprint(daily_stocks)
app.register_blueprint(facings)
app.register_blueprint(leave)
app.register_blueprint(logs_folder)
app.register_blueprint(mcp)
app.register_blueprint(mcp_adjustment)
app.register_blueprint(nex)
app.register_blueprint(planograms)
app.register_blueprint(product_discrepancy)
app.register_blueprint(promotions)
app.register_blueprint(request_overtime)
app.register_blueprint(rtv_pullout_and_create)
app.register_blueprint(shelf_availability)
app.register_blueprint(stock_transfer)
app.register_blueprint(tc)
app.register_blueprint(tcp)
app.register_blueprint(team_attendance)
app.register_blueprint(update_position)
app.register_blueprint(web_initials)
app.register_blueprint(confirmation_product_discrepancy)
app.register_blueprint(sync_all)
return app

For anybody else having this kind weird issues. Please use python env while pip installing, that might fix the issue.
There is really good blog post by one of python core developers why you should use python -m pip instead of pip read about it here.

Add location of CKEditor static files in apache/Nginx sites-available
location /static/ckeditor/ {
root /home/username/myProject/env/lib/python3.8/site-packages/ckeditor;
}

How to reference create_app from different sub-domain?

The relevant part of my app structure looks like the following:
hp/
|
|---app/
|---admin/
|---auth/
|---errors/
|---main/
|---__init__.py
|---email.py
|---models.py
|---search.py
|---config.py
|---quiz.py
I want to create a scripts/ domain in either hp/ or app/. In those scripts I need to be able to reference config values. I'm using dotenv to do that. In order to use dotenv, I need app to be available so that I can call app.config['CONFIG_NAME'].
Here's more or less what I'm trying to do:
import requests
from app import app
access_key = app.config['ACCESS_KEY']
secret_key = app.config['SECRET_KEY']
data = requests.get(f'https://api.foo.com/search?client_id={access_key}&page=1&query=foo').json()
If I try from app import app, as I have above, I get a ModuleNotFoundError: No module named 'app' error. If I try from .. import app I get a ValueError: attempted relative import beyond top-level package error.
Any guidance/advice is greatly appreciated!

I ended up solving this by changing the sys.path.
I added the following:
import sys
import os
import inspect
currentdir = os.path.dirname(os.path.abspath(inspect.getfile(inspect.currentframe())))
appdir = os.path.dirname(currentdir)
hpdir = os.path.dirname(appdir)
sys.path.insert(0, hpdir)
from app import create_app
app = create_app()
After that I was able to successfully call app.config

sys.path.append causing 500 error in Flask

I have a flask app that is giving me 500 when I import a file using sys.path.append('path/to/file.py)
Here is my file located in /var/www/html/ip.py which flask is calling:
import sys
sys.path.append('/auto/conf/')
from config import config
server_username = config['server_username']
server_password = config['server_prod_password']
def check_ip(ip_addr):
return "test"
Here is my /auto/conf/config.py:
import os
import ConfigParser
# Define names of dir and file
CRED_DIR = 'cred'
FILE_NAME = 'cred.conf'
# Path to cred file
my_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
cred_dir = os.path.join(my_dir, CRED_DIR)
file_path = os.path.join(cred_dir, FILE_NAME)
# Load the conf file with ConfigParser
config = ConfigParser.SafeConfigParser()
config.read(file_path)
# Build a config dictionary
config = {
'server_username': config.get('server', 'username'),
'server_password': config.get('server', 'password'),
}
and cred.conf is located in /auto/cred/cred.conf and contains server info.
Now, here is the issue. If I run python ip.py, it runs fine. I added print statement and it was fetching proper server username and password. But when I run it via Flask, it gives me 500 error.
Here are some of the things I tried:
-- If I comment out "from config import config" from ip.py, Flask runs returns "test" meaning it worked. It will not get server un and pw but atleast it does not 500.
-- If I move cred.conf and config.py to same directory as ip.py and comment out "sys.path.append('/auto/conf/')" and uncomment "from config import config", Flask works.
Any ideas why its happening? I am thinking Flask does not like sys.path.append. Is there any alternative I can use so Flask works?
Edit:
I changed ip.py to this:
import sys
sys.path.append('/auto/conf/')
import config
and removed all code in config.py and it is still gving me error. If I comment out "import config", Flask works. Definitely it does not like importing in this fashion..

Does __init__.py have to be in every directory of python application?

__init__.py are use to mark directories on disk as a Python package directories. lets say we have the files
py/src/__init__.py
py/src/user.py
and py is on your path, you can import the code in user.py as:
import src.module
or
from src import user
If we remove the __init__.py file, Python will no longer look for submodules inside that directory, so attempts to import the module will fail.
But on my app the __init__.py is not working.
__init__ .py
__all__ = ['models', 'common']
from src.models.user import User
from src.models.post import Post
from src.models.blog import Blog
When I run python app.py
errors:
Traceback (most recent call last):
File "src/app.py", line 5, in <module>
from src.models.user import User
ModuleNotFoundError: No module named 'src'
App.py file:
import os
import sys
from flask import Flask, render_template, request, session
from src.models.user import User
app = Flask(__name__) #'__main__'
#app.route('/') # 'OUrsite.com/api/'
# def hello world ot test the first api call
def hello_method():
return render_template('login.html')
#app.route('/login')
def login_user():
email = request.form['email']
password = request.form['password']
if User.login_valid(email, password):
User.login(email)
return render_template("profile.html", email=session['email'])
if __name__ == '__main__':
app.run(port = 9100)
Am I missing something?

There are two things to understand.
There is a difference in root directory of virtualenv and your system environment.
Virtualenv sets your project dir to it's root. But in case you are not using virtualenv, the root would be your system directory (at least, in my experience).
So, while creating __init__py, keep in mind that, if there is no virtualenv or you haven't imported anything, it can be empty.
simple __init_.py examples:
# in your __init__.py
from file import File
# now import File from package
from package import File
My __init__.py:
import src
from src import *
from .models.user import User
from .models.post import Post
from .models.blog import Blog
And when you import in app.py or src/models/user.py which is in a same directory & subdirectory, you shall not include src:
from models.user import User
from common.database import Database

You didn't import src first.
import src
__all__ = ['models', 'common']
from src.models.user import User
from src.models.post import Post
from src.models.blog import Blog
If this is src/__init__.py, you can use a relative import:
from __future__ import absolute_import
from .models.user import User
from .models.post import Post
from .models.blog import Blog