Develop Reference

When running Python Flask Server, Jenkins is not successful and continues to load

I am building CICD through Jenkins.
But there are problems.
It is planning to upload source code first and turn on flask server through batch file.
I wrote a shell script for Jenkins' Build>Execute Shell.
postCommand=/cygdrive/c/workspace/ContactPortal_Flask/run.bat
sshpass -p ${deployPassword} ssh -o StrictHostKeyChecking=no ${deployUser}#${deployServer} ${postCommand}
Here is run.bat file
set FLASK_ENV=development
set path=%path%;C:\Program Files\Microsoft SQL Server\110\Tools\Binn\;C:\develop\instantclient_12_1;C:\develop\Anaconda3;C:\develop\Anaconda3\Library\mingw-w64\bin;C:\develop\Anaconda3\Library\usr\bin;C:\develop\Anaconda3\Library\bin;C:\develop\Anaconda3\Scripts;
set "START=C:\workspace\ContactPortal_Flask\start.bat"
cd C:\workspace\ContactPortal_Flask
python -m flask run
then, The source code upload was successful, and turning on the flask server was also successful, but Jenkins was not marked Success and continued to load.
please help!!

I think the main problem here is that python -m flask run starts the server and will not finish until user hit Ctrl+C.
Since the target system is on Windows, you may want to create custom service and have jenkin start that service at the end instead. For service creation see https://learn.microsoft.com/en-us/troubleshoot/windows-client/deployment/create-user-defined-service And by starting this service (e.g. with NET START <service-name>) jenkin can finish, and flask can start running in the background.
Also for a production system, you may want to consider checking this and pick a proper web server instead of using the builtin web server provided by flask.

Deploying a python Flask application with Jenkins and executing it

I am trying to do auto-deployment of a Python Flask application using Jenkins and then run it by using shell command on a Raspberry Pi server.
Here are some background info,
Before using Jenkins, my deployment and execution process was manual described below:
FTP to the directory where my Python scripts and Python venv are located
Replace Flask application scripts using FTP
Activate virtual environment to of Python(3.5) through the terminal on Raspberry Pi ("./venv/bin/activate")
Run myFlaskApp.py by executing "python myFlaskApp.py" in terminal
Now I have integrated Jenkins with the deployment/execution process described below:
Code change pushed to github
Jenkins automatically pulls from github
Jenkins deploy files to specified directories by executing shell commands
Jenkins then activates virtual environment and run myFlaskApp.py by bashing a .sh script in the shell terminal.
Now the problem that I am having is on step 4, because a Flask app has to always be alive, my Jenkins will never "finish building successfully", it will always be in a loading state as the Flask app is running on the shell terminal Jenkins is using.
Now my question:
What is the correct approach that I should be taking in order to activate myFlaskApp.py with Jenkins after deploying the files while not causing it to be "locked down" by the build process?
I have read up about Docker, SubShell and the Linux utility "Screen". Will any of these tools be useful to assist me in my situation right now and which approach should I be taking?

The simple and robust solution (in my opinion) is to use Supervisor which is available in Debian as supervisor package. It allows you do make a daemon from script like your app, it can spawn multiple processes, watch if app doesn't crash and if it does it can start it again.
Note about virtualenv - you don't need to activate venv to use it. You just need to point appropriate Python executable (your_venv/bin/python) instead of default one. For example:
$ ./venv/bin/python myFlaskApp.py

You need to create these files for deployment over jenkins.
Code can be found: https://github.com/ishwar6/django_ci_cd
This will work for both flask as well as django.
initial-setup.sh - This file is the first file to look at when setting up this project. It installs the required packages to make this project work such as Nginx, Jenkins, Python etc. Refer to the youtube video to see how and when it is used.
Jenkinsfile - This file contains the definition of the stages in the pipeline. The stages in this project's pipeline are Setup Python Virtual Environment, Setup gunicorn service and Setup Nginx. The stages in this pipeline just does two things. First it makes a file executable and then runs the file. The file carries out the commands that is described by the stage description.
envsetup.sh - This file sets up the python virtual environment, installs the python packages and then creates log files that will be used by Nginx.
gunicorn.sh - This file runs some Django management commands like migration commands and static files collection commands. It also sets up the gunicorn service that will be running the gunicorn server in the background.
nginx.sh - This file sets up Nginx with a configuration file that points Nginx to the gunicorn service that is running our application. This allows Nginx serve our application. I have followed a digital ocean article to setup this file. You can go through the video once to replicate sites-available and sites-enabled scanerio.
app.conf - This is an Nginx server configuration file. This file is used to setup Nginx as proxy server to gunicorn. For this configuration to work, change the value of server_name to the IP address or domain name of your server.

Heroku django manage shell with environment setup (db, vars)

I'm trying to get to the django shell so that I can set some superuser permissions for our staff accounts. Doing this by ssh tunneling via ps:exec
We just moved over to heroku, and are using their auto-configured (read: automagical black box) module import django_heroku; django_heroku.settings(locals()) to set database settings.
When I open the shell, those config settings aren't run, and I get an error saying that
django.db.utils.OperationalError: no such table: user
Basically, I can't get the shell instance to use Heroku's nice postgres configurer. Any ideas? Workarounds to manually query accounts and set some params? I've tried to import that same magical module from Heroku as part of the shell and instantiate it, but it only works when run in the settings.py file.
I'm trying to not actually log into the postgres instance so that I don't have to open any IP ranges, etc.
__________edit_________
On a similar note, it looks like the shell also can't load Heroku's env vars... getting some issues there as well. Is there a way to run the shell with Heroku's environment completely?

You can run python manage.py shell directly through the heroku cli:
heroku run python manage.py shell --app=your-app-name

You can access the Django shell through the Heroku Dashboard by using the view console button.
So, you can access your shell (Django shell) or the Linux shell of Heroku directly.

Using python flask and pexpect works on flask development server fails with WSGI

I am writing a tool for internal use at work. A user enters a router or switch IP address, username and password into a web form. The app then uses pexpect to SSH into the device, downloads a configuration and tests that the configuration complies with various standards by running true/false tests (e.g. hostname is set). Leaving aside whether this is a good idea or not, my problem is that when I run the program under the Flask development program it works fine. When I set it up to run under WSGI it fails at the SSH portion with the error:
pexpect.exceptions.ExceptionPexpect: The command was not found or was not executable: ssh.
I tried uWSGI and Unicorn and played with the number of workers etc. to no avail.
I suspect this is a setuid root thing. Google searches do not point to a solution. Can anyone lead me to a fix? If pexpect will not work, I may give up and require the user to upload a config file they save themselves but I am frustrated that this works on the flask development server but not a production server.

You probably just need to replace ssh with its full path, e.g. /usr/bin/ssh.
You can find the full path with which ssh.

Deploy Flask app on Apache Server using mod_wsgi and WinSCP

I want to deploy my Flask application on an Apache server. I have an account on the server, and have been told that "The server can be used to run scripts and web apps written in Python (using django and mod_wsgi)".
I am on Windows, and to transfer files I have to use an FTP client - so I am using WinSCP.
Installing mod_wsgi is not as straightforward as I expected and I cannot get any clear documentation online.
Because the server can already run Python scripts using mod_wsgi does that mean that I just have to create a .wsgi file or do I still need to download it?
I don't know how to go about this.

First you need to check if mod_wsgi is really enabled on the server, then you have to check how your virtual host is configured in apache. There you will find the name you have to give to the wsgi file.
If you have shell access to the server you can do that by using the following commands:
Check mod_wsgi:
sudo apache2ctl -t -D DUMP_MODULES | grep wsgi
Check what name the .wsgi file should have:
sudo grep WSGIScriptAlias /etc/apache2/sites-enabled/yoursite.conf