I am planning on making a website using Python + Flask + Mysql. My database requirements are very simple and hence I don't plan on using an ORM. Now I have been reading through the tuorials of mysqldb which is a C based mysql connector for Python. How can I install it on openshift?
For installing it on my local pc, I had to run the following two commands.
python setup.py build
python setup.py install
Here is doc that shows how to spin up a Flask app on OpenShift
https://developers.openshift.com/en/python-flask.html
All you do is add the dependencies for the MySQL drivers into your requirements.txt as well and OpenShift will pull the deps and then put them in your app.
Related
I'm working to an apllication that uses selenium for scraping a webpage and stores the data in a mysql database. I'm using mysql server 8.0 and i want to be able to use this application in a mac computer.
Is there a way to do that? If yes how?
You can install Python and MySQL server on your Mac using Homebrew. From here you can install your necessary Python dependencies via pip and run your application.
Alternatively, you could setup the application as a Docker service with containers for each component. This is probably the easiest way, though you'll need to navigate the docker learning curve.
Good day!
I need to deploy an app on Python Django with following libs:
certifi==2017.4.17
chardet==3.0.4
Django==1.11.2
httplib2==0.10.3
idna==2.5
oauth2==1.9.0.post1
psycopg2==2.7.1
pytz==2017.2
requests==2.18.1
urllib3==1.21.1
And I have to connect it to PostgreSQL database.
So, what environment shall I use? Standard or Flexible?
And how shall I install those libraries to the environment?
P.S.
I've tried everything: app.yaml, appengine_config.py, I has installed libs directly to source ('libs' folder), and adding libs to app.yaml, and even google.appengine.ext.ndb.django_middleware.NdbDjangoMiddleware
In the end I have:
ImproperlyConfigured: Error loading psycopg2 module: dynamic module
does not define init function (init_psycopg)
So, what environment shall I use? Standard or Flexible?
Your starting point should be the Choosing an App Engine Environment guide, taking into consideration all your requirements.
And I have to connect it to PostgreSQL database.
This requirement makes the choice pretty easy. From Connecting from App Engine:
This page provides language-specific links to informaton about how to
set up a connection from an App Engine flexible environment
application to a Cloud SQL for PostgreSQL instance.
Note: Connection from an App Engine standard environment application
to a PostgreSQL instance is not supported.
So flexible environment it is.
And how shall I install those libraries to the environment?
This dependends on the environment you use. What you tried was the standard env way.
In the flexible environment dependencies are handled differently. From Dependencies:
The runtime looks for a requirements.txt file in your
application's source directory and uses pip to install any
dependencies before starting your application. For more information on
declaring and managing packages, see Using Python Libraries
I'm trying to deploy a Flask web app with mysql connectivity. It's my first time using Azure, and coming off Linux it all seems pretty confusing.
My understanding is that one includes within the requirements.txt to include the packages required. When I build the default Flask app from Azure the file looks like this:
Flask<1
At this stage the site loads fine.
If I then include an additional line
https://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.1.14.tar.gz
As per this answer https://stackoverflow.com/a/34489738/2697874
Then in my views.py file (which seems to be broadly synonymous to my old app.py file) I include...import mysql.connector
I then restart and reload my site...which then returns the error The page cannot be displayed because an internal server error has occurred.
Error logging spits out a load of html (seems pretty weird way to deliver error logs - so I must be missing something here). When I save to html and load it up I get this...
How can I include the mysql.connector library within my Flask web app?
Per my experience, the resoure https://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.1.14.tar.gz is for Linux, not for Azure WebApps based on Windows, and the link seems to be not available now.
I used the command pip search mysql-connector to list the related package. Then, I tried to use mysql-connector instead of mysql-connector-python via pip install, and tried to import mysql.connector in local Python interpreter that works fine.
So please use mysql-connector==2.1.4 instead of mysql-connector-python== in the requirements.txt file of your project using IDE, then re-deploy the project on Azure and try again. The package will be installed automatically as the offical doc said as below.
Package Management
Packages listed in requirements.txt will be installed automatically in the virtual environment using pip. This happens on every deployment, but pip will skip installation if a package is already installed.
Any update, please feel free to let me know.
I'm trying to build automated testing on Django app on a private BitBucket repo, using BitBucket pipelines.
I have all the tests in place, and they can be run using ./manage.py test or using tox. This works fine locally, where the tests build a (temporary) PostGIS test database. PostGIS and stuff like gdal is necessary for all the requirementst.txt to properly install.
I'm having problems finding a Docker image that gives me a full Python + PostGIS/gdal etc stack on Docker, and I lack the skills to build it myself.
My failed attemps are documented on GitHub:
https://github.com/zostera/docker-django-ci
Can someone point me in the right direction (tutorial) or perhaps help me out with a working example?
I've written a detailed post about Building, Testing and Deploying Django App with Bitbucket Pipelines
It can be a good start for extending it with PostGIS and I will point you in the right direction. In addition to the post, we've open sourced Dockerfiles which you can extend to support PostGIS.
You'll need a good tutorial on installing PostGIS, e.g. try this one.
Start with the centos7-postgresql9.4 Dockerfile, and add commands for installing PostGIS, right before VOLUME line:
...
# install PostGIS
RUN yum -y install postgis2_94 postgis2_94-client
VOLUME ["/var/lib/pgsql/9.4"]
Finally, you have to enable extensions in database. Add this code to start_postgres.sh:
if [ -n "${POSTGRESQL_DATABASE}" ]; then
echo "Creating database \"${POSTGRESQL_DATABASE}\"..."
sudo -u postgres psql -c "CREATE DATABASE \"${POSTGRESQL_DATABASE}\" OWNER \"${POSTGRESQL_USER}\";"
# create db gistest, connect to db gistest and create postgis extension
sudo -u postgres psql -c "CREATE DATABASE gistest;\c gistest;CREATE EXTENSION postgis;"
fi
We have recently figured it out, and published our own Docker image with support for Python, Django, Postgres, PotGIS and a few other things.
https://github.com/zostera/docker-django-ci
I'm following this guide and trying to develop a Flask app to run on the Google App Engine. I followed the guide to the letter but when I launch the dev app server from the Launcher and go to http://localhost:8080/, I get a HTTP 500 error.
I check the logs and it says No module named flask. Then I check the interactive console in the admin console by running import flask and I get the same error message. I can import flask in any other python file without error.
Is there a way to fix this?
Working a bit with GAE and Flask I have realized this:
Running directly with Python
To run the app with python directly (python app.py) you need have the dependents packages installed in your environment using the command: pip install flask
Running with dev_appserver.py
To run the app with the dev_appserver.py provided by GAE SDK you need have all dependent packages inside your project, as: Flask, jinja2... Look in my another answer a example how to configure this packages : https://stackoverflow.com/a/14248647/1050818
UPDATED
Running Python, Virtualenv, Flask and GAE on Windows
Install Python
Install Python http://www.python.org/ftp/python/2.7.2/python-2.7.2.msi
Click in Windows Start button and search by "Edit the system environment" and open
Go to the tab Advanced and click on button "Environment Variables…"
When the Environment Variables window opens, choose Path from the System variables list and click Edit…
Add this ;C:\Python27;C:\Python27\Scripts at the end of the value and save
Install setuptools MS Windows installer (Necessary to install PIP on Windows)
Choose the correct installer for you in this page http://pypi.python.org/pypi/setuptools#files( I used this one: http://pypi.python.org/packages/2.7/s/setuptools/setuptools-0.6c11.win32-py2.7.exe#md5=57e1e64f6b7c7f1d2eddfc9746bbaf20)
Download the installar and Install that
Install PIP
Download PIP http://pypi.python.org/pypi/pip#downloads
Extract it to any folder
From that directory, type python setup.py install
Install Virtualenv
Execute pip install virtualenv
Execute this mkdir c:\virtualenvs to create a folder to the Virtual Envs
Execute this cd c:\virtualenvs to access that folder
Execute virtualenv flaskdemo to create a virtualenv for you project
Active the virtualenv c:\virtualenvs\flaskdemo\scripts\activate
Install Google App Engine SDK
Install the SDK https://developers.google.com/appengine/downloads
Create the project
Create a directory for your project
Create the main of your application https://github.com/maxcnunes/flaskgaedemo/blob/master/main.py
Create the configuration of your appliction for Google App Engine https://github.com/maxcnunes/flaskgaedemo/blob/master/app.yaml
Create a file to let GAE initialize your application https://github.com/maxcnunes/flaskgaedemo/blob/master/initialize_gae.py
(Look a example of the code here: https://github.com/maxcnunes/flaskgaedemo )
Install Flask to run Locally
Execute pip install flask
Install Flask to run on the GAE
Download Flask https://github.com/mitsuhiko/flask/archive/0.9.zip and extract the folder flask inside your project
Download Werkzeug https://github.com/mitsuhiko/werkzeug/archive/0.8.3.zip and extract the folder werkzeug inside your project
Download Jinja2 https://github.com/mitsuhiko/jinja2/archive/2.6.zip and extract the folder jinja2 inside your project
Download Simple Json https://github.com/simplejson/simplejson/archive/v3.0.5.zip and extract the folder simplejson inside your project
Running the application with GAE SDK
Open Google App Engine Launcher
Add a new application
Run the application
Click in Browse button to open your application on browser
Finally click on Deploy button to deploy your application
Usually, templates come with a requirements.txt. If not, add your dependencies there and then run pip install -t lib -r requirements.txt to force the libraries to be saved in the lib folder.
Make sure you've added lib to appengine_config.py with vendor.add('lib') if it's not already there.
I was also facing the same issue and after spending 1 day on it have found out my silly mistake actually while refactoring my flask app I have changed
appengine_config.py to some other name.
Ideally appengine_config.py should look like below if you are having all your dependencies in lib folder only
from google.appengine.ext import vendor
#Add any libraries installed in the "lib" folder.
vendor.add('lib')
And because it was not able to find and execute appengine_config.py so lib folder was not registered as a dependency folder. To check you can try printing something in appengine_config.py to check if it's being executed on server startup.
tldr: use appengine_config.py and copy your virtualenv to a folder called lib, then make SURE you are running the app via dev_appserver.py
(the below is via bash in ubuntu)
SO after a long battle, I find that virtual env and gcloud dont play nice -
I copied everything from my virtual env dir
.../.virtualenvs/nimble/local/lib/python2.7/site-packages
into
[projectdir]/lib
and my appengine_config.py finally worked locally like it does in the cloud, but I absolutely HAVE to run
dev_appserver.py [my proj dir here]
or the google.appengine module wont load. did not know I should be using dev server. I feel very dumb.
for reference, heres the appengine_config.py
"""`appengine_config` gets loaded when starting a new application instance."""
print 'running app config yaya!'
from google.appengine.ext import vendor
vendor.add('lib')
print 'I am the line after adding lib, it should have worked'
import os
print os.getcwd()
Do you have Extra Libraries component for Python installed? It can be installed with
gcloud components install app-engine-python-extras
After installing this extra library you should be able to use built-in flask library without a problem. For more information, refer to this page
Source