GAE: Best practice for dynamically generated projects - python

Let's say I am creating a python-based CMS on GAE (similar to Squarespace/Shopify) which allows users to create a website.
The platform will (automatically?) create a subdomain for each new user and duplicates the application.
Now there are two options:
1) Create a new Database for the new user, WITHIN the master GAE project. (I'm worried that if one user gets a lot of traffic it might slow down ALL websites.)
2) Duplicate the entire project. (This method seems difficult to accomplish because either I have to manually create an instance of the application for each user, or I have to figure out how to hijack gcloud.py (or appcfg.py) somehow and store my login credentials in the code.)
Which choice will most likely provide the most performance for the price? Is choice 2 allowed by Google (or even possible)?
Edit:
I've done some more research about this, and it's not documented very much. I found this in the docs https://cloud.google.com/sdk/docs/scripting-gcloud which talks about running gcloud from scripts, although I don't think that means from python. I am looking into appengine-jenkins to see if it will work for my purpose. Let me know if you have any additional information about this.
Also, it seems like gcloud is adding a create command within the projects command which might be useful for me if I can figure out how to run gcloud from my script. https://cloud.google.com/sdk/gcloud/reference/alpha/projects/create

Related

Deploy a Python application to web that interacts with local files

Background: Am comfortable in Python, know nothing about web deployment. I am looking into it as an alternative to compiling into .exe or .app for Win or Mac distributions.
Issue: I have a simple application that uses BeautifulSoup, openpyxl, and PySimplyGUI. It interacts with some local excel-files and creates new ones. I want to be able to, using minimum effort, make it accessible on my own web page or something similar, and make the created excel-files available for browsing/download. I have no idea how to do any of this. I've been looking into Flask and cloud foundry, but it feels like there should be some easy alternative that I'm missing. Ideally I would want a page where someone can log in (given a username and password I supply), which then directs to a page where the user can interact with my application.
Request: Is there a relatively easy way to do this that doesn't involve setting up a lot of stuff in html, etc., and where excel-files can still be interacted with by openpyxl? I ideally would just want some template, where I can "fill in the blanks" for the python method I would want to execute for each button!
Hope this makes sense. Thanks in advance :)
The easiest way to create a web app with a simple interface yet effective which does not require frontend programming is Streamlit. It is primarily used by data scientist to create simple web apps quickly.

python celery for every work is efficient?

I'm making API server with python flask.
In my case, it is real production level, So I have to be careful when developing server.
After google searching, found that celery&redis is suitable for task queueing.
So I installed celery&redis via pip3 install 'celery[redis]' and defined task, and run.
Everything was fine, but I got some question about it.
Assume that there is user model. Maybe CRUD for user model like this.
Register user(with photo)
Delete user
Get a single user
In my personal think, only Register user need to celery&redis.
Because upload photo can take long time, so it have to treated with asynchronize work.
Delete user and Get a slngle user just query to db and retreive it.
So it doesn't takes longer time. (it means, do not need to work with celery)
It is right? Or, any missing feature I do not know?
To summarize my question, I want to know that is there any standard for celery?
Thanks!
You have it about right. You can put whatever processing you want in celery, but the rule that you just used--use celery for things that take a long time--is the one we use most in our production environment. You can also use celery when you want to scale out an operation across servers more easily. For example, when scraping a large number of pages, you might want to execute that in parallel to speed up what would otherwise be a long-running task.
I do think there is a great tutorial about this topic.
using-celery-with-flask
And you can also check out this repo.

Standalone Desktop App with Centralized Database (file) w/o Database Server?

I have been developing a fairly simple desktop application to be used by a group of 100-150 people within my department mainly for reporting. Unfortunately, I have to build it within some pretty strict confines similar to the specs called out in this post. The application will just be a self contained executable with no need to install.
The problem I'm running into is figuring out how to handle the database need. There will probably only be about 1GB of data for the app, but it needs to be available to everyone.
I would embed the database with the application (SQLite), but the data needs to be refreshed every week from a centralized process, so I figure it would be easier to maintain one database, rather than pushing updates down to the apps. Plus users will need to write to the database as well and those updates need to be seen by everyone.
I'm not allowed to set up a server for the database, so that rules out any good options for a true database. I'm restricted to File Shares or SharePoint.
It seems like I'm down to MS Access or SQLite. I'd prefer to stick with SQLite because I'm a fan of python and SQLAlchemy - but based on what I've read SQLite is not a good solution for multiple users accessing it over the network (or even possible).
Is there another option I haven't discovered for this setup or am I stuck working with MS Access? Perhaps I'll need to break down and work with SharePoint lists and apps?
I've been researching this for quite a while now, and I've run out of ideas. Any help is appreciated.
FYI, as I'm sure you can tell, I'm not a professional developer. I have enough experience in web / python / vb development that I can get by - so I was asked to do this as a side project.
SQLite can operate across a network and be shared among different processes. It is not a good solution when the application is write-heavy (because it locks the database file for the duration of a write), but if the application is mostly reporting it may be a perfectly reasonable solution.
As my options are limited, I decided to go with a built in database for each app using SQLite. The db will only need updated every week or two, so I figured a 30 second update by pulling from flat files will be OK. then the user will have all data locally to browse as needed.

GUI interface for sqlite data entry in Python

I am making a simple sqlite database for storing some non-sensitive client information. I am very familiar with python+sqlite and would prefer to stick with this combo on this project. I would like to create an simple GUI interface for data entry and searching of the database... something very similar to what MS Access provides. I want my wife to be able to enter/search data easily, so PHPmyadmin style things are out of the question.
I understand I could just give in and get MS Access, but if reasonbly possible would rather just write the code myself so it will run on my computers (*nix) and is flexible (so I can later integrate it with a web application and our smart phones.)
Can you developers recommend any interfaces/packages/etc (preferably pythonic) that can accomplish this with reasonable ease?
Thanks!
Since you're interested in future integration with a web application, you might consider using a Python web framework and running the app locally on your machine, using your web browser as the interface. In that case, one easy option would be web2py. Just download, unzip, and run, and you can use the web-based IDE (demo) to create a simple CRUD app very quickly (if you really want to keep it simple, you can even use the "New application wizard" (demo) to build the app). It includes its own server, so you can run your app locally, just like a desktop app.
You can use the web2py DAL (database abstraction layer) to define and create your SQLite database and tables (without writing any SQL). For example:
db = DAL('sqlite://storage.db')
db.define_table('customer',
Field('name', requires=IS_NOT_IN_DB(db, 'customer.name')),
Field('address'),
Field('email', requires=IS_EMAIL()))
The above code will create a SQLite database called storage.db and create a table called 'customer'. It also specifies form validators for the 'name' and 'email' fields, so whenever those fields are filled via a form, the entries will be validated ('name' cannot already be in the DB, and 'email' must be a valid email address format) -- if validation fails, the form will display appropriate error messages (which can be customized).
The DAL will also handle schema migrations automatically, so if you change your table definitions, the database schema will be updated (if necessary, you can turn off migrations completely or on a per table basis).
Once you have your data models defined, you can use web2py's CRUD system to handle all the data entry and searching. Just include these two lines (actually, they're already included in the 'welcome' scaffolding application):
from gluon.tools import Crud
crud = Crud(db)
And in a controller, define the following action:
def data():
return dict(form=crud())
That will expose a set of pre-defined URLs that will enable you to create, list, search, view, update, and delete records in any table.
Of course, if you don't like some of the default behavior, there are lots of ways to customize the CRUD forms/displays, or you can use some of web2py's other forms functionality to build a completely custom interface. And web2py is a full-stack framework, so it will be easy to add functionality to your app as your needs expand (e.g., access control, notifications, etc.).
Note, web2py requires no installation or configuration and has no dependencies, so it's very easy to distribute your app to other machines -- just zip up the entire web2py folder (which will include your app folder) and unzip it on another machine. It will run on *nix, Mac, and Windows (on Windows, you will either need to install Python or download the web2py Windows binary instead of the source version -- the Windows binary includes its own Python interpreter).
If you have any questions, there's a very helpful and responsive mailing list. You might also get some ideas from some existing web2py applications.
I normally use GTK+ that has well documented Python bindings.
The biggest advantage is that you can use a fairly intuitive GUI editor (Glade) and automagically link callbacks to events (to be honest most other major graphical toolkits have this possibility too, like for example QT, but my perception is that GTK+ enjoys wider adoption in the Python community). EDIT: additionally GTK is used by Gnome and many other desktop environments (KDE uses QT though).
That said, if all you need is truly just data insertion from a trusted person, you could use something already done like SQLite manager (it's a FireFox plugin).
Radically alternative solution: use django and you can literally pass from reading the tutorial to have your app up and running in a matter of hours, inclusive of user authentications, back-end administrative interface etc. (your project is exactly what I did with it to allow my wife to insert expenses in our family budget).
Django is written in Python and you can use SQLite as a back-end.

I'm learning python and am interested in using it for web-scripting. What frameworkes are out there and do I need one?

I've been learning python for use in ArcGIS and some other non-web applications. However, now that I've taken on building a personal website I am interested in using it for web development (as it is the only scripting language I currently know).
I've noticed that there are a lot of these things called "frameworks", such as Django. From what I understand they are just a collection of packages to save you from re-inventing the wheel but I don't really know how they work.
Furthermore, I do not like GUIs, if I need a framework I would like to find one that could be used through a terminal, starts out simple and can be scaled for more complexity when I'm ready. Any advice or ideas on frameworks and why I would want to use one?
The Python web frameworks have nothing to do with GUIs, and can all be used via the terminal.
The benefits of a framework, as you say, are all to do with making your life easier by supplying the components you need to build a website: the main ones are database interaction through an ORM, a templating system, and URL routing. On top of that, the big frameworks also included optional extras like user authentication, administration interface, and so on.
Personally I like Django, but your mileage may vary: I would say, though, that whatever you do with Python and the web will require some sort of framework, even if it's one of the absolute minimal ones like Flask which basically do just the routing part. There's simply no point in writing all this stuff from scratch when it's been done for you.
I'd second the post above: Django is a great framework and will save you loads of time in the long run.
Pretty much every challenge you'll come across when writing a web application will already have been solved, e.g. How do I send emails? What about an admin interface to edit the data? User security?
In my view picking the best framework is all about the ecosystem around that framework. How well used is it? Is it discussed widely on the internet? Have others encountered, and solved, the problems I'm facing?
In terms of where you start, see the Django Tutorial here:
http://docs.djangoproject.com/en/1.2/intro/tutorial01/
If you think Django offers you too much, I'd recommend that you take a look at CherryPy just to compare the different, and much simpler, approach.
With Python, you've got lots of options. To start, I would recommend looking here -- it explains the basics and provides a fairly complete list of frameworks.
If you're looking for something that starts out simple but can also handle more complexity, then you should take a look at web2py. It requires no installation or configuration, has no dependencies, and includes a web server and a relational database. It also includes an optional web-based integrated development environment and admin interface, but you can work through the terminal instead if you prefer.
It's very easy to learn and was designed for ease of use, faster development, and security. You can get a lot done with very little code thanks to the included scaffolding app along with many sensible default behaviors. As things get more complex, web2py can handle it, as it is a well-integrated full-stack framework with lots of built-in functionality, including a database abstraction layer, form handling and validation, access control, web services, and easy Ajax integration.
Personnally, I don't use any framework, I write either from scratch on BaseHTTPServer, or using WSGI (with mod_wsgi).
It is a bit long to write the skeleton, but I think it is faster (I mean at runtime), there is less constraints, and there is lesser to learn.

Categories

Resources