Develop Reference

Deploying an app that can upload, create new files, and download created files

I am currently working on creating a flask app that can take uploaded files(.xslx), use them for analysis, and then redirect to a page with the output and a download button to download the output that they see.
The local version I have right now just stores these files in ./uploads/. It then uses the uploaded file as input for some analysis. To do this analysis it uses another file stored in ./analysis/ and the results of this analysis are then stored in ./results/.
Currently, this flask app works but to deploy it, as far as I know, I can no longer use ./uploads/ and ./results/ nor can expect for there to be ./analysis/.
While I have worked on a few flask apps as projects, I have never really had to worry about deploying them (since I'm a student and have mainly worked on personal projects.)
What is the best way to deploy this app? I was going to use Heroku since that's the only deployment platform I've used in the past to deploy a full stack application and I'm completely lost on how to deploy this app.

appengine set up local host with datastore for testing

I have tried to follow googles documentation on how to set up local development using a database (https://cloud.google.com/appengine/docs/standard/python/tools/using-local-server#Python_Using_the_Datastore). However, i do not have the experience level to follow along. I am not even sure if that was the right guide. The application is a Django project that uses python 2.7. To run the local host, i usually type dev_appserver.py --host 127.0.0.1 .
My questions are:
how do i download the data store database on google cloud. I do not want to download the entire database, just enough data to populate local host so i can do tests
once the database is download, what do i need to do to connect it to the localhost? Do i have to change a parameter somewhere?
do i need to download the datastore? Can i just make a duplicate on the cloud and then connect to that datastore?
When i run localhost, should it not already be connected to the datastore? Since the site works when it is running on the cloud. Where can i find the connection URI?
Thanks for the help

The development server is meant to simulate the whole App Engine Environment, if you examine the output of the dev_appserver.py command you'll see something like Starting Cloud Datastore emulator at: http://localhost:PORT. Your code will interact with that bundled Datastore automatically, pushing and retrieving data according to the code you wrote. Your data will be saved on a file in local storage and will persist across different runs of the development server unless it's explicitly deleted.
This option doesn't provide facilities to import data from your existing Cloud Datastore instance although it's a ready to go solution if your testing procedures can afford populating the local database with mock data through the use of a custom created script that does so programmatically. If you decide for this approach just write the data creation script and execute it before running the tests.
Now, there is another option to simulate local Datastore using the Cloud SDK that comes with handy features for your purposes. You can find the available information for it under Running the Datastore Emulator documentation page. This emulator has support to import entities downloaded from your production Cloud Datastore as well as for exporting them into files.
Back to your questions:
Export data from the Cloud instance into a GCS bucket following this, then download the data from the bucket to your filesystem following this, finally import the data into the emulator with the command shown here.
To use the emulator you need to first run gcloud beta emulators datastore start in a Cloud Shell and then in a separate tab run dev_appserver.py --support_datastore_emulator=true --datastore_emulator_port=8081 app.yaml.
The development server uses one of the two aforementioned emulators, in both cases it is not connected to your Cloud Datastore. You might create another project aimed for development purposes with a copy of your database and deploy your application there so you don't use the emulator at all.
Requests at datastore are made trough the endpoint https://datastore.googleapis.com/v1/projects/project-id although this is not related to how the emulators manage the connections in your local server.
Hope this helps.

Running a Python app on a remote server and displaying the output files

I have a Python 2.7 script at https://github.com/jhsu802701/dopplervalueinvesting . When I run the screen.py script locally, the end result is a new screen-output sub-directory (within the root directory) and a results.csv file within it.
What I'm trying to do is put this script on a remote server, run this screen.py script every night, and make the results.csv file publicly readable.
I've tried to do this on Google App Engine, but I can't get it to work. The Google App Engine tutorial revolves around trying to dynamically create a web site, and I haven't been able to figure out how to make anything other than an index.html file in the root directory work. HOW DO I MAKE OTHER FILES PUBLICLY READABLE?
Is Google App Engine the way to go, or am I barking up the wrong tree? I understand that another route is using WebFaction, a web hosting provider that offers a whole Linux system. (Running my app on my current web host, MDDHosting, is not an option because lxml is not available without a much more costly VPS.)
In summary, my questions are:
How do I run my Python script in Google App Engine and make the output results.csv file publicly available?
If Google App Engine isn't the solution for me, should I use WebFaction? (I already tried Heroku, and it didn't work for me.)
What are my other options?
I'm willing to pay for a solution, but only if I get web hosting as well. (I'm not willing to pay for MDDHosting for my dopplervalueinvesting.com web site AND another host for running my script.)

I think GAE should be good for what you want, but you may need to work differently because, as a comment pointed out, you can't write to the file system but have to use the datastore instead.
So you need in your app.yaml list of handlers, something like
- url: /results.csv
script: deliver_results_file.py
- url: /screen
login: admin
script: screen.py
screen.py needs to save the results to the datastore in some convenient format. deliver_results_file.py then queries the datastore, and if the results are not already in CSV format then it converts them accordingly. It then writes the formatted data directly to the output (usually using self.response.out.write within a webapp request handler) as if it was a dynamically generated webpage.
Finally you want to schedule it to run once a night - I believe this is possible using cron jobs.

Creating a KhanAcademy clone via Google App Engine - issues with application name in app.yaml

I'm trying to create a KhanAcademy (KA) clone on Google App Engine (GAE). I downloaded the offline version of KA (http://code.google.com/p/khanacademy/downloads/list) for Mac, and set it up with GoogleAppEngineLauncher (https://developers.google.com/appengine/). Because KA was produced on Python 2.5, I have the setup running through the Python 2.5 included in the KA offline version download, and I added these extra flags to the app (to essentially duplicate the functionality of the included Run file):
--datastore_path=/Users/Tadas/KhanAcademy/code/datastore --use_sqlite
As is, GAELauncher is able to get that up and running perfectly fine on a localhost. However, to get it up on my Google appspot domain, I need to change the application name in app.yaml. When I change "application: khan-academy" in app.yaml to a new name and try to run the local version via GAELauncher (or the included Run file), the site comes up but all the content (exercises, etc.) has disappeared (essentially, the site loses most of its functionality). If I try to "Deploy" the app in this state, I received a 500 Server Error when I try to go on the appspot website. Any ideas as to what could be going wrong?
Thanks.

The problem is that your 'clone' application does not have access to Khans Academy's AppEngine datastore so there is no content to display. Even if you do use all of the code for their application, you are still going to have to generate all of your own content.
Even if you are planning to 'clone' their content, too, you are going to have to do a lot of probably manual work to get it in to your application's datastore.

Importing Sqlite data into Google App Engine

I have a relatively extensive sqlite database that I'd like to import into my Google App Engine python app.
I've created my models using the appengine API which are close, but not quite identical to the existing schema. I've written an import script to load the data from sqlite and create/save new appengine objects, but the appengine environment blocks me from accessing the sqlite library. This script is only to be run on my local app engine instance, and from there I hope to push the data to google.
Am I approaching this problem the wrong way, or is there a way to import the sqlite library while running in the local instance's environment?

I would make suitable CSV files from the Sqlite data, in a separate script, then use bulk loading to push the data from the CSV files up to app engine.

If you need to access your datastore outside of the App Engine environment (like if you need to use libraries not present in App Engine or do other things App Engine does not support with the datastore) then the best option is the Remote Api.
There is an excellent tutorial on that here:
http://code.google.com/appengine/articles/remote_api.html
Essentially you import the remote_api module, authenticate with Google to access your datastore, then run your data access commands (query, update, delete, etc) as you normally would in app engine.

According to Google, you're doing it backwards. The app should be pulling data from you where you have more flexibility in converting to the new model anyway.

I have not had any trouble importing pysqlite2, reading data, then transforming it and writing it to AppEngine using the remote_api.
What error are you seeing?