What are the different options, with pros and cons, for periodically adding records to a Django app hosted on GAE?
Use a custom django management command on the remote datastore
Write an API in Django that exposes the datastore to be updated
Use a cron task on GAE to update
(am I missing anything else?)
1: Custom Django management command on "remote"
I'm currently using #1: django-nonrel on GAE and using custom management/django-admin commands for my models. For example, this is how I call my custom management command on the remote datastore:
manage.py remote mycommand
The advantage of this command is ease of development: I can test the the management command locally and simply add "remote" to use it on GAE.
2: Write an API in Django that exposes the datastore
I would have to use an extra server with cron to update.
3: Use a cron task in Google
I don't know how GAE likes having its users run a scraper periodically. Also, GAE doesn't have a real cron -- it simply hits a URL at a set intervals.
Use a cron job. That's what they're designed for. Whether or not scraping is okay depends on the terms of service on the site you're scraping.
Related
I am trying to deploy a Django REST API on Heroku. Normally I wouldn't have any issues with this but for this app, I am using a legacy database that exists on AWS. Is it possible for me to continue to use this remote database after deploying Django to Heroku? I have the database credentials all set up in settings.py so I would assume that it should work but I am not sure.
It should not pose any problem to connect with an database on AWS.
But be sure that the database on AWS is configured to accept external access, so that Heroku can connect.
And I would sugest that you take the credentials out of the source code and put it in the Config Vars that Heroku provide (environment variables).
Will it work? I think yes, provided you configure your project and database for external access.
Should you want it? How may queries does an average page execute? Some applications may make tens of queries for every endpoint and added wait can combine into seconds of waiting for every request.
Flask doesn't have any available add-ins to be able to build business process management capabilities
Django has many add-ins available for these capabilities. The reason being, Django is pre-built with user management and permissions, making it possible to build libraries. In our case, we migrated to FAB to get these capabilities.
The examples are viewflow (http://viewflow.io/) , (http://demo.viewflow.io/workflow/shipment/shipment/start/) and activflow (https://github.com/faxad/ActivFlow)
Any ideas if there exist libraries to help implement the process in Flask or is there a plan to implement them in FAB
I have an existing .NET Core / asp.net app service hosted on Azure. I need to call (on demand) a python script to return data based on custom user input.
It does not appear that I can use IronPython, since I need python modules that are built in CPython, which unfortunately aren't supported by IronPython.
The two options I see are:
I might be able to install the right python version and libraries on the app service and call it from the .NET code. This seems like it might be deprecated: https://learn.microsoft.com/en-us/visualstudio/python/publishing-python-web-applications-to-azure-from-visual-studio?view=vs-2017
I can create a whole new and separate app service for just the python script and call it as a REST API on demand from the .NET app service. This seems like overkill, and introduces the problem of opening up a whole new service publicly, which I don't want to do. This also appears to have the limitation that Flask isn't mean for production, so hosting many calls at once is not really workable. https://learn.microsoft.com/en-us/visualstudio/python/publish-to-app-service-windows?view=vs-2017
What is the best way to call a python script on demand from .NET app service on Azure?
Per my experience, there are two ways to call a Python script in C# without IronPython.
Directly use System.Diagnostics.Process in C# to run a command as same as the SO thread Run Command Prompt Commands to get the result via parse the content of the process standard output. Simply to do it, you can use py2exe to wrap a Python script as a .exe file to avoid for installing Python modules and setting environment variables on Azure App Service. However, considering for concurrency, it's not a good idea for performance.
The second option as you said is to deploy a Python script as a REST API in the same instance of Azure Web App. You can follow the blog Deploying multiple virtual directories to a single Azure Website to deploy a flask app with your Python script as a child project via Visual Studio with PTVS to expose an API url like https://<your web app name>.azurewebsites.net/pyapi which can be called from your ASP.NET via HttpClient. I tried this solution, it works.
Note: Due to the restriction of Azure Web App sandbox for Local Address Requests, you have to use <your web app name>.azurewebsites.net as hostname, neither localhost or 127.0.0.1.
I have successfully developed a Django app. The requirement is to deploy the app over an on-premise server. The application is accessible on the intranet using the development environment. Is any web server is preferred to deploy this application locally (or) should I leave the terminal running the server as it is so that the users can access the application as they are already doing it? I am using a unix server.
If the question is actually on deployment model, I suggest to take a look at django docs on deployment. Depending on load, uWSGI/Gunicorn [+ nginx] will be good choice.
Independent of tools you use, no need to leave running terminal on your server. There's a lot of tools to "daemonize" processes. Simplest would be supervisor
I just deployed a site on GAE which requires me to stage some data for dropdown fields (i.e. us states, status, etc.).
In development, I have created an entity for each type of data (US State entity for example) and was able to preload the data using the interactive console by creating the entity and then calling the put() method.
Now that the application is deployed I don’t know of a way to preload this data. How would you recommend doing this in a deployed instance?
I am using SDK version 1.7.0, python 2.7, High Replication Datastore (HRD), and memcache when data is retrieved.
Thanks in advance for your help!
If you want to do it programmatically, you may use the interactive console in production. Check out How do I activate the Interactive Console on App Engine?
You may also create a temporary request handler that'll do the job, deploy it (e.g. as a different version of the app to make it easy to delete) and launch the respective URL in your browser.
You can use the bulkloader to upload your entities to your deployed version. See the doc Uploading and Downloading Data for details and examples.