Develop Reference

Python reload app on Azure App Configuration change

For a Python Flask application I am using envconsul to read config from Consul KV store and inject it as environment variables into the app and watch for updates.
I am planning migration to Azure and considering switching to Azure App Configuration. Since there is nothing like envconsul (is there?) I will probably use azure-appconfiguration client library. I can see it's easy to plug in and read config, however:
With envconsul and environment variables approach it was easy to run the app locally without Consul - I just needed to set the variables and all works fine. With Azure it seems I need to mock the endpoint or wrap config loading in some abstraction layer. Or is there an easier way?
How do I manage watching for config updates? I see it's possible with SDKs for .NET, but not Python. I could write something like envazureappconfiguration ;) Or manually implement polling. Or maybe somehow pass an event through Event Grid to trigger reload...? Or, again, is there an easier way?

You are recommended to use the Python Provider library to access Azure App Configuration. The Python provider library uses the Python SDK under the cover but is designed to make it easier to use. Once data is loaded, you can access App Configuration just like a dictionary. It also offers other features like configuration composition from multiple labels, key name trimming, automatic resolution of Key Vault references, etc.
The Python provider doesn't support configuration refresh yet. It's a feature on our roadmap to add. Please feel free to open a feature request on our GitHub repo, so you will get updates. Please include details like how you expect the refresh to work. It helps us to shape the feature.

Use production App Engine datastore on development machine?

Is it possible to setup the App Engine SDK on my local machine to use the live datastore while developing? Sometimes it's just easier for my workflow to work live.
If not, is there an easy way to download or sync the live data to development machine?
Thanks!

TL;DR: We do not support having the dev_appserver use the real app-engine datastore. Even with the suggested use of "remote_api", AFAIK, the dev_appserver does not know how to use it.
If you really want to make this work, you could write your own low-level API and have your own datastore abstraction that uses your API instead of the actual datastore, however this is a non trivial amount of work.
Another option is to have a servlet that can pre-populate your dev datastore with the data you need from checked in files. The checked in raw data could be non-real data or obfuscated real data. At dev_appserver startup, you hit this URL and your database becomes pre-populated with data. If you take this route, you get the bonus of not operating on your live data with dev code.
HTH!

Instead of "touching" live data from the development server I'd recommend downloading a copy of all your production data locally with appcfg.py download_data/upload_data if you wish to move changes from development to production you can explicitly use the same commands to override existing entities.

Google-App-Engine-Like Datastore for Python

Is there any software library that provides an interface for storing and querying data like the Google App Engine Datastore, but uses a local file or service instead of running on App Engine?
The specific features I am looking for are:
Stores data as Entities with Named Properties
Query support
Atomic transactions
Python language bindings
Runs on my local machine
either stores to a single file
or connects to a local database
service
Free and open source
Thanks

You can also check MongoDB. It is an open source document-oriented database system.

You may also want to check out Appscale (http://www.appscale.com). It lets you run your App Engine apps without modification outside of Google (on your laptop, on your local cluster / behind your firewall, or in Amazon EC2). AppScale is and does each of the requirements you list here. It automatically installs/configures/manages the datastore service (and all other APIs/services) for your apps to use, so you don't have to.

Have a look at ZODB - not exactly alike but similiar http://www.zodb.org/
from the docs
Some of the features that ZODB brings to you:
Transparent persistence for Python objects
Full ACID-compatible
transaction support (including savepoints) History/undo ability
Efficient support for binary large objects (BLOBs)
Pluggable storages
Scalable architecture

Deploying an app to users' appspot

I am working on a Python App, which runs on App Engine. Is there a way I can publish the app on each customers' appSpot account, so that the App uses the users' cloud storage? Instead of running the App on my AppSpot account and all the users storing the data on my Cloud space?

Yes, absolutely.
You just need to have each client create an App Engine account with an application to which you have administrator access. You can adjust the settings on the application to forbid downloads of your code by the other administrators if that's appropriate for your agreement with the client. This also allows the clients to be billed directly for their instances' usage, and makes it completely impossible for data to leak between different clients' instances.
Using multiple applications for multiple clients who are licensing your application almost certainly does not violate part 4.4 of the TOS, although don't take this as legal advice.

No, you cannot do that. The app is hosted and run in the administrator's account which would be you. What you can do is, release the source code and point your users do install it in their appspot account, just like creating a new application.

I suppose it's not exactly what you need. But it can give you an idea where to go. Please check DryDrop project. There is small Python application you can ask each user to install on their account, then they can configure it to fetch your site files from your GitHub repo through webhooks functionality. I didn't try it, but, theoretically, you update your site, commit it to your repo, and all users get your updated application automatically. You can share your thoughts if that works for you.

Maybe. If it's an open source app that you're giving away, you can publish the source and instruct users to upload it to their own accounts.
If you're selling the app, displaying ads or otherwise trying to monetize the service, you probably want to stick with one instance. Using multiple instances to avoid paying for quota usage is direct violation of the App Engine TOS:
4.4. You may not develop multiple Applications to simulate or act as a
single Application or otherwise access
the Service in a manner intended to
avoid incurring fees.

No. Writing an application that deploys other applications is in violation of the terms of service.
Note we don't have any 'hard' limits - those limits that aren't billing enabled can be increased on application to us if you provide a reasonable use-case.

Migrating Django Application to Google App Engine?

I'm developing a web application and considering Django, Google App Engine, and several other options. I wondered what kind of "penalty" I will incur if I develop a complete Django application assuming it runs on a dedicated server, and then later want to migrate it to Google App Engine.
I have a basic understanding of Google's data store, so please assume I will choose a column based database for my "stand-alone" Django application rather than a relational database, so that the schema could remain mostly the same and will not be a major factor.
Also, please assume my application does not maintain a huge amount of data, so that migration of tens of gigabytes is not required. I'm mainly interested in the effects on the code and software architecture.
Thanks

Most (all?) of Django is available in GAE, so your main task is to avoid basing your designs around a reliance on anything from Django or the Python standard libraries which is not available on GAE.
You've identified the glaring difference, which is the database, so I'll assume you're on top of that. Another difference is the tie-in to Google Accounts and hence that if you want, you can do a fair amount of access control through the app.yaml file rather than in code. You don't have to use any of that, though, so if you don't envisage switching to Google Accounts when you switch to GAE, no problem.
I think the differences in the standard libraries can mostly be deduced from the fact that GAE has no I/O and no C-accelerated libraries unless explicitly stated, and my experience so far is that things I've expected to be there, have been there. I don't know Django and haven't used it on GAE (apart from templates), so I can't comment on that.
Personally I probably wouldn't target LAMP (where P = Django) with the intention of migrating to GAE later. I'd develop for both together, and try to ensure if possible that the differences are kept to the very top (configuration) and the very bottom (data model). The GAE version doesn't necessarily have to be perfect, as long as you know how to make it perfect should you need it.
It's not guaranteed that this is faster than writing and then porting, but my guess is it normally will be. The easiest way to spot any differences is to run the code, rather than relying on not missing anything in the GAE docs, so you'll likely save some mistakes that need to be unpicked. The Python SDK is a fairly good approximation to the real App Engine, so all or most of your tests can be run locally most of the time.
Of course if you eventually decide not to port then you've done unnecessary work, so you have to think about the probability of that happening, and whether you'd consider the GAE development to be a waste of your time if it's not needed.

Basically, you will change the data model base class and some APIs if you use them (PIL, urllib2, etc).
If your goal is app-engine, I would use the app engine helper http://code.google.com/appengine/articles/appengine_helper_for_django.html. It can run it on your server with a file based DB and then push it to app-engine with no changes.

It sounds like you have awareness of the major limitation in building/migrating your app -- that AppEngine doesn't support Django's ORM.
Keep in mind that this doesn't just affect the code you write yourself -- it also limits your ability to use a lot of existing Django code. That includes other applications (such as the built-in admin and auth apps) and ORM-based features such as generic views.

There are a few things that you can't do on the App Engine that you can do on your own server like uploading of files. On the App Engine you kinda have to upload it and store the datastore which can cause a few problems.
Other than that it should be fine from the Presentation part. There are a number of other little things that are better on your own dedicated server but I think eventually a lot of those things will be in the App Engine