I have an appengine app that I'm happy with and I localized it to several countries and languages. Now I want to localize it for China and Hong Kong but I believe that these areas block appengine so that google cannot be used. What is then the best development plan to make my app available in China? Should I change to a different backend (django + mysql?) and deploy it to some Hong Kong or China hosting or is there a simpler way? I use gae blobstore, gae models, the gae search API, memcache, gnu gettext, jinja2 and the gae mail API (but no compute engine). So the best way I can think of would be to migrate the backend to something than can run and get accessed from China and Hong Kong. Or is there a better way?
There are a few different options:
If you're planning on deploying a separate, localized version for China (with a separate data set), you can use AppScale and running in a Chinese public cloud (Aliyun is pretty easy to get started with). This ensures accessibility for your users in China, but means that this deployment would function as an isolated, China-specific deployment.
If you want to have users around the globe (including Chinese/Hong Kong users) access a single app with a shared backend, you could:
keep the app in GAE and use a reverse proxy (such as CloudFlare) to
get around the firewall issues
use AppScale to run the app (unmodified) in a different public cloud, such as EC2
rewrite the app
In my experience, a reverse proxy works is only a viable solution if performance isn't crucial. For a user-facing web app like KoolBusiness, it might pose a few challenges latency-wise. But I work for AppScale, so most of the users I talk with are already exploring options other than reverse proxy.
Proxying your website will be easier than moving to a different host. You could try using CloudFlare for this.
A potentially simpler option is just mapping the app to a custom domain, see https://stackoverflow.com/a/19093384/4495081.
I see used the http://www.greatfirewallofchina.org tool to check your app and it is reported all green:
http://www.greatfirewallofchina.org/index.php?siteurl=http%3A%2F%2Fwww.koolbusiness.com
There are different solutions available according to your budget:
1- Host your servers in Hong Kong. Contact the hosting platform and inform them that you need your data to be available in China, they would be able to host your data on the fastest data center (the one with the fastest connection to Mainland China). Won’t be the fastest option available but it will be the cheapest option.
2- Use Public Cloud providers such as AWS or Microsoft Azure (outside of China) + their respective CDNs. Most of the cloud providers have endpoints in Hong Kong and you will be able to get an acceptable band-with. Will be cost-efficient.
3- Use Chinese Cloud providers such as AliCloud (China branch, not international), AWS China or Azure China. For that, you will need to have a Chinese company established and you may need to get an ICP license (https://en.wikipedia.org/wiki/ICP_license if you have public content). Expensive but that’s the best option if you decide to operate in China for the long run.
4- Use your own backend and add a professional CDN such as Akamai, they have an option called China CDN that will help you go through the Great Firewall (budget +/- 10k$ per month). Will be very expensive.
Related
Let's say you have some proprietary python + selenium script that needs to run daily. If you host them on AWS, Google cloud, Azure, etc. are they allowed to see your script ? What is the best practice to "hide" such script when hosted online ?
Any way to "obfuscate" the logic, such as converting python script to binary ?
Can the cloud vendors access your script/source code/program/data?
I am not including government/legal subpoenas in this answer.
They own the infrastructure. They govern access. They control security.
However, in the real world there are numerous firewalls in place with auditing, logging and governance. A cloud vendor employee would risk termination and/or prison time for bypassing these controls.
Secrets (or rumors) are never secret for long and the valuation of AWS, Google, etc. would vaporize if they violated customer trust.
Therefore the answer is yes, it is possible but extremely unlikely. Professionally, I trust the cloud vendors with the same respect I give my bank.
Here you can find information regarding Google Cloud Enterprise Privacy Commitments.
It is described how Google protect the privacy of Google Cloud Platform and Google Workspace customers.
You control your data. Customer data is your data, not Google’s. We
only process your data according to your agreement(s).
We never use your data for ads targeting. We do not process your
customer data or service data to create ads profiles or improve Google
Ads products.
We are transparent about data collection and use. We’re committed to
transparency, compliance with regulations like the GDPR, and privacy
best practices.
We never sell customer data or service data. We never sell customer
data or service data to third parties.
Security and privacy are primary design criteria for all of our
products. Prioritizing the privacy of our customers means protecting
the data you trust us with. We build the strongest security
technologies into our products.
Therefore I believe is extremely unlikely that Google will check the content of your scripts.
In my project we are trying to create micro service type architecture using djnago/django-rest-framework.
We've some services like:
User management service
Asset Management Service
Tool management service
All three services running on different Ports with different database.
Now my question is,
Can we use user management service in Asset and tools service for Token authentication?
This is quite a broad question and without knowing more of your specific architecture, what you have tried and what features you want/need, it will be hard to give anything more than a starting point.
That being said, you could look at setting up a Central Authentication Service. (CAS) There's multiple packages for Django that help you with this.
Try looking into Django CAS NG and Django MamaCAS. Django CAS NG seems to be the more actively developed of the two.
Remember that with microservices, one of the big advantages is that there's nothing forcing you to use the same technology across your entire stack. It may well make sense to have your auth components provided by something entirely different, for example rolling a KeyCloak server to handle SSO.
I'm working on my last semester project which is based on real-time transit. So, I want to develop website (using Python + Django Fw) that will show all our university buses' real-time Geo-location (Google map API or any other please suggest the best one) to the university students and professors, and also shows the bus arrival times for bus stop nearest to them (From wherever they are). This is main module of my project. So the question is that
May I use Google transit for that? Because the statement
"If you provide a transportation service that is open to the public,
and operates with fixed schedules and routes, we welcome your
participation"
in Google's documentation on
"Why Google Maps?"
So in this case my website can only be accessible by our university persons, and I think it may violet Google's policy. And second question,
If I implement this is it accessible from any place other than my website? If it is, then it will become meaning less for those who are not our university persons and so may cause nasty situations.
About Q1:（May I use Google transit for that? ...）
You might be able to use Google transit by using Google Maps Directions API(https://developers.google.com/maps/documentation/directions/).
But, there are several limitations. Transportation agency must be in a supported city(http://www.google.com/landing/transit/cities/index.html) and a public transportation agencies(https://developers.google.com/transit/gtfs-realtime/).
About Q2:（If I implement this is it accessible ...）
Other peoples may access your transit feed info.
But, there are Quality Assurance Review(https://support.google.com/transitpartners/answer/1106422?hl=en&ref_topic=1095593) about the info before launch it.
I am working on a Python App, which runs on App Engine. Is there a way I can publish the app on each customers' appSpot account, so that the App uses the users' cloud storage? Instead of running the App on my AppSpot account and all the users storing the data on my Cloud space?
Yes, absolutely.
You just need to have each client create an App Engine account with an application to which you have administrator access. You can adjust the settings on the application to forbid downloads of your code by the other administrators if that's appropriate for your agreement with the client. This also allows the clients to be billed directly for their instances' usage, and makes it completely impossible for data to leak between different clients' instances.
Using multiple applications for multiple clients who are licensing your application almost certainly does not violate part 4.4 of the TOS, although don't take this as legal advice.
No, you cannot do that. The app is hosted and run in the administrator's account which would be you. What you can do is, release the source code and point your users do install it in their appspot account, just like creating a new application.
I suppose it's not exactly what you need. But it can give you an idea where to go. Please check DryDrop project. There is small Python application you can ask each user to install on their account, then they can configure it to fetch your site files from your GitHub repo through webhooks functionality. I didn't try it, but, theoretically, you update your site, commit it to your repo, and all users get your updated application automatically. You can share your thoughts if that works for you.
Maybe. If it's an open source app that you're giving away, you can publish the source and instruct users to upload it to their own accounts.
If you're selling the app, displaying ads or otherwise trying to monetize the service, you probably want to stick with one instance. Using multiple instances to avoid paying for quota usage is direct violation of the App Engine TOS:
4.4. You may not develop multiple Applications to simulate or act as a
single Application or otherwise access
the Service in a manner intended to
avoid incurring fees.
No. Writing an application that deploys other applications is in violation of the terms of service.
Note we don't have any 'hard' limits - those limits that aren't billing enabled can be increased on application to us if you provide a reasonable use-case.
We've had some good experiences building an app on Google App Engine, this first app's target audience are Google Apps users, so no issues there in terms of it being hosted on Google infrastructure.
We like it so much that we would like to investigate using it for another app, however this next project is for a client who is not really that interested in what technology it sits on, they just want it to work, and work all of the time.
In this scenario, given that we have the technology applicability and capability side covered, are there any concerns that this stuff is still relatively new and that we may not be as much "in control" as if we had it done with traditional hosting?
You are correct: you are not in as much control vs. traditional hosting. However, hopefully the gains outweight the negatives. App Engine is extremely scalable -- it runs on the same hardware that runs Google itself. How often have you visited http://google.com and had that page or a search result fail?
Although you are letting Google run your code, the code is still your's to do as you please. With new projects like django-nonrel, you can create and run native Django apps directly on top of App Engine, and if it doesn't meet your needs down the line, it's fairly easy to take that app to an ISP that hosts Django apps (and there are plenty of those). More on this project below.
You don't have to worry about hardware, operating systems, coming up with a machine image, databases, web servers, front-end load balancers, CDNs/edge caching, software/package upgrades, license fees, etc. All these things are tangential to the web or other application that you have or will create to solve a particular problem. All this additional infrastructure is required whether you like it or not; but with App Engine, you only need to think about your app/solution and none of this extra stuff.
Obviously another thing you lose is some of your execution environment. To ensure that you're playing nicely with your cloud neighbors (resource hogging, security issues, etc.), you must execute in a sandbox, meaning your app cannot create local files, open network sockets, etc. However, App Engine provides a rich set of APIs and product features so that you at least can create meaningful apps:
scalable distributed object datastore (see below)
Memcache
URLFetch
images service (resize, crop, etc.)
users service/authentication task queues for background processing
Django web templating
blobstore for large files
denial-of-service blacklisting
transational tasks
datastore cursors
sending (and/or receiving) of email
sending (and/or receiving) of chat/IM/instant messages via XMPP
You also have a full dashboarded administration console which will let you monitor your app's usage, your billing settings and history, a full dump of your quota usage, and even your application logs which you can view or download.
To address the "main sore points" from #Anurag:
1a. the free quotas are fairly generous... enough to power a website that gets 5MM views/month. also, if you trust Google to give them your credit card, they will bump up the free quota levels even higher. look at their quota page and refer to the numbers in both the "Free Default Quota" and "Billing Enabled Default Quota" columns... here are some examples: a) # of Requests: 1.3MM default, 43MM w/billing enabled (wBE), b) Datastore API calls: 10MM default, 140MM wBE, c) URL Fetches: 657K default, 46MM wBE
1b. 30s max for requests: this is more security for you, because your app is now in a playground with others. Google has to ensure that all cloud neighbors play nicely with each other and not hog the CPU. However, the App Engine team is working on a way to allow for longer running background tasks... there's no timetable yet, but it is on the public roadmap.
1c. writing a chat server on App Engine is not only possible, but it is quite simple. here's one created using App Engine's XMPP API -- it's pretty dumb and just echoes back to the sender what they transmitted to us (be aware that you must have already invited the user to chat):
from google.appengine.api import xmpp
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class XMPPHandler(webapp.RequestHandler):
def post(self):
msg = xmpp.Message(self.request.POST)
msg.reply("I got your msg: '%s'" % msg.body)
application = webapp.WSGIApplication([
('/_ah/xmpp/message/chat/', XMPPHandler),
], debug=True)
def main():
run_wsgi_app(application)
if __name__ == '__main__':
main()
1d. another item on the the public roadmap is future "[support] for Browser Push (Comet) communication", so that's coming too.
2a. "not SQL" is one of Google App Engine's greatest strengths! relational databases don't scale and must be sharded at some point to keep an RDBMS from falling over. it is true however, that it is slightly more difficult to port because it is not traditional! Based on Google Bigtable, you can think of the App Engine datastore as a scalable distributed object database. App Engine lets you query the datastore using a Query object model, or if you insist, they also provide a SQL-like GqlQuery interface.
2b. with new avantgarde projects like django-nonrel, if you create a Django app and use its ORM, you can take a pure Django app and run it directly on top of App Engine. likewise, you can it off of App Engine and move it directly to more traditional ISP vendor that hosts Django applications. the queries stay exactly the same, and you don't have to care if it does SQL or not.
3a. long-running processes are already addressed in 1b above. Google is aware of this need and are working on it.
3b. the TaskQueue API supports 100k calls, but that's bumped to 1MM wBE... and this is on a daily basis.
3c. Google strongly encourages breaking up tasks into multiple subtasks. low latency apps are seen not to "hog the system" and are given better treatment than those which are slow and consume more resources from their cloud neighbors.
Yes, you would not be in as much control as with traditional hosting. Main sore points of GAE are
Quotas etc, 30 sec max for a request, so comet/reverse ajax etc out of window or very difficult. Try writing a chat server on google app engine.
Not Sql database, so difficult to port to other server if need be and sometime limitation with google database e.g. try sorting a query which has comparison on different column other than the sorted one.
Long running process, there is a Task api but that doesn't suffice if you want to do long running background processing, otherwise you will have to break your task in subtasks, so things get complicated and there are even quotas on how many tasks per sec you can run.
GAE is good if you app can be modeled as request-response registry, with little background processing.
See this too
Feedback on using Google App Engine?