My GCP app has been abused by some users. To stop their usage I have attempted to eliminate features that can be abused, and have employed firewall rules to block certain users. But bad users continue to try to access my app via certain legacy URLs such as myapp.appspot.com/badroute. Of course, I still want users to use the default URL myapp.appspot.com .
I have altered my code in the following manner, but I am still getting Instances to start from them, and I do not want Instances in such cases. What can I do differently to avoid the bad Instances starting OR is there anything I can do to force such Instances to stop quickly instead of after about 15 minutes?
class Dummy(webapp2.RequestHandler):
def get(self):
logging.info("Dummy: " )
self.redirect("/")
app = webapp2.WSGIApplication(
[('/', MainPage),
('/badroute', Dummy)],debug=True)
(I may be referring to Instances when I should be referring to Requests.)
So whats the objective? you want users that visit /badroute to be redirected to some /goodroute ? or you want /badroute to not hit GAE and incur cost?
Putting a google cloud load balancer in front could help.
For the first case you could setup a redirect rule (although you can do this directly within App Engine too, like you did in your code example).
If you just want it to not hit app engine you could setup google cloud load balancer to have the /badroute route to some file in a GCS bucket instead of your GAE service
https://cloud.google.com/load-balancing/docs/https/ext-load-balancer-backend-buckets
However you wouldnt be able to use your *.appsot.com base url. You'd get a static IP which you should then map a custom domain to it
DISCLAIMER: I'm not 100% sure if this would work.
Create a new service dummy.
Create and deploy a dispatch.yaml (GAE Standard // GAE Flex)
Add the links you want to block to the dispatch.yaml and point them to the dummy service.
Set up the Identity Aware Proxy (IAP) and enable it for the dummy service.
???
Profit
The idea is that the IAP will block the requests before they hit the dummy service. Since the requests never actually get forwarded to the service dummy you will not have an instance start. The bots will get a nice 403 page from Google's own infrastructure instead.
EDIT: Be sure to create the dummy service with 0 instances as the idea is to not have it cost money.
EDIT2:
So let me expand a bit on this answer.
You can have multiple GAE services running within one GCP project. Each service is it's own app. You can have one service running a python Flask app and another running a Java Springboot app. You can have each be either GAE Standard or GAE Flex. See this doc.
Normally all traffic gets routed to the default service. Using dispatch.yaml you can make request to certain endpoints go to a specific service.
If you create the dummy service as a GAE Standard app, and you don't actually need it to do anything, you can then route all the endpoints that get abused to this dummy service using the dispatch.yaml. Using GAE Standard you can have the service use 0 instances (and 0 costs).
Using the IAP you can then make sure only your own Google account can access this app (which you won't do). In effect this means that the abusers cannot really access the service, as the IAP blocks it before hitting the service, as you've set it up so only your Google account can access it.
Note, the dispatch.yaml is separate from any services, it's one of the per-project configuration files for GAE. It's not tied to a specific service.
As stated, the dummy app doesn't actually need to do anything, but you need to deploy it once though, as this basically creates the service.
Consider using cloudflare to mitigate bot abuse, customize firewall rules regarding route access, rate limit ips, etc. This can be combined with Google cloud load balancer, if you’d like—as mentioned in https://stackoverflow.com/a/69165767/806876.
References
Cloudflare GCP integration: https://www.cloudflare.com/integrations/google-cloud/
There is a little information I did not provide in my question about my app.yaml:
handlers:
- url: /.*
script: mainapp.app
By simply removing .* from the url specification, no Instance start is created. The user gets Error: Not Found, though. So that satisfies my needs.
Edo Akse's Answer pushed me to this answer by reading here, so I am accepting his answer. I am still not clear how to implement Edo's Answer, though.
How can I connect multiple Google App Engine apps to my one Django app engine service so that I can write to another apps datastore? Is this even possible?
Directly accessing an app's datastore from another application is possible (you don't really need to write to the app itself for that!)
The fact that the other app is also a GAE app or not doesn't really matter, setting up the access control and accessing the respective datastore are the same.
I captured the details in How do I use Google datastore for my web app which is NOT hosted in google app engine?
If you don't want to give direct datastore access to the outside app then you could implement an inter-app communication protocol to achieve what you want:
the app owning the datastore would act as a server for the other apps and would perform itself the datastore accesses on their behalf
the other apps would be clients, sending requests to the server app to get it to perform the desired actions
With this approach you can implement any access control/restriction scheme you want on the server side, which is not really possible with the direct datastore access method.
I've created a shopping site with a backend and a frontend.
The backend is python (3.6.5) with Flask.
I want to deploy my site to Google App Engine (gae).
When in development, everything works fine.
When deployed (in production) each rpc gets it's own 'thread' and everything is a mess.
I tried slapping gunicorn on it with sync and gevent worker class, but to no avail.
In deployment, how can I make each connection/session remember it's own 'instance of the backend'?
-instead of gae/flask/gunicorn serving a new instance of the backend for each request?
I need each user connection to be consistent and 'its own'/'private'.
It's not possible to do this. App Engine will spread the request load to your application across all instances, regardless of which one previously handled a request from a specific IP. A specific instance may also come online or go offline due to load or underlying changes to App Engine (e.g., a data center needs maintenance).
If you need to maintain session state between multiple requests to your app, you have a couple options depending on the architecture:
Keep session state in cookies with Flask.session
Keep session state in storage with Memorystore
We have created an app for a production facility that is very simple using django and python. But throughout prototyping we used Runserver command and localhost. The problem is this: We want to deploy the app without using localhost and the command line every time. The people using it wont be able to do this. It will be on implemented on one computer so it shouldnt be that challenging. The app pulls data from one database and stores data in another. It would be nice to have our own URL. Do we need to do it through wsgi? Apache? I know the problem is simple but there seem to be so many ways to deploy and many of them are overcomplicated for our needs.
Follow up question: I read that it just using Localhost isnt the best for this type of thing. Is this true?
Any help would be great
It sounds like you want to deploy the app live. So, I'd recommend using a dynamic hosting service like AWS/Azure/Firebase etc. If you want your own URL, purchase a domain, and in the configuration for the domain set up a CNAME file as well so you can redirect your domain to the live instance on the cloud.
Local host is better used for testing, and making changes without affecting the client and then you deploy/push to the cloud instance for production.
I've built an appeninge project so, how can I run some piece of code on the appserver only once, i.e when ever I upload the whole project on to the server.
How should I achieve this task???
There isn't an official way to discover if your application has been modified altought each time you upload your application it gets a unique version number {app version.(some unique number)} but since there isn't a document API on how to get it I wound't take a risk and use it.
What you need todo is to have a script that will upload your application and when the script is done you can call a handler in your application that set a value in the datastore that marks the application as new.
Once you have that, you can look for it in the datastore in your handlers and run the code if you find it.