I am trying to prevent all search engines bots from accessing my website.
I read that these permissions can be specified to robots.txt file but I am not sure where to access this file in order to edit it on Google Cloud.
I developed my web server using Python with Gcloud.
Note that I read the following topic
http://stocksonfire.in/seo/edit-robots-txt-google-cloud-solved/
However, I did not find any VM instance in my resources.
Do I need to create one first?
edit: This is my app.yaml file after appling #Dac Saunders suggestion
runtime: python
env: flex
entrypoint: gunicorn -b :$PORT main:app
runtime_config:
python_version: 3
handlers:
- url: /robots\.txt
static_files: robots.txt
upload: robots\.txt
My robots.txt(with python appengine) looks like this.
User-agent: *
Disallow: /q?
Sitemap: /sitemap.xml.gz
I placed it in <MYAPP>/static/robots.txt. Then I mapped it in app.yaml like this:
handlers:
- url: /(robots\.txt|favicon\.ico)
static_files: static/\1
upload: static/.*
This way my robots.txt becomes accessible in the root of my domain.
Related
I am currently trying to setup a gcloud appspot domain. I am consistently getting this error when I attempt to open up my appspot project on the browser:
Error: Not Found
The requested URL / was not found on this server.
Here is my code. How can I fix this?
app.yaml -
runtime: python
env: flex
entrypoint: gunicorn -t 120 -b :$PORT main:app
runtime_config:
python_version: 3
manual_scaling:
instances: 1
resources:
cpu: 2
memory_gb: 8
disk_size_gb: 20
handlers:
- url: /.*
script: auto
main.py
app=Flask(__name__)
api=Api(app)
.
.
.
api.add_resource(Multi,'/sentiment/<num>')
api.add_resource(Analyze,'/analyze/<query>')
api.add_resource(AddLike,'/addLike/<likedMovie>')
api.add_resource(AddDislike,'/addDislike/<dislikedMovie>')
api.add_resource(GetRecommendation,'/getRecommendation/<userID>')
if __name__=='__main__':
app.run(port=os.getenv('PORT',5000))
app.logger.addHandler(logging.StreamHandler(sys.stdout))
app.logger.setLevel(logging.ERROR)
(In between, it contains the classes and methods to serve the api calls)
and my file structure looks like:(inside root folder)
app.yaml
main.py
(other files)
What you're trying to deploy on App Engine is a web app. You cannot deploy Android apps on App Engine. If you look at the picture in the Final System section, the system has two main parts and that is your Android app (client) and a request server. You are deploying the request server in App Engine Flex (which is a managed Compute Engine VM under the hood).
From what I can understand, your request server is designed to handle backend API calls. If you want to display some sort of UI interface when users access /, then register the URL to a function that renders your view page.
I suggest that you study how Flask app routing works. You get that error because the URL (/) is not associated to any of your functions on your app. You can get started at Flask docs. Here's a sample from GitHub
So, I'm trying to host a website on the google cloud app engine, but css and other static files are not showing up.
Here is the relevant directory structure:
myapp
-app.yaml
-manage.py
subapp
-apps.py
-models.py
-urls.py
-views.py
static
-style.css
And the relevant portion of app.yaml:
runtime: python
env: flex
api_version: 1
threadsafe: yes
handlers:
- url: /static
static_dir: static
- url: .*
script: myapp.wsgi.application
I would expect that when I go to https://myapp.appspot.com/static/style.css, they see the style.css file, but instead I get the 404 page not found error, and I see that gcloud compares the 'static/style.css' against the list of urls found in urls.py instead of the static directory specified in app.yaml.
What could be going wrong?
Sadly, static handlers do not work on the flexible environment. You should just serve your CSS the way you normally would from your python application.
You can try this in your app.yaml
handlers:
- url: /static/style.css
static_files: static/style.css
upload: static/style.css
I have Google App Engine and Python running on http://localhost, which is fine working. But for making it reliably working i need to add https, Google App Engine do not have it.
So i have been trying stunnel and apache to make https working. I tried following but its still not working.
NameVirtualHost example.stackoverflow.com:443
<VirtualHost example.stackoverflow.com:443>
SSLEngine on
SSLProxyEngine On
ProxyPreserveHost On
#ProxyRequests Off
SSLProtocol all -SSLv2
SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
SSLCertificateFile /etc/stunnel/a.crt
SSLCertificateKeyFile /etc/stunnel/a.key
SSLCertificateChainFile /etc/stunnel/b.ca
ServerName localhost
ProxyPass / http://localhost
ProxyPassReverse / http://localhost
#ProxyPassReverseCookiePath /MYSITE/ /
CacheDisable *
</VirtualHost>
when a user visit: https://example.stackoverflow.com, basically his browser address get changed from https:// to http://example.stackoverflow.com and he is able to use the Google App Engine and Python instances.
But i need to have https for Google App Engine with my python. How can i resolve it please?
EDIT: secure is used but still not working
application: rtc
version: 6
runtime: python27
threadsafe: true
api_version: 1
handlers:
- url: /html
static_dir: html
- url: /images
static_dir: images
- url: /js
static_dir: js
- url: /css
static_dir: css
- url: /.*
script: rtc.app
secure: always
inbound_services:
- channel_presence
libraries:
- name: jinja2
version: latest
Google App Engine does support HTTPS, you just need to sign up for it. It is not free, but it isn't too expensive either.
More information on the types of SSL provided is available at: SSL for a Custom Domain.
Also, if you want to automatically redirect users to the secure site, you can use the secure keyword in Google App Engine's app.yaml configuration file. See Secure URLs.
I'm working on my first GAE project and I'm having some trouble getting custom pages to show up in the Admin Console. I'm following the Google Docs on it, but it doesn't seem to be working for me. I have a feeling it could have something to do with this note:
Note: Only custom pages defined by the default version will be shown
in the Admin Console.
but I'm not entirely sure what they mean (the default version of the app?).
The page URLs work fine if I visit them directly, but the links will not show in the Admin Console sidebar.
YAML:
application: namegenerator
version: 1
runtime: python27
api_version: 1
threadsafe: true
handlers:
- url: /css
static_dir: css
- url: /admin/.*
script: main.app
login: admin
- url: /.*
script: main.app
libraries:
- name: jinja2
version: latest
builtins:
- remote_api: on
admin_console:
pages:
- name: Manual DB Entry
url: /admin/db/add
- name: Clear DB
url: /admin/db/clear
Python routing:
app = webapp2.WSGIApplication([('/', MainHandler),
('/vote', SubmitVote),
('/clear_session', ClearUserSession),
('/admin/db/clear', ClearDatabase),
('/admin/db/add', ManualAddToDatabase)],
debug=True)
This is all being tested and run on my local machine, by the way. Thanks for any help.
Your code looks fine to me. Did you deploy your application? I think the admin pages will not be shown in your SDK.
By the way: An administrator can change which major deployed version of the application is default using Administration Console: https://appengine.google.com/
I am deploying a django-nonrel app on Google App Engine. The app deploys alright but I can't login to the remote shell.
This is my app.yaml file:
application: masnun
version: 1
runtime: python
api_version: 1
builtins:
- remote_api: on
inbound_services:
- warmup
handlers:
- url: /_ah/queue/deferred
script: djangoappengine/deferred/handler.py
login: admin
- url: /media/admin
static_dir: django/contrib/admin/media
expiration: '0'
- url: /.*
script: djangoappengine/main/main.py
But I am getting an error:
urllib2.URLError: <urlopen error HTTP Error 500: Internal Server Error
Couldn't reach remote_api handler at https://masnun.appspot.com/_ah/remote_api(/.*)?.
Make sure you've deployed your project and installed a remote_api handler in app.yaml.>
Please help me out!
Update: When using Python2.5, getting this error:
DEBUG:google.appengine.tools.appengine_rpc:Got http error, this is try #3
DEBUG:google.appengine.tools.appengine_rpc:Sending HTTPS request:
GET /_ah/remote_api(/.*)? HTTPS/1.1
Host: masnun.appspot.com
X-appcfg-api-version: 1
Content-type: application/octet-stream
User-agent: Google-remote_api/1.0 Linux/2.6.35-25-generic Python/2.5.5.final.0
Add this on app.yaml section handlers, first item:
handlers:
- url: /remote_api
script: $PYTHON_LIB/google/appengine/ext/remote_api/handler.py
login: admin
and deploy again.
The problem is with
GET /_ah/remote_api(/.*)? HTTPS/1.1
If you notice, the URL contains invalid characters "(/.*)?" towards the end.
Assuming you are using django-nonrel, it is an easy fix. Open the file
djangoappengine/db/base.py
and change the line
self.remote_api_path = handler.url
to
self.remote_api_path = handler.url.split('(')[0] # remove '(/.*)' introduced in newer GAE
and that should take care of ensuring the URL is correct.
You can connect to the remote shell using
python manage.py remote shell
and only if you created your App Engine app with Google Accounts Authentication. But remote_api requires a deployed app and since your
python manage.py deploy
fails, the above error is normal.
What error do you get when you try to deploy?