Develop Reference

How can I create random urls in gae?

I want to create random urls. i mean , let first url be myapp.appspot.com/868DF7.html
it ll be connected to test.py in app.yaml. when user opens this url, test.py ll generate new url. I ll save the next url at datastore. and the previous url will not be accessible. I guess the word is "disposable".
Is that possible to create dynamic url like this?
my current test.py : it gets previous string from datastore and creates next url and saves it.
import webapp2
from google.appengine.ext import db
from google.appengine.api import users
import jinja2
import os
import uuid
class Saved(db.Model):
urls=db.StringProperty()
date = db.DateTimeProperty(auto_now_add=True)
def _rKey(r_name=None):
return db.Key.from_path("r", r_name or 'default_r')
class MainPage(webapp2.RequestHandler):
def get(self):
r_name="none"
saveds_query = Saved.all().ancestor(
_rKey(r_name)).order('-date')
urls = saveds_query.fetch(10)
q=db.Query(Saved).order('-date')
print "previous url:", q[0].urls
print "<br>"
save = Saved(parent=_rKey(r_name))
save.urls=str(uuid.uuid4().get_hex().upper()[0:6])+".html"
print "next url:",save.urls
save.put()
APP = webapp2.WSGIApplication([
('/give/', MainPage),
], debug=True)
and app.yaml
handlers:
- url: /favicon\.ico
static_files: favicon.ico
upload: favicon\.ico
- url: /.*
script: helloworld.APP
libraries:
- name: webapp2
version: "2.5.2"
- name: jinja2
version: "2.6"
rest of the files are pretty much the same guestbook example of google.
https://github.com/GoogleCloudPlatform/appengine-guestbook-python/blob/master/guestbook.py

app.yaml lists "rules" that a server uses to match handlers. These rules may use wildcards and regular expressions. For example, you can add the following rule:
- url: /movie/.*
A server would interpret this rule to send all matching URLs to a file or a script that you specify. For example, the following URLs would match this rule:
myApp.appspot.com/movie/1234
myApp.appspot.com/movie/1234.mp4
You can use regular expressions to be as specific as you need in your matching rules.

You should define a handler that is active on a wildcard or a regular expression that matches the format you chose for the random strings.
When you get a request there, have the handler check the specific route that was used, and validate it against (as you correctly noted) the Datastore, Cloud SQL, your own Redis server, etc., which stores resources or references to resources that should be accessible from that special route.
You would then have the handler serve the resource, or at that point, if you want/need, validate authentication somehow (hopefully OAuth of their google account)

Requested url not found on server error

I made a website which I am temporarily hosting in app engine. I am able to navigate through the pages when I open the HTML files on my computer. It only fails when I head to http://www.alshafarconstruction.appspot.com/index.html.
Error Message:
Error: Not Found
The requested URL /contact.html was not found on this server
app.yaml:
application: alshafarnew
version: 1
runtime: python
api_version: 1
handlers:
- url: /(.*\.(gif|png|jpg|ico|js|css|swf|xml))
static_files: \1
upload: (.*\.(gif|png|jpg|ico|js|css|swf|xml))
- url: /(.*\.html)
static_files: \1
upload: index.html
- url: /.*
script: main.py
main.py:
from google.appengine.ext import webapp
from google.appengine.ext.webapp.util import run_wsgi_app
class IndexHandler(webapp.RequestHandler):
def get(self):
if self.request.url.endswith('/'):
path = '%sindex.html'%self.request.url
self.redirect(path)
def post(self):
self.get()
application = webapp.WSGIApplication([('/.*', IndexHandler)], debug=True)
def main():
run_wsgi_app(application)
if __name__ == "__main__":
main()
I've checked the dashboard and find that the HTML files are there. I am new to website deployment and design. Can anyone suggest possible errors? I have already visited many sites for a solution.

The documentation hints that static and application files are uploaded separately, so while the files may be there, it might be that they're being uploaded in the application-files part.
You currently have an upload directive telling it to only upload index.html:
- url: /(.*\.html)
static_files: \1
upload: index.html
(actually, it's a regular expression, so it would also upload, for example index#html if it existed)
This suggests that you might want to make that regular expression match all HTML files:
- url: /(.*\.html)
static_files: \1
upload: .*\.html

Error with new file, 404 Not Found, App Engine for Python

I have a website running on Google App Engine for Python. I've inherited the code-base and am trying to piece things together myself. I'm trying to accomplish the "simple" task of creating a new page on the website and linking to it from a toolbar. So, I created a new file, lets call it mypage.html, in the main directory.
Here's the code I use for linking, within the index.html file:
<li {% if mypage_selected %}class='active'{% endif %}>
<a href='/mypage'>My Page</a>
</li>
And here's the code I use for handling the link, as declared in my main.py file:
class MyPagePage(BaseHandler):
def get(self):
template_values = {
'mypage_selected': True,
'session': self.session,
}
self.response.out.write(
template.render(get_path("mypage.html"), template_values))
Finally, I added a reference to the page here (also in main.py):
app = webapp2.WSGIApplication([
('/', MainPage),
...
('/mypage/?', MyPagePage),
], debug=True, config=webapp2conf)
My new file is indeed at mypage.html. However, when I run this, the link shows up, but clicking it takes me to http://mydomain.net/mypage which gives a 404 Not Found error. Any ideas what could be happening?
EDIT: Here is the handler portion of my app.yaml:
handlers:
- url: /
script: main.app
- url: /index\.html
script: main.app
- url: /static
static_dir: static
- url: /api.*
script: api.app
- url: /blog/.*/edit/?
script: dj.app
- url: /.*
script: main.app
On an additional note: I tried navigating to mypage.html, and that gives me a 404 also. Very stumped here. Any help is greatly appreciated!

Note the /mypage/? in your handler. That at least has to match, but your link is to '/mypage' so you get a 404.

a few things.
I would highly recommend using a boilerplate application like gae-boilerplate. This will help with a lot of these first time problems with google appengine. It's lightweight and will get you started on making your own projects quickly. The reason I recommend it is because by default it resolves strict slash problems (the difference between /mypage and /mypage/) by using
from webapp2_extras.routes import RedirectRoute
from web.handlers import handlers
secure_scheme = 'https'
_routes = [
RedirectRoute('/', handlers.MainPage, name='main', strict_slash=True),
RedirectRoute('/mypage/', handlers.mypagepage, name='mypage', strict_slash=True),
...
and in your handlers you could have:
class mypagepage(BaseHandler):
#user_required
def get(self):
params = {}
return self.render_template('mypage.html', **params)

Google App Engine WebApp/Python Custom 404 Handler Implementation

I'm using GAE, Google App Engine with Webapp using Python.
I've been trying to implement a custom error handler, or template, or along those lines. GAE provides some documentation here however it doesn't provide enough in the example for implementation (for me).
I've also looked at all these wonderful examples on another StackOverflow question here - but cannot understand how to implement it into my current main.py file.
import os
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp import template
class MainHandler(webapp.RequestHandler):
def get (self, q):
if q is None:
q = 'static/index.html'
path = os.path.join (os.path.dirname (__file__), q)
self.response.headers ['Content-Type'] = 'text/html'
self.response.out.write (template.render (path, {}))
def main ():
application = webapp.WSGIApplication ([('/(.*html)?', MainHandler)], debug=False)
util.run_wsgi_app (application)
if __name__ == '__main__':
main ()
I've tried making another class, and implementing it before the MainHandler as well as a new def in the MainHandler. The only times I've gotten the 404 to display, it's displayed globally, meaning even the index.html file was a "404".
The last thing I tried was implementing something along the lines of:
if not os.path.exists (_file_):
self.redirect(/static/error/404.html)
My app.yaml file is:
application: appname
version: 1
runtime: python
api_version: 1
error_handlers:
- file: static/error/404.html
- error_code: over_quota
file: static/error/404.html
handlers:
- url: /(.*\.(gif|png|jpg|ico|js|css))
static_files: \1
upload: (.*\.(gif|png|jpg|ico|js|css))
- url: /robots.txt
static_files: robots.txt
upload: robots.txt
- url: /favicon.ico
static_files: favicon.ico
upload: favicon.ico
- url: .*
script: main.py
I cannot find any guides/tutorials to the implementation of the 404 handler, just code extracts.
Much appreciated everyone!
Edit:
As per clarification from Hans below, I want to return a 404 when a file (or directory) is not present in the filesystem.
I've tried:
class ErrorHandler(webapp.RequestHandler):
def get(self):
if not os.path.isfile(_file_):
self.redirect('/static/error/404.html')
to no avail. What's the correct implementation/syntax?
Edit:
To Voscausa
import os
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp import template
import logging
import webapp2
def handle_404(request, response, exception):
logging.exception(exception)
response.write('Oops! I could swear this page was here!')
response.set_status(404)
def handle_500(request, response, exception):
logging.exception(exception)
response.write('A server error occurred!')
response.set_status(500)
app = webapp2.WSGIApplication([
webapp2.Route('/', handler='handlers.HomeHandler', name='home')
])
app.error_handlers[404] = handle_404
app.error_handlers[500] = handle_500
class MainHandler(webapp.RequestHandler):
def get (self, q):
if q is None:
q = 'static/index.html'
path = os.path.join (os.path.dirname (__file__), q)
self.response.headers ['Content-Type'] = 'text/html'
self.response.out.write (template.render (path, {}))
def main ():
application = webapp.WSGIApplication ([('/(.*html)?', MainHandler)], debug=True)
util.run_wsgi_app (application)
if __name__ == '__main__':
main ()

Have a look at: http://webapp-improved.appspot.com/guide/exceptions.html for handling 404 and 500 exceptions.
Webapp2 is now the preferred framework for GAE python applications.

SOLUTION 1: Have appengine take care of your static files
You can have GAE handle your static files for you, like this:
application: appname
version: 1
runtime: python
api_version: 1
handlers:
- url: /static
static_dir: static
- url: .*
script: main.py
This is better as serving static files is much easier when it is done for you, and it will also not count as much towards your CPU quota.
There is just one warning. You cannot specify a custom error handled in case a file is not found below /static. As soon as a handler picks up on a url, it will try to finish the request. If the static dir handler cannot find the url, it will not hand control back to main.py, which is now your wildcard solution. It will simply return the default 404 error page from appengine.
SOLUTION 2: Serve static files from your application
If you really must create a custom 404 page for your static you have to do it in your application. (I do not advice you to do so, in case that's not clear.) In that case you were on the right track. You should create an extra handler like in your example. Just rewrite one line:
path = os.path.join(os.path.dirname(__file__), 'static', q)
In you application line, add your handler like this:
('/static/(.*)?', MainHandler),
If the file is not found, you should call self.response.set_status(404) and write whatever custom response message you need.

Google App Engine static pages Python 2.5 directories etc

I am new at this and am planning to move my stuff from a paid web service to GAE (nothing dynamic just static pages). Believe me I have spent countless hours trying to make this work but am at an impasse whereby I achieve one result at the exclusion of another and visa versa.
I am sure it is a simple answer and that I am violating some basic principles. What I want is that the app engine page (mygaeid.appspot.com) delivers a static landing page such that other pages are available with the addition of a suffix e.g. mygaeid.appspot.com/oranges.html mygaeid.appspot.com/grapes.html etc.
I am unable to achieve this such that I either am able to get the other pages when I add the suffix e.g. mygaeid.appspot.com/apples.html; mygaeid.appspot.com/oranges.html but not the landing page OR with a slightly different yaml the landing page (mygaeid.appspot.com) works but there is no access to the other pages (mygaeid.appspot.com/oranges.html etc) that have a suffix.
The py file (abcdefg.py) is below and is common to the two different yamls that follow:
import os
from google.appengine.ext import webapp
from google.appengine.ext.webapp import util
from google.appengine.ext.webapp import template
class MainHandler(webapp.RequestHandler):
def get (self, q):
if q is None:
q = 'htmdir/apples.html'
path = os.path.join (os.path.dirname (__file__), q)
self.response.headers ['Content-Type'] = 'text/html'
self.response.out.write (template.render (path, {}))
def main ():
application = webapp.WSGIApplication ([('/(.*html)?', MainHandler)], debug=True)
util.run_wsgi_app (application)
if __name__ == '__main__':
main ()
Using the following yaml the landing page (mygaeid.appspot.com) works perfectly (delivering the content of apples.html), but I cannot access the page if I add a suffix e.g. mygaeid.appspot.com/apples.html or mygaeid.appspot.com/static/htmdir/apples.html etc, as soon as I add the suffix it does not work. In the directory (htmdir) I have placed apples.html along with other html pages e.g. oranges.html etc and I cannot access any of them with this yaml.
application: mygaeid
version: 1
runtime: python
api_version: 1
handlers:
- url: /(.*\.(html))
static_files: static/htmdir/\1
upload: static/htmdir/(.*\.(html))
- url: /css
static_dir: css
- url: /js
static_dir: js
- url: /images
static_dir: images
- url: .*
script: abcdefg.py
If I modify the yaml as follows then the landing page (mygaeid.appspot.com) does not work but when I add the suffixes then it works perfectly e.g. mygaeid.appspot.com/apples.html; mygaeid.appspot.com/oranges.html etc deliver the appropriate pages:
- url: /(.*\.(html))
static_files: htmdir/\1
upload: htmdir/(.*\.(html))
Finally if I dispense with the directories altogether and using the same abcdefg.py (without the directory) the following very simple yaml actually delivers the results I want but is very unruly as all the files are stuffed in the root directory.
application: mygaeid
version: 1
runtime: python
api_version: 1
handlers:
- url: /(.*\.(png|js|css))
static_files: \1
upload: (.*\.(png|js|css))
- url: .*
script: abcedfg.py
any help would be much appreciated on figuring this out.
thanks
thanks wooble and thanks dave I went back yet again and carefully read the logs Wooble's solution works but I needed to put the htmdir (that contains the html) inside a directory called static. GAE is a great (and free) solution for static websites
your help and feedback is very much appreciated
SiteDirectory
-mygaeid
-static
-htmdir
-js
-css
-images
app.yaml
index.yaml
(py file was removed)

If you declare files as static in app.yaml, they are not available to your application's handlers.
However, if they're really static, using the django template engine to "render" them is kind of silly; just add mappings in app.yaml to display the static files, including one to display apples.html for /:
application: mygaeid
version: 1
runtime: python
api_version: 1
handlers:
- url: /(.*\.html)
static_files: static/htmdir/\1
upload: static/htmdir/.*\.html
- url: /css
static_dir: css
- url: /js
static_dir: js
- url: /images
static_dir: images
- url: /
static_files: static/htmdir/apples.html
upload: static/htmdir/apples\.html
(no python files needed)

Woobie has the right answer. Let me phrase it differently.
When you put .html in static_files, they're served by separate services that are dedicated to serving static content. Your app will not be able to see those files. The app can only see files that are resources.
Django templates must be resources, not static files. Otherwise, the application can't see them, and the template.render(path, ... will fail.
A next step to getting your problem solved is (if you haven't done so aleady) is to update your post to show your current app.yaml, and also to show us what's being reported up in the application's logs.

There are technical reasons why it works this way
The app.yaml config functions in a very simple top->bottom procedural manner.
Matching happens in the following order:
1. - url: /(.*\.(html))
static_files: static/htmdir/\1
upload: static/htmdir/(.*\.(html))
2. - url: /css
static_dir: css
3. - url: /js
static_dir: js
4. - url: /images
static_dir: images
5. - url: .*
script: abcdefg.py
To put it simply, if the file has a .html suffix it gets matched in step 1 and reroutes that request from mygaeid.appspot.com/.html to mygaeid.appspot.com/htmdir/.html. The *.html handler in step 5 never gets hit because all *.html routes are already spoken for.
In addition -- as the answers have already covered -- directories marked as static will not be available locally to your app handlers. I'll try to address the technical reasons why.
app.yaml acts as a configuration file for GAE's reverse proxy. Static files only change when they're uploaded so they're ideal for caching. By immediately pushing the static files to a cache server when they're deployed, GAE increases loading performance and removes unnecessary load from the app servers.
There's a good reason that static requests are counted separately and cost less than regular app requests. Every time you request a static file you're essentially pulling fetching the file from GAE's CDN. If you were to only fetch static files, then your server would have no reason to spool up in the first place.
The obvious downside to that approach is that those files don't physically exist on the same server as your app so you can't fetch or upload them directly in your app.
Note: Also following what other answers have covered. Don't mark your template folder as static. When your app goes to load the template -- instead of grabbing it from an adjacent directory -- it'll have to send out a web request and fetch the file from a remote location. Obviously, fetching a remote file instead of a local file is going to increase load time and latency.