Getting information about static files in Python App Engine; workarounds - python

I'm working on an App Engine project that will have customizable themes. I'd like to be able to use jQuery UI themes. The problem is figuring out what the CSS file is going to be named. (Typically, "jquery-ui-1.7.2.custom.css". Version numbers will change, and people tend to rename things, but there should only be one CSS file, and I'm OK with it being an error condition if there's two or more for some reason.) Because it's a static file (static files are uploaded to App Engine separately from the rest of the application's resources), I can't just glob the directory for a CSS file. I can't just assume that it's hard-coded, and I really don't want to make it a configuration setting, because that's a bad user experience.
Guido told me to symlink it so that App Engine sees two copies and can treat one as static and the other as an application resource, but symlinks don't work on Windows, and since this will ultimately be open source, I can't control which SDK the user uses. Another suggestion was to use a deploy-time script, but Mac users have this nice "Deploy" button in their version of the SDK and I'd rather not have to tell them, "Oh hey, sorry for the inconvenience, but you can't use that for this project."
I clearly need an out-of-the-box solution to this one, but I'm at a loss. Anyone have any good suggestions for how to get a custom jQuery UI theme out of the ThemeRoller and into an App Engine app? Some post-processing is already needed, because the only files in the zip file that ThemeRoller gives you are in the "css" directory. Maybe I can write something that takes a raw theme as input and spits out something useful on the other side (the deploy-time script trick, but somehow less user-unfriendly). The trick here is presentation — I want the user to spend as little time on the command line as possible. An ideal solution assumes the person performing this task is non-technical for the most part. No part of the solution can be much harder than installing something like WordPress or Drupal, and in a perfect world, it should be way, way easier.

To accomplish what you are asking, I would use the datastore for serving the CSS files. Since this would allow easy listing, sorting and even modification and uploading.
Other than that, your next best options would be to store the CSS data inside a script (a dictionary where the filename is the key name, and the CSS code is the value). Or, as you suggested, to run a script before deploying to AppEngine.
Personally, I would go for the storing in the datastore option, since it will allow for a great deal more user customization (such as each user being able to provide their own CSS file), just be sure to use memcache to avoid needing to access the datastore when possible (which should be a very common occurrence), as well as using HTTP headers to tell the browser to cache the CSS file locally.

Related

change static some static files location from /static/file.js to /file.js

I've a website running on Django, Heroku.
I need to add few static JavaScript files for a third-party plugin.
My newly added files are available at domain.com/static/filename.js.
I need them to be available at domain.com/filename.js.
How to make ONLY the newly added Javascript files available at domain.com/filename.js?
If the info is not sufficient please ask which code is needed in the comments.
My first choice in this situation would be to fix whatever is stopping you from putting it into /static/. I can't imagine any half-decent third-party plugin would demand that the files be in the root; there must be some way to configure it to work from a subdirectory. If there isn't, I'd fork the project and add the option, then try to get them to merge it back. I realise you've probably already explored this option, but can you give us some more details about the plugin you're trying to use, and the reason it needs to go into the root? This really would be the best solution.
If you really must have the file in the root, and want to keep it as part of your django project, I'd try symlinking the files into the public root. This would mean it would be available in both locations; I can't see why that would be a problem, but you do specify "ONLY" in the root and I'm sure you have your reasons; in that case, perhaps you could configure your web server to redirect from /static/filename.js to /filename.js?
Lastly, you technically could change the settings STATIC_URL and STATIC_ROOT to point at the root directory, but that sounds like a pretty terrible idea to me. If you've got this far and still need to do it, it would be far better to take the file out of your django project altogether and just manually place it in your web root.
If there are only a couple of these files, I guess you could do the following:
Create URLs for each of the files you want to serve
Hook those URLs up to a view that returns the file with the right content
refer to this snippet for an example view

Is there a django app that provides a file chooser for the files on the server?

I need a component that's a browser-based file browser, and I expect some django app to currently provide this. Is there such a thing?
The full story:
I'm building a django app that is used for testing. I want to use it to serve files (and strings, and etc.) and attach custom headers to it.
Currently, I have a model FileSource which has a single file_path field, which is of type django.db.models.FileField.
When creating a FileSource from the admin, the user has a nice file upload dialog, and when saving, the file he chose, is saved on the server (in a really weird location, inside the directory where django is installed, or something weird like that, because i didn't customize the storage, nor will it help me in any way)
My problem: I only want to use the file dialog for the user to select a full path on the server. The file that the user chose must be only referenced, not copied (like currently), and it must reside on the server.
The server must thus be able to list the files it has, so i basically need a little browser-based file-browser.
At that point, I expect to be able to save a full path in my DB, and then I'll be able to access that file and serve it (together with whatever custom headers the user will chose from my app).
Currently, as you might know, the browsers always lie about the full path of the file. Chromium appends "C:\fakepath" to the file name, so I need support of the backend to accomplish this.
Also, I checked out django-filebrowser and django-filer and from what I understood, they weren't built for this. If I'm wrong, a little assistence in configuring them would be awesome.
You can use a FilePathField for that. It won't upload a file, but rather allow you to choose a pre-existing file. A caveat is that you can only use one directory. If you need multiple directories, then you'd need do go with something like django-filer.

web2py site doesn't load all images/videos (especially larger ones)

First of all, I must say I have seen something similar to this in the web2py discussion group, but I couldn't understand it very well.
I've set up a database-driven website using web2py in which the entries are just HTML text. Most of them will contain img and/or video tags that point to relative URLs; these files are stored in folders with the address pattern static/content/article/<article-name> and the document's base href is set via the controller to make these links work. So, the images are stored and referenced directly, without all the upload/download machinery.
I'm testing it locally and using Rocket server because I'm not allowed to install Apache in this PC.
The problem:
Everything works fine, except, as it seems, when there are several "large" files being requested. By "large" I mean 4Mb files were enough, which isn't really a lot (and I think slightly smaller files would produce the same result). I'm pretty sure the links aren't broken since 1) by copying/pasting their URLs in the browser they show up normally, 2) the images/videos appear well/broken randomly when I refresh the page and 3) sometimes a video loads until a certain point and then stops, and the browser inspector shows a 'fail' signal. When I replaced these files with smaller ones (each with a dozen kb), all of them loaded. Another thing to consider is that sometimes it takes a really long time until the page finishes loading (from 2 seconds to several minutes).
The questions:
Is this the simplest/optimal way of getting the job done? I'm aware that web2py has some neat features like upload fields, but I don't know how I could make these files be effortlessly referenced in the document, considering there will be some special features in such pages involving the static files. So the solution I've come up with so far was to create a directory which name equals to the entry's and store the files there, as I said before. Is it an overkill considering what web2py has to offer?
If the answer to the first question is something like "yes", then (obvious question) what may be causing the problem and how do I fix it? Does it have something to do with the fact that web2py sends static files in chunks of 1Mb? Might it be the Rocket server? Or because I'm testing it locally?
Thanks in advance!
It's hard to give you an answer without knowing some details...
Where is hosted your Web2py application?
Do you use apache? nginx?
Did you deploy using a one step-deploy script? (http://web2py.googlecode.com/hg/scripts/setup-web2py-ubuntu.sh)
But in any case, you can (should) :
Configure Apache/Nginx to serve your static files directly (files in /YourApp/static/.). See "setup-web2py-" scripts in the "scripts" folder for more informations
Use scripts/zip_static_files.py to create gzipped versions of your static files. You can create a cron to run "python web2py.py -S myapp -R scripts/zip_static_files.py"
More details about efficiency in the book : http://web2py.com/books/default/chapter/29/13/deployment-recipes?search=static+files#Efficiency-and-scalability

Create a 'single-serving site' with python

I want to make a Python script available as a service on the net. The script, which is my first 'proper' Python program, takes a txt file as argument and writes an image into the work directory. So:
How difficult is it for somebody who is new to Python and web development?
How much work is it?
Do I need a framework (Django, cherryPy, web2py)?
Are there good tutorials?
How do I avoid the server to be compromised?
What are my next steps?
==> What is the easiest way?
In the end it is enough, if it is a white page, with some text, and a button, which when clicked, opens a file dialog. After the txt is processed, the server should just return the image, which was written on the hard drive. Already I have access to a server which has Ubuntu installed through a friend.
[update]
Thanks for all your answers. After reading them I want to stress again, that I want to have it as minimal as possible. Srikar's suggestion sounds like the easiest one:
Put it in executable directory of your OS (commonly known as CGI
path). Provide a simple HTML form & upon form submission hit this
script which executes & returns back the image you want to display.
Any objections or comments? Do you know any tutorials for that?
[udpate2]
I found this SO answer: File Sharing Site in Python Is this a sensible approach?
It's not too difficult. Actually, it sounds like a good first project.
That too subjective to answer. An hour to days.
No, you don't need one, but I'd use one if I were you. They abstract away some of the stuff you really don't care about, and you'll learn a tool you can use again in the future.
Plenty. If you want a real rundown of how Python works for the web, read the HOWTO from Python.org. If you just want to learn how to do this one project, pick a framework and do their tutorial.
This question is so broad and complex that I'm not going to try to answer it. Search this site, or Google, for questions like that.
Your next step should be to pick a framework; I've used Django successfully. Just download it, follow the installation instructions, and work your way through their tutorial; it should tell you everything you need to know to do what you want. If you still have questions once you've learned how to do the basics, come back and ask again!
Edit: The answer to that other question will certainly work for you. There, they just receive a GET request and respond with data from a Python file. You need to receive a GET request, respond with an HTML page (easy enough), then respond to a POST request that includes an uploaded file (slightly more complicated) and run your python routine on the uploaded file and then respond with the created image (or a link to it).
Take a look at this page which includes a simple Python script to do file uploads. You should easily be able to modify it to do what you want.
How difficult is it for somebody who is new to Python and web development?
Depends on your level of knowledge.
How much work is it?
Depends on which method you choose to solve the problem.
Do I need a framework (Django, cherryPy, web2py)?
Not necessarily - you could get started by using the CGI (http://docs.python.org/library/cgi.html)
Are there good tutorials?
Yes, there are plenty. The Python docs are an excellent place to start.
How do I avoid the server to be compromised?
Again, depends on the method you choose to solve the problem, although there are commonalities.
What are my next steps?
Dare I say it again, choose a method, read the docs, have a play!
If its just as simple as you have described it. Then you might not even need Django. You could simply use CGI scripting. All of these design decisions, depend on whether
You need (or foresee) a SQL storage?
or a Content-Management-System?
Will you need multiple-user support?
Do you need tight security?
Do you need different privileges for different users?
Do you need an Admin to manage your site?
If the answer to above questions is atleast 60% correct, then you might consider Django. otherwise, just write a python script. Put it in executable directory of your OS (commonly known as CGI path). Provide a simple HTML form & upon form submission hit this script which executes & returns back the image you want to display. So, it all depends on the features you need...
In the end, I created what I needed with Flask.
They have a well documented pattern / tutorial on Uploading Files. The tutorial is understandable even for people with little python and web expericence.
To get a first working version it took me 2h and the resulting code was only 50 lines. This includes, starting the webserver, having a html file/form with file upload and serving a file back to the user.

How to create an internationalized Google App Engine application

I would like to provide my Python GAE website in the user's own language, using only the tools available directly in App Engine. For that, I would like to use GNU gettext files (.po and .mo files).
Has someone successfully combined Python Google App Engine and gettext files? If so, could you please provide the steps you used?
I had started a discussion in GAE's Google group, but haven't been able to extract from it how I'd like to do it: I don't want to add external dependencies, like Babel (suggested in the discussion). I want to use plain vanilla Google App Engine, so no manual update of Django or this kind of stuff.
At first, I will start using the language sent by the browser, so no need to manually force the language by using cookies etc. However, I might add a language changing feature later, once the basic internationalization works.
As a background note to give you more details about what I'm trying to do, I would like to internationalize Issue Tracker Tracker, an open source application I've hosted on Launchpad. I plan to use Launchpad's translation platform (explaining why I'd like to use .mo files). You can have a look at the source code in it's Bazaar branch (sorry no link due to stackoverflow spam prevention limit for new users...)
Thanks for helping me advance on this project!
As my needs were simple, I used a simple hack instead of (unavailable) gettext. I created a file with string translations, translate.py. Approximately like this:
en={}
ru={}
en['default_site_title']=u"Site title in English"
ru['default_site_title']=u"Название сайта по-русски"
Then in the main code I defined a function which returns a dictionary with translations into the most suitable language from the list (the first one to have a translation is used or English):
import translate
def get_messages(languages=[]):
msgs=translate.en
for lang in languages:
if hasattr(translate,lang):
msgs=getattr(translate,lang)
break
return msgs
Usage:
msgs = get_messages(["it","ru","en"])
hi = msgs['hello_message'] % 'yourname'
I also defined a helper function which extracts a list of languages from Accept-Language header.
It's not the most flexible solution, but it doesn't have any external dependencies and works for me (in a toy project). I think translate.py may be generated automatically from gettext files.
In case you want to see more, my actual source is here.
You can use the Django internationalisation tool, like explained here.
They are also saying that there is no easy way to do this.
I hope that helps you :)

Categories

Resources