Develop Reference

What goes into content type scope mangager from pmr2-oauth (a package in plone)

I am trying to connect my custom api to a plone website using the pmr2.oauth provider on the plone website and using oauthV1. Everything goes smoothly from requesting a temporary key to recieving the oauth access tokens.
But when I try to access the resources I get an invalid scope. I have been told to fill in the content type scope manager in pmr2.oauth package, I have to fill a mapping for each of the following portal types to a list of permitted subpaths: Plone site, Collection, File, Folder, Manager Folder, Page, Page and subsite.
But I have no idea what to fill in here, so I hope maybe you guys can help me.

Package owner here, first off I must apologize and say I can probably document this better (sure it's documented via doctests but it can be made better)!
What you need to fill there is the view that you want to access for that particular content type. For instance, if you wish to show the listing of all the items from within a collection you would figure out what the name of the view is (in this case, atct_topic_view). For a standard Page it would be document_view. That said, OAuth typically targets web-services that typically communicates via a more concise format, such as json, so it is possible for developers to develop new views specific to some existing (or custom) Plone content types and then these can be added to the mappings to be made available.
Lastly, the mappings are essentially the endpoint, plus an optional subpath which can be a wildcard. As of writing, specifying a wildcard for custom subpaths does not imply the root parent view being available, so let's say you have a custom download view that can let user list the formats which links to subpaths within for the actual content, you might do something like this:
Collection:
download_feed
download_feed/*
Would make available the download_feed view within the Collection content type and then any subpaths within (generally made available by implementing zope.publisher.interfaces.IPublishTraverse for the custom view).
Alternatively you can write your own ScopeManager. Just create a class that inherit from pmr2.oauth.scope.BaseScopeManager (or BTreeScopeManager), implement all the methods (and tests) and then register this within your code or with a zcml like this:
<adapter
for="zope.annotation.interfaces.IAnnotatable
your.app.interfaces.IAppLayer"
factory="your.app.ScopeManager"
provides="pmr2.oauth.interfaces.IScopeManager"
/>
Which should then override the default (Portal) Content Type based scope manager with your own.

Flask templates including incorrect files

I'm having a rather strange problem using a jinja2.ChoiceLoader (also tried with multiple paths with FileSystemLoader, no joy) in Flask.
I have several "theme" directories, like so.
/templates/
themes/
default/
layout.html
menu.html
blue/
layout.html
grey/
menu.html
...
And I'd like to fallback to default/ if the selected theme doesn't have the required template, so I used a ChoiceLoader, like so.
#app.before_request
def setup_request():
current_theme = get_theme()
logging.info('Using theme %s'%(current_theme))
app.jinja_loader = jinja2.ChoiceLoader([
jinja2.FileSystemLoader('/templates/themes/%s/'%(current_theme)),
jinja2.FileSystemLoader('/templates/themes/default/')
])
That's great, but if I change the <current_theme> it still loads the theme from the old folder until I reload Apache or restart the Flask development server.
It should be using the new theme. Logging says that it's using the changed theme, but apparently app.jinja_loader is a bit like honey badger... it's completely ignoring it until I reload Apache.
Edit: This appears to be related to Flask considering all files of the same name to be the same file. I can reproduce with the builtin server (with DEBUG=True), Cherry, and mod_wsgi. This person seems to have a similar problem, but no simple solution: flask blueprint template folder My situation is different in the sense that I require cascading templates for a single app. His problem is related to cascading templates between blueprints, but it may be the same issue under the hood.
Here's the code that's in the "get_theme()" call:
def get_theme():
# I know this is unsafe, testing only
return request.args.get('theme','default')
Edit 2: I need to change the HTML and JavaScript between themes, not just the CSS. This is why I'm not just loading different CSS files. Also, some of these themes are for mobile devices, and have very little in common with the other themes.
Edit 3: Two solutions. Solution 1: Name the files uniquely, like "blue.layout.html" and "default.layout.html". This works perfectly, but it doesn't cascade as required. Solution 2: Use relative paths, so instead of include 'file.html', use include 'theme/blue/file.html. I achieved cascading by creating a get_theme_file() function that checks for the active theme, checks if the file exists (if not, fallback to "default" theme), and returns the relative path. I just have to make sure everything I include looks like {% include get_theme_file('file.html') %}. This is not elegant, but I find it to be more elegant that the low-level fiddling with Flask used here.

By the way, you can pass multiple locations to FileSystemLoader, and it is the recommended way to load templates
This is expected behavior in Apache with mod_wsgi (which I assume you are using). File system changes don't trigger a reload of all the processes. See this entry in the Flask docs that talks about this, and offers a workaround which is to add:
WSGIScriptReloading On
To the configuration section for your application and then touching the wsgi file to trigger a reload of the child processes.
Are you sure this is what you intend? Most theme switching tricks rely on the cascading part of cascading style sheets (CSS) to control themes.

Well, I'm not the only one to encounter this problem. The issue is that Flask caches based on filename, and if you don't include the relative path it just caches the first one to be loaded. There are three ways to accomplish dynamic, cascading, templates.
Override Jinja builtins. Which I found to be quite confusing. I'm not smart enough for this solution.
Serve a different WSGI process for each file. The setup here seems a bit too much for a dynamic site. Flask caches the filenames per WSGI process, so you can do something with multiple Cherry WSGI servers, for example.
Include the theme in the load path. Create a function, load it in with context_processor, and only load template files using that function. The function needs to check for a non-default theme, check if the file exists, and return it. So a template call would be get_theme_file('layout.html') which would return the relative path (like themes/blue/layout.html).
An example of Option 3.
def get_theme_file(fname):
theme = get_theme()
if os.path.exists(os.path.join(theme.theme_dir, fname)):
return os.path.join('themes', theme.name, fname)
return os.path.join('themes', 'default', fname)
...
# Each render_template should reference this
return render_template(get_theme_file('layout.html'))
If you include theme files in templates:
{% include get_theme_file('layout.html') %}
Unfortunately, this doesn't cache, but I can see a few ways to optimize it. Maybe cache an os.listdir of the get_theme().theme_dir, and instead of reading the filesystem for each get_theme_file call, just do an in check against the cached listdir list.
It's worth noting that this is Flask specific. I was unable to reproduce this behavior with plain Jinja2 and my own WSGI server. One might say that Flask was a poor choice for this particular project, but I'd argue that the savings from everything else Flask does was well worth it.

In plone how can I make an uploaded file as NOT downloadable?

I wish to make the uploaded file contents only viewable on the browser i.e using atreal.richfile.preview for doc/xls/pdf files. The file should not be downloadable at any cost. How do I remove the hyperlink for the template in a particular folder for all the files in that folder? I use Plone 4.1 There is AT at_download.

Cue tune from Hotel California: "You can check out any time you like, but you can never leave."
You do not not really want to disable all downloading, I believe that you really just want to disable downloads from all users but Owner. There is no practical use for putting files into something with no vehicle for EVER getting them back out...
...so you need to solve this problem with workflow:
Use a custom workflow definition that has a state for this behavior ("Confidential"). Ensure that "View" permission is not inherited from folder above in the permissions for this state, and check "Owner" (and possibly "Manager" if you see fit) as having "View" permission.
Set the confidential state as the default state for files. You can do this using Workflow policy support ("placeful workflows") in parts of the site if you do not wish to do this site-wide.
Should you wish to make the existence of the items viewable, but the download not, you are best advised to create a custom permission and a custom type to protect downloading with a permission other than "View" (but you still should use workflow state as permission-to-role mapping templates).

Script (Python) at /mysite/portal_skins/archetypes/at_download Just customize to contain nothing. Thought this will be helpful to someone who would like to keep files/ image files in Plone confidential by sharing the folders with view permission and disable checkout and copy option for the role created

Django: "projects" vs "apps"

I have a fairly complex "product" I'm getting ready to build using Django. I'm going to avoid using the terms "project" and "application" in this context, because I'm not clear on their specific meaning in Django.
Projects can have many apps. Apps can be shared among many projects. Fine.
I'm not reinventing the blog or forum - I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
If so... in terms of Django's project.app namespace, my inclination is to use myproduct.myproduct, but of course this isn't allowed (but the application I'm building is my project, and my project is an application!). I'm therefore lead to believe that perhaps I'm supposed to approach Django by building one app per "significant" model, but I don't know where to draw the boundaries in my schema to separate it into apps - I have a lot of models with relatively complex relationships.
I'm hoping there's a common solution to this...

Once you graduate from using startproject and startapp, there's nothing to stop you from combining a "project" and "app" in the same Python package. A project is really nothing more than a settings module, and an app is really nothing more than a models module—everything else is optional.
For small sites, it's entirely reasonable to have something like:
site/
models.py
settings.py
tests.py
urls.py
views.py

Try to answer question: "What does my
application do?". If you cannot answer
in a single sentence, then maybe you can
split it into several apps with cleaner
logic.
I read this thought somewhere soon after I've started to work with django and I find that I ask this question of myself quite often and it helps me.
Your apps don't have to be reusable, they can depend on each other, but they should do one thing.

What is to stop you using myproduct.myproduct? What you need to achieve that roughly consists of doing this:
django-admin.py startproject myproduct
cd myproduct
mkdir myproduct
touch myproduct/__init__.py
touch myproduct/models.py
touch myproduct/views.py
and so on. Would it help if I said views.py doesn't have to be called views.py? Provided you can name, on the python path, a function (usually package.package.views.function_name) it will get handled. Simple as that. All this "project"/"app" stuff is just python packages.
Now, how are you supposed to do it? Or rather, how might I do it? Well, if you create a significant piece of reusable functionality, like say a markup editor, that's when you create a "top level app" which might contain widgets.py, fields.py, context_processors.py etc - all things you might want to import.
Similarly, if you can create something like a blog in a format that is pretty generic across installs, you can wrap it up in an app, with its own template, static content folder etc, and configure an instance of a django project to use that app's content.
There are no hard and fast rules saying you must do this, but it is one of the goals of the framework. The fact that everything, templates included, allows you to include from some common base means your blog should fit snugly into any other setup, simply by looking after its own part.
However, to address your actual concern, yes, nothing says you can't work with the top level project folder. That's what apps do and you can do it if you really want to. I tend not to, however, for several reasons:
Django's default setup doesn't do it.
Often, I want to create a main app, so I create one, usually called website. However, at a later date I might want to develop original functionality just for this site. With a view to making it removable (whether or not I ever do) I tend to then create a separate directory. This also means I can drop said functionality just by unlinking that package from the config and removing the folder, rather than a complex delete the right urls from a global urls.py folder.
Very often, even when I want to make something independent, it needs somewhere to live whilst I look after it / make it independent. Basically the above case, but for stuff I do intend to make generic.
My top level folder often contains a few other things, including but not limited to wsgi scripts, sql scripts etc.
django's management extensions rely on subdirectories. So it makes sense to name packages appropriately.
In short, the reason there is a convention is the same as any other convention - it helps when it comes to others working with your project. If I see fields.py I immediately expect code in it to subclass django's field, whereas if I see inputtypes.py I might not be so clear on what that means without looking at it.

I've found the following blog posts very useful about django applications and projects:
http://www.b-list.org/weblog/2006/sep/10/django-tips-laying-out-application/
http://web.archive.org/web/20080302205555/www.pointy-stick.com/blog/2007/11/09/django-tip-developing-without-projects/
In principle, you have a lot of freedom with django for organizing the source code of your product.

If so... in terms of Django's project.app namespace, my inclination is to usemyproduct.myproduct, but of course this isn't allowed
There is nothing like not allowed. Its your project, no one is restricting you. It is advisable to keep a reasonable name.
I don't see any portion of my product being reusable in any context. Intuitively, I would call this one "application." Do I then do all my work in a single "app" folder?
In a general django project there are many apps (contrib apps) which are used really in every project.
Let us say that your project does only one task and has only a single app (I name it main as thethe project revolves around it and is hardly pluggable). This project too still uses some other apps generally.
Now if you say that your project is using just the one app (INSTALLED_APPS='myproduct') so what is use of project defining the project as project.app, I think you should consider some points:
There are many other things that the code other than the app in a project handles (base static files, base templates, settings....i.e. provides the base).
In the general project.app approach django automatically defines sql schema from models.
Your project would be much easier to be built with the conventional approach.
You may define some different names for urls, views and other files as you wish, but I don't see the need.
You might need to add some applications in future which would be real easy with the conventional django projects which otherwise it may become equally or more difficult and tedious to do.
As far as most of the work being done in the app is concerned, I think that is the case with most of django projects.

Here Django creators points out that difference themselves.
I think that thinking about Apps as they have to be reusable in other projects is good. Also a good way of thinking about Apps in Django provide modern web applications.
Imagine that you are creating big dynamic web app basing on JavaScript.
You can create then in django App named e.g "FrontEnd" <-- in thins app you will display content.
Then you create some backend Apps. E.g App named "Comments" that will store user comments. And "Comments" App will not display anything itself. It will be just API for AJAX requests of your dynamic JS website.
In this way you can always reuse your "Comments" app. You can make it open source without opening source of whole project. And you keep clean logic of your project.

How to externally populate a Django model?

What is the best idea to fill up data into a Django model from an external source?
E.g. I have a model Run, and runs data in an XML file, which changes weekly.
Should I create a view and call that view URL from a curl cronjob (with the advantage that that data can be read anytime, not only when the cronjob runs), or create a python script and install that script as a cron (with DJANGO _SETTINGS _MODULE variable setup before executing the script)?

There is excellent way to do some maintenance-like jobs in project environment- write a custom manage.py command. It takes all environment configuration and other stuff allows you to concentrate on concrete task.
And of course call it directly by cron.

You don't need to create a view, you should just trigger a python script with the appropriate Django environment settings configured. Then call your models directly the way you would if you were using a view, process your data, add it to your model, then .save() the model to the database.

I've used cron to update my DB using both a script and a view. From cron's point of view it doesn't really matter which one you choose. As you've noted, though, it's hard to beat the simplicity of firing up a browser and hitting a URL if you ever want to update at a non-scheduled interval.
If you go the view route, it might be worth considering a view that accepts the XML file itself via an HTTP POST. If that makes sense for your data (you don't give much information about that XML file), it would still work from cron, but could also accept an upload from a browser -- potentially letting the person who produces the XML file update the DB by themselves. That's a big win if you're not the one making the XML file, which is usually the case in my experience.

"create a python script and install that script as a cron (with DJANGO _SETTINGS _MODULE variable setup before executing the script)?"
First, be sure to declare your Forms in a separate module (e.g. forms.py)
Then, you can write batch loaders that look like this. (We have a LOT of these.)
from myapp.forms import MyObjectLoadForm
from myapp.models import MyObject
import xml.etree.ElementTree as ET
def xmlToDict( element ):
return dict(
field1= element.findtext('tag1'),
field2= element.findtext('tag2'),
)
def loadRow( aDict ):
f= MyObjectLoadForm( aDict )
if f.is_valid():
f.save()
def parseAndLoad( someFile ):
doc= ET.parse( someFile ).getroot()
for tag in doc.getiterator( "someTag" )
loadRow( xmlToDict(tag) )
Note that there is very little unique processing here -- it just uses the same Form and Model as your view functions.
We put these batch scripts in with our Django application, since it depends on the application's models.py and forms.py.
The only "interesting" part is transforming your XML row into a dictionary so that it works seamlessly with Django's forms. Other than that, this command-line program uses all the same Django components as your view.
You'll probably want to add options parsing and logging to make a complete command-line app out of this. You'll also notice that much of the logic is generic -- only the xmlToDict function is truly unique. We call these "Builders" and have a class hierarchy so that our Builders are all polymorphic mappings from our source documents to Python dictionaries.