I'm going back over an old project where I added preprocessor functionality to Essence' and I realised that my previous solution of writing a domain specific language and associated lexer/parser was overkill.
Instead I just need to be able to embed dynamic language code into the file, isolate it at runtime, eval and insert the results. In other words very similar to the PHP model of inserting dynamic code into HTML. I'd rather not use PHP as Python is much easier to distribute as part of a larger project (IronPython or Jython)
So the question goes, how best to implement something like the following:
<code>Python goes here</code>
Lots of essence <code>Python</code> prime code goes here
I don't want to have to alter the structure of the Essence' file (if I remove all the code blocks everything left should be able to be syntactically correct. It needs to be able to insert text in place of a code block like PHP.
Finally security wise I'm not bothered about code injection, as it would be the user themselves choosing the file to execute although if there were security benefits to one model over another with no extra costs that would obviously be good.
Cheers in advance
Your best bet is to use one of the already made (and battle tested) Templating Engines. The two big ones that I've used are Mako, and Cheetah. They allow you to embed code right in the page, and are mostly used as the View in an MVC architecture.
If you feel that using one of those engines is overkill for your project, here is a small tutorial on how to implement basic templates yourself. Keep in mind that the example will need to be modified to suit your particular project/needs.
Related
I'm new to python and I'm comming from the c# world.
Over there it seemed like the repository pattern was the way to go, but I am having trouble finding any tutorials of how to best do this on Python.
edit I understand that it can be implemented, I'm just wondering if there is any reason why I am finding close to nothing for how to go about doing this.
Thanks!
I wasn't immediately familiar with the "repository pattern", so I looked it up. It appears to be the idea of putting a more general API, like a dictionary-like key/value lookup, in front of a database or other data store. It seems that the idea is to add an additional layer of abstraction that can allow multiple types of data sources (like both a relational database and a CVS file) to be accessed transparently via a common API.
Given this definition, I can think of no reason why this design pattern wouldn't be equally applicable to a problem addressed with Python vs any other programming language.
I just need pointers on where to begin. I have some experience with Python, but nothing to brag about.
My end goal is to create a website that will allow multiple users to access it from different computers to fill the table with simple data, very simillar to what Google Sheets alows, and then printing it on a single sheet of paper. Idealy I want my programm to intelegently determine the width of rows and columns so that the table would look decently and would fill the page accordingly.
Right now all I need is some pointers on where to begin. Like can I use SQL to create these tables and have online fuctionality for users to access and fill the spreadsheet, and how to go about printing it.
I know this is very noob question, but I can't seem to find anything relevant here on by just using google.
Thank you.
I don't think this is a very good StackOverflow question because it is very broad and not programming specific. You are asking how to start a new software project which in my opinion belongs more in the software engineering meta: https://softwareengineering.stackexchange.com/
Anyhow, how I would take on such a project:
First I would define my project scope. What is the functionality of the end-product? What must it be able to do and what not? Who are the end-users using the product and what do they expect? These are so called functional requirements.
In which way does the product deliver value? Is it fast, modifiable, distributed... These are so called non-functional requirements.
Develop a basic software architecture based on the previous requirements using patterns and tactics and identify the different subsystems. On the top of my hat I would divide it in a frontend component using a web application, backend component in your favourite language and a database component for persistence.
Research possible languages frameworks for each component, decide and start coding!
For the 4th step I suggest you have a look at Python Django which includes all of this stuff out-of-the-box.
i'm working on a project (written in Django) which has only a few entities, but many rows for each entity.
In my application i have several static "reports", directly written in plain SQL. The users can also search the database via a generic filter form. Since the target audience is really tech-savvy and at some point the filter doesn't fit their needs, i think about creating a query language for my database like YQL or Jira's advanced search.
I found http://sourceforge.net/projects/littletable/ and http://www.quicksort.co.uk/DeeDoc.html, but it seems that they only operate on in-memory objects. Since the database can be too large for holding it in-memory, i would prefer that the query is translated in SQL (or better a Django query) before doing the actual work.
Are there any library or best practices on how to do this?
Writing such a DSL is actually surprisingly easy with PLY, and what ho—there's already an example available for doing just what you want, in Django. You see, Django has this fancy thing called a Q object which make the Django querying side of things fairly easy.
At DjangoCon EU 2012, Matthieu Amiguet gave a session entitled Implementing Domain-specific Languages in Django Applications in which he went through the process, right down to implementing such a DSL as you desire. His slides, which include all you need, are available on his website. The final code (linked to from the last slide, anyway) is available at http://www.matthieuamiguet.ch/media/misc/djangocon2012/resources/compiler.html.
Reinout van Rees also produced some good comments on that session. (He normally does!) These cover a little of the missing context.
You see in there something very similar to YQL and JQL in the examples given:
groups__name="XXX" AND NOT groups__name="YYY"
(modified > 1/4/2011 OR NOT state__name="OK") AND groups__name="XXX"
It can also be tweaked very easily; for example, you might want to use groups.name rather than groups__name (I would). This modification could be made fairly trivially (allow . in the FIELD token, by modifying t_FIELD, and then replacing . with __ before constructing the Q object in p_expression_ID).
So, that satisfies simple querying; it also gives you a good starting point should you wish to make a more complex DSL.
I've faced exactly this problem - a large database which needs searching. I made some static reports and several fancy filters using django (very easy with django) just like you have.
However the power users were clamouring for more. I decided that there already was a DSL that they all knew - SQL. The question was how to make it secure enough.
So I used django permissions to give the power users permission to make SQL queries in a new table. I then made a view for the not-quite-so-power users to use these queries. I made them take optional parameters. The queries were run using Python's lower level DB-API which django is using under the hood for its ORM anyway.
The real trick was opening a read only database connection to run these queries just to make sure that no updates were ever run. I made a read only connection by creating a different user in the database with lower permissions and opening a specific connection for that in the view.
TL;DR - SQL is the way to go!
Depending on the form of your data, the types of queries your users need to use, and the frequency that your data is updated, an alternative to the pure SQL solution suggested by Nick Craig-Wood is to index your data in Solr and then run queries against it.
Solr is an added layer of complexity (configuration, data synchronization) but it is super-fast, can handle large datasets, and provides a (relatively) intuitive query language.
You could write your own SQL-ish language using pyparsing, actually. There is even pretty verbose example you could extend.
I'm looking for a way to translate my Django project. Built in mechanism provided with Django is great, but has several weak points which made me go looking for an alternative.
Project owner must be able to edit every translation including English (original translation). With gettext it is possible to edit translations with tools like Pootle, but the original strings stay hardcoded inside file sources or templates. There is no way that product owner can change them.
Possible solution is to make gettext translate some unique identifiers, and just translate them to all languages including English, like this:
_('form_sumbit_button')
But this makes tools like pootle almost impossible to use for translators.
Question: are there any tools for Django project translation that could fit my needs?
If you use some message IDs, they would either be incomprehensible ("message_2215") or you'd be forced to synchronise the message IDs to the actual messages ("Please press any key" = "please_press_any_key" => "Any key to continue" = "any_key_to_continue"). Either way, real strings are better for the programmers and for the tools.
However, if you employ a separate proof-reader for your strings, you can do the following:
Create an English "translation" file (yes, this works)
Let your proof-reader "translate" from English to English using Pootle or any other tool
Make sure your programmers keep that translation file untranslated by updating the strings in code.
(optional) Create a way to deploy translations independently of your main code so you can fix a typo quickly.
You may be able to use Pootle with the _("message_id") approach, depending on how easy Pootle is to customise (I don't know the internals so I can't say, but IIUC it uses Django where template changes are usually straightforward).
For example, Pootle's translation screens have "Original" and "Translation" sections; you could perhaps adapt the templates to show, under the "Original" section, a "Reference" section which displays some canonical translation using a specific reference language (e.g. English).
Or you may be able to use Pootle's alternative source language functionality, without needing to customise Pootle. You could store the canonical versions of the translations using an unused language code (or a made-up one).
Using identifiers is definitely possible with Gettext and there are tools which support this. However it might be unusual for some translators as they are used to downloading only .po file for offline translation, what does not work with monolingual translations.
For example Weblate supports monolingual Gettext files just fine (I'm author of this tool): https://docs.weblate.org/en/latest/formats.html#monolingual-gettext
I got a project in mind that makes it worth to finally take the plunge into programming.
After reading a lot of stuff, here and elsewhere, I'm set on making Python the one I learn for now, over C# or java. What convinced me the most was actually Paul Graham's excursions on programming languages and Lisp, though Arc is in the experimental stage, which wouldn't help me do this web app right now.
As for web app fast, I've checked out Django, Turbo Gears and Py2Web. In spite of spending a lot of time reading, I still have no clue which one I should use.
1) Django certainly has the nicest online presence, and a nicely done onsite tutorial, they sure know how to show off their thing.
2) Web2Py attracted me with its no-install-needed and the claim of making Django look complicated. But when you dig around on their website, you quickly find content that hasn't been updated in years with broken external links... There's ghosts on that website that make someone not intimately familiar with the project worry if it might be flatlining.
3) Turbo Gears ...I guess its modular too. People who wrote about it loved it... I couldn't find anything specific that might make it special over Django.
I haven't decided on an IDE yet, though I read all the answers to the Intellisense code completion post here. Showing extra code snippets would be cool too for noobs like me, but I suppose I should choose my web frame work first and then pick an editor that will work well with it.
Since probably no framework is hands down the best at everything, I will give some specifics on the app I want to build:
It will use MySQL, it needs register/sign-in, and there will be a load of simple math operations on data from input and SQL queries. I've completed a functional prototype in Excel, so I know exactly what I want to build, which I hope will help me overcome my noobness. I'll be a small app, nothing big.
And I don't want to see any HTML while building it ;-)
PS: thanks to the people running Stackoverflow, found this place just at the right moment too!
You should look at the web2py online documentation (http://web2py.com/book). It comes with a Role Based Access Control (the most general access control mechanism) and it is very granular, you can grant access for specific operation on specific records. It comes with a web based IDE but you can use WingIDE, Eclipse and PyCharm too. It comes with helper system that allows you to generate HTML without using HTML. Here is an example of a complete app that requires users to register/login/post messages:
db.define_table('message',Field('body'),Field('author',db.auth_user))
#auth.requires_login()
def index():
db.message.author.default=auth.user.id
db.message.author.writable=False
return dict(form=crud.create(db.message),
messages=db(db.message.id>0).select())
The web2py project is very active as you can see from the list of changes http://code.google.com/p/web2py/source/list
If you have web2py related questions I strongly suggest you join the web2py mailing list:
http://groups.google.com/group/web2py/topics
We are very active and your questions will be answered very quickly.
I have to say as not particularly skilled developer, the speed at which I have been able to create using web2py has blown my mind. In large part due to the amazing community and the core value Massimo has of making the framework accessible.
When I started I had written 0 lines of code in Python
Never heard of web2py
I've been at it seriously for about a month and have progressed (in my usual fashion) from asking questions that no one could answer (because they didn't make any sense) to coding for hours at a time without picking up a book or asking a question.
I'm really impressed.
I've had positive experiences with Django.
Built-In Authentication and easy to use extensions for registration
Very good documentation
You probable write your HTML templates mostly in base.html, then just use template inheritance (Note: You'll need to write at least a little bit of HTML)
In contrast to Turbogears, Django is more 'out-of-the-box'
I don't have any experience with web2py, but from my impression, it tries to do a little to much 'out-of-the-box'
If you decide to go with Django, make sure that you use its Generic Views. They will save you from writing lots of code, both Python and HTML.
Also, unless there is a very specific reason for you to use MySQL, I advise you to switch to PostgreSQL. Django is much more oriented towards PostgreSQL and it's a much better database anyway.
The online Django documentation is great, this is what put it apart from all the other frameworks. I also recommend the book Practical Django Projects by James Bennett
Django: Heard it has the best administrative
interface. But uses it's own ORM, i.e. doesn't use SQL-Alchemy.
Web2py: Didn't research this.
Turbogears2:
Uses SQL-Alchemy by default, uses Catwalk for admin
interface, but documentation isn't as
great.
I chose Turbogears2 because it uses popular components, so I didn't have to learn anything new...
I've used both web2py and RoR extensively, and while RoR has gotten a lot of popularity and support in the past few years, web2py is simpler, cleaner, less "magical", and yet also offers more (useful) out-of-the-box functionality. I'd say that web2py has more potential than RoR, but it is a relatively new framework and does yet not have the maturity of RoR. (Despite that, though, I'd choose web2py over RoR any day...)
If you "don't want to see any HTML while building it" then you can forget Django. It is not focused on "point-click-done," it is focused on pros going from concept to production in the shortest time possible. The hierarchical nature of the templating language can lead to some very clean overall site layouts. I use Django for all of my larger sites and I love it.
Although it's written in PHP, not Python, you might take a look at the major new version of WordPress that came out about 2 or 3 months ago. In 3.0 they have come a long way from being a "blogs only" environment and there are tons of ready-made templates for it. Of course if you want to tweak a template, well, there's that nasty old HTML again. I am considering using it for my smaller clients that can't deal with the admin of a dedicated server, etc., that tends to come with a Django site.
Update:
Ah, I missed the semi-joke -- I was up too early and that tends to make me tone deaf to humor. As far as using templates from existing sites, I have done this quite successfully with a couple of sites, both those that were static and those originally driven by well-written PHP scripts. I recommend a careful reading of the {% extends %} and {% include %} docs. Both take either a string literal or a variable. I have used the later method and it can be quite useful for a site that has strong hierarchy distinguished by style changes across branches.
It is also worth the time to understand the search order for templates -- it can be used to good effect, but it can be puzzling if you don't grok it. See the template-related items in the settings.py file for this and other useful goodies.