I'm a fan of using the right tool for the job.
At my company, the data analysts (incl myself) primarily use Python because of the powerful scientific libraries; the web people downstairs use Ruby on Rails for building our own HR management webapp as well as maintaining our online presence.
We would like to have the two teams working a little closer together allowing the development of scientific webapps but are unsure about how to proceed.
We have significant investment in both technologies with a substantial codebase that we would need to continue to use.
Are there any suggestions about the best way to integrate the two domains of scientific programming and web apps using the two separate languages?
In lieu of substantially hacking apart either codebase to fit the other, I would first propose you evaluate an SOA solution.
e.g. create an API interface to your Python system, and have
the Rails app call that API for what the HR and public-site systems
require.
I assume that the Python part is a stack of back-end libraries/modules of scientific functions while the Ruby-on-Rail part is mostly as front-end. Then an relatively simple solution would be wrap those Python modules/libraries into services and let the Ruby-on-Rail-built front-end functions call those services.
To build the Python services, you can use xmlrpc for simple implementation, or some sophisticated framework such as twisted for larger scale implementation. You can also created only one portal service that hosts all your Python modules/libraries, that would basically give you an application server that handles all the requests form the Ruby-on-Rails front-end. Using which strategy depends on the scale and complexity of your Python libraries/modules.
Related
I'm slightly confused on what is meant by a "web application". I.e. Django positions itself as "a web framework" for the development of "web apps", quite similar to the description of Pyramid. What I'm looking for is a framework for an application that has a browser interface but is very database driven. In short: It's a system that should handle transaction and subscriptions. These transactions are registered through a web interface. The backend consists of XML/JSON feeds of these subscriptions/transactions and an administrative interface to generate statistics and reports. Quite similar to a ticketing/box office system.
Now: Is it a good decision to start out with something like Django/Pyramid even though they seem more focussed on the creation of "websites" rather than (scalable) input/output "applications". Is there any python framework that I'm better of using or are they perfectly suited for the kind of software that I'm describing?
Any help would be greatly appreciated!
I don't know Pyramid at the time of writing so I'll answer the Django part, though I can imagine the answer to apply to Pyramid as well as the distinction between websites and web application is quite vague.
Django is suitable for both websites and web applications: you can create an informative website for your local library as well as an all-in web application with lots of interactive features. Django simply facilitates the creation of the backend of a web system. Both websites and web applications use databases, have caching and can have interactive parts with forms.
It's just that websites are usually just for informing the user (most sites out there) or interacting with others (such as forums). A web application is best described with some examples, such as webmail, administrative interfaces, Google Docs, etc; it serves as a replacement of a desktop application.
From a technical perspective, there's no clear cut thing that separates the world of websites from web applications, it's just a matter of what you create with your web framework.
To answer your question, Django is quite suitable for creating a database-driven web application as it supports multiple databases, caching, autmatic creation and handling of forms, etc. I can imagine Pyramid to be suitable as well but I can't answer that. It's good to explore your options.
I believe that at this point you might want to think more about the overall architecture of your application rather than frameworks. For Example
Single Tier - Just a simple webapp, using simple file or object storage
Two Tier - Webapp + Relational Database. The webapp contains the presentation logic + business logic using MVC principles
Three Tier - Webapp + Middle Tier + Relation Database. In this setting, the Middle Tier will basically contain all the business & processing logic. The Webapp is then just a presentation layer
Once you decide the architecture, you can then start thinking about which framework to use for each of the component of the architecture.
I believe progressing this way will be less confusing and will give clarity of thought.
I need to create web application, which can be reached by user as regular web site and as XML-RPC web service. Also web site should have mobile version. I'm planning to use next technologies:
Django (for web frontends (regular and mobile)).
Pyramid (for web service).
SQLAlchemy, Memcached (for persistence level)
Later other projects can reach this data and providing logic, so I think it is better to make two tiers. I see it in next way:
Tier 1. Main logic service level. This level will provide API for frontend applications (Django powered web site, for example).
Tier 2. Different mostly end client applications (web site, API for remote client devices).
For communication between this tiers I'm planning to use XML-RPC protocol.
In this case it will be easy to scale it and add new front end application or connect another projects to this (I believe it).
I have main question, -- what can I use to make it easy build first tier? Maybe there is some framework good for that?
And what do you think about this whole architecture. Because I'm filling that I'm thinking in Java terms developing in Python. Maybe there is some another idioms in Python world for such situations.
Thanks for you time and help.
P. S.
Some links for reading are welcome.
This architecture really makes no sense. You're using Django, a full-stack web framework, for the front end, but not using it for the database. And you're using Pyramid, another full-stack web framework, for the web service side, thus ensuring that you duplicate all the business logic.
Much as I am an advocate of Django, I would say it has no place in your architecture. It looks like the only thing you're really using it for is URL routing and templates, both of which Pyramid does itself fine - you can even use Jinja2, which is based on Django's template language, as the template language in Pyramid if you like.
Doing it this way means that you can share the business logic between the front-end and web service code, since you'll almost certainly find that a lot of it will be the same.
I must say also that I don't understand the division into tiers, which you have described as separate from the front-end/web service division. To me, the web service is the second tier. It makes no sense to have a further division.
You should checkout the Turbogears framework as it is composed of several popular components: ORM with sqlalchemy, pylons for logic and support for WSGI, permits support for several templating engines for the frontend... endless.
I use it for several web-services behind AJAX-enabled front-ends (like Flex-based apps, among others). You can front end the TG2-based webapp with apache or your favorite WSGI-enabled web server too.
Checkout their website since they have a tutorial to setup a wiki in 20 minutes.
Cheers!
I've been learning python for use in ArcGIS and some other non-web applications. However, now that I've taken on building a personal website I am interested in using it for web development (as it is the only scripting language I currently know).
I've noticed that there are a lot of these things called "frameworks", such as Django. From what I understand they are just a collection of packages to save you from re-inventing the wheel but I don't really know how they work.
Furthermore, I do not like GUIs, if I need a framework I would like to find one that could be used through a terminal, starts out simple and can be scaled for more complexity when I'm ready. Any advice or ideas on frameworks and why I would want to use one?
The Python web frameworks have nothing to do with GUIs, and can all be used via the terminal.
The benefits of a framework, as you say, are all to do with making your life easier by supplying the components you need to build a website: the main ones are database interaction through an ORM, a templating system, and URL routing. On top of that, the big frameworks also included optional extras like user authentication, administration interface, and so on.
Personally I like Django, but your mileage may vary: I would say, though, that whatever you do with Python and the web will require some sort of framework, even if it's one of the absolute minimal ones like Flask which basically do just the routing part. There's simply no point in writing all this stuff from scratch when it's been done for you.
I'd second the post above: Django is a great framework and will save you loads of time in the long run.
Pretty much every challenge you'll come across when writing a web application will already have been solved, e.g. How do I send emails? What about an admin interface to edit the data? User security?
In my view picking the best framework is all about the ecosystem around that framework. How well used is it? Is it discussed widely on the internet? Have others encountered, and solved, the problems I'm facing?
In terms of where you start, see the Django Tutorial here:
http://docs.djangoproject.com/en/1.2/intro/tutorial01/
If you think Django offers you too much, I'd recommend that you take a look at CherryPy just to compare the different, and much simpler, approach.
With Python, you've got lots of options. To start, I would recommend looking here -- it explains the basics and provides a fairly complete list of frameworks.
If you're looking for something that starts out simple but can also handle more complexity, then you should take a look at web2py. It requires no installation or configuration, has no dependencies, and includes a web server and a relational database. It also includes an optional web-based integrated development environment and admin interface, but you can work through the terminal instead if you prefer.
It's very easy to learn and was designed for ease of use, faster development, and security. You can get a lot done with very little code thanks to the included scaffolding app along with many sensible default behaviors. As things get more complex, web2py can handle it, as it is a well-integrated full-stack framework with lots of built-in functionality, including a database abstraction layer, form handling and validation, access control, web services, and easy Ajax integration.
Personnally, I don't use any framework, I write either from scratch on BaseHTTPServer, or using WSGI (with mod_wsgi).
It is a bit long to write the skeleton, but I think it is faster (I mean at runtime), there is less constraints, and there is lesser to learn.
I was watching the tutorials for python and the guy told that he coded the Address books and spell checker for yahoo mail in python.
Now initially i was thinking that if i build the website then i have to use one language either php or java or asp or anything.
But i am confused how can we make make separate modules in diff languages and combine to make one website
Any ideas
Phisical architecture of web application can be different from the logical one visible through browser. Basically it is achieved by putting front web server (think of apache with mod_proxy, but it can be any other moder web server supporting reverse proxying) and mounting web application servers (java/python/whatever) to different paths (like /app1 for java app, /app1/subapp for python app, /app2 for php app). Of course those applications work independently by default, so if you want to pass some data between you have to establish some communication between (direct socket-to-socket or indirect with some messaging middleware or database).
In general it is very broad topic, so if you're interested, try with some basic keywords: application servers, load balancing, reverse proxy, url rewriting.
You can use any language to provide a web service, so you can for example provide a REST/SOAP web service that returns JSON or XML. The web service can be written in any language, and the language used to interact with the web service can be any language, as all languages nowadays have JSON and XML parsers.
You can setup different subdomains to be used by different servers and setup those applications in any language you'd like.
I know in Ruby on Rails, you can execute bash commands.
Example: puts ls
If you just need to call simple scripts from PHP, you can of course use exec(). I use it quite often to be able to code something in ruby, which I prefer for non-web tasks, and call the ruby script from PHP (it can of course be done the other way round).
If they're different pages, they can easily be created by different software. So if a mail application written in Java offers a link to an address book, the address book can easily be Python--that's just a matter of configuring the server.
If you need an addressbook component within the mail application, that's a bit more complicated, but still doable. Especially with Java and .NET it's possible to run various languages on the same platform (e.g. Jython and Ironpython run Python on the JAVA and .NET VMs respectively).
But i am confused how can we make make separate modules in diff languages and combine to make one website
Please read this: http://httpd.apache.org/docs/2.0/sections.html
Then read this: http://php.net/manual/en/configuration.changes.php
Then read this: http://code.google.com/p/modwsgi/
Then read this: http://www.modruby.net/en/
Then read this: http://tomcat.apache.org/
Apache can support a large number of languages via plug-in modules.
Using mod_rewrite as well as the fundamental "section" definitions available in Apache, you can separate your URI's into separate modules in different languages.
This year me and a friend have to make a project for the final year of university. The plan is to make a proxy/sever that allows to store ontologies and RDF's, by this way this data is "chained" to a web, so you can make a request for that web and the proxy will send you the homepage with metadata.
We have been thinking to use python and rdflib, and for the web we don't know which framework is the best. We thought of django, but we think that is very big for our purpose, and we decided that webpy or web2py is a better option.
We don't have any python coding experience, this will be our very first time. We have been always programming in c++ and java.
So taking into account everything we've mentioned our question is, which would be the best web framework for our project? And will rdflib suit fine with this framework?
Thanks :)
I have developed several Web applications with Python framworks consuming RDF data. The choice always depends on the performance needed and the amount of data you'll have to handle.
If the number of triples you'll handle is in the magnitude of few thousands then you can easily put together a framework with RDFlib + Django. I have used this choice with toy applications but as soon as you have to deal with lots of data you'll realise that it simply doesn't scale. Not because of Django, the main problem is RDFlib's implementation of a triple store - it is not great.
If you're familiar with C/C++ I recommend you to have a look at Redland libraries. They are written in C and you have bindings for Python so you can still develop your Web layer with Django and pull out RDF data with Python. We do this quite a lot and it normally works. This option will scale a bit more but won't be great either.
In case your data grows to millions of triples then I recommend you to go for a Scalable Triple store. You can access them through SPARQL and HTTP. My choice is always 4store. Here you have a Python client to issue queries and assert/remove data 4store Python Client