Is Django a good choice for a security critical application?
I am asking this because most of the online banking software is built using Java. Is there any real reason for this?
Actually, the security in Java and Python is the same. Digest-only password handling, cookies that timeout rapidly, careful deletion of sessions, multi-factor authentication. None of this is unique to a Java framework or a Python framework like Django.
Django, indeed, has a security backend architecture that allows you to add your own LDAP (or AD) connection, possibly changing the digest technique used.
Django has a Profile model where you can keep additional authentication factors.
Django offers a few standard decorators for view function authorization checking. Since Python is so flexible, you can trivially write your own decorator functions to layer in different or additional authentication checking.
Security is a number of first-class features in Django.
Probably the reason behind Java is not in the in the security. I think Java is more used in large development companies and banks usually resort to them for their development needs (which probably are not only related to the web site but creep deeper in the backend).
So, I see no security reasons, mostly cultural ones.
The reasons for building banking apps in Java are not related to security, at least IMHO. They are related to:
Java is the COBOL of the 21st century, so there is a lot of legacy code that would have to be rewritten and that takes time. Basically banking apps are old apps, they were built in java some ten years ago and nobody wants to throw all the old code away (which BTW is almost always a bad decision),
some people believe that a static typed language is somewhat "safer" than the dynamic typed language. This is, IMHO, not true (take for instance collections prior to Java 5).
I find your connection between Java and banking wrong ended.
Most Banking Software has terrible security. And much banking software is written in Java. Does ths mean Java makes it more difficult to write secure software than other languages?
Probably it's not Java's fault that there is so little quality security (and safety) wise in Banking software. Actually, like the other posters mention, the choice of your Language usually has very little consequences for your security - unless you select one of the few languages where only hotshot coders can write secure code in (C and PHP come to mind).
Many huge E-Commerce sites are written in Python, Ruby and Perl using various frameworks. And I would argue that the security requirements for merchants are much higher than the requirements of the banking industry. That is because merchants have to provide security and good user experience, while banking customers are willing to put up with unusable interfaces SecureID tokens and whatever.
So yes: Django is up to the task.
You should not rely the security of the application on the framework. even though Django does come in with a pretty good number of measures against classical security issues, it can not guarantee that your application will be secure, you need much more than a programming Framework to get a security critical application.
I'd say yes, Django is a good choice as long as you know its powers and limitations and are aware of the security flaws of every application.
You can build a secure application with Django just as you can with any popular Java framework. One part where Java does shine is its extensive cryptographic library.
For the minimal encryption tasks that are required by Django, Python’s cryptographic services are sufficient, however its lack of strong block ciphers make the encryption mechanism in Django insecure for data at rest.
Python does natively support secure hashing algorithms to include SHA1, SHA224,
SHA256, SHA384, and SHA512, however Django’s authentication mechanism has yet
to be updated to use anything other than SHA1, making it potentially vulnerable to cryptographic analysis.
Are you referring to the fact that the complete application is built in Java, or just the part you see in your browser? If the latter, the reason is probably because in the context of webpages, Java applets can be downloaded and run.
Related
I have a two questions regarding best practices and technologies being used in banking apps and similar "high risk" applications.
Let's say we were trying to build a banking app on which the customer can register an account with his or her credentials and transfer and receive money. Just built like every common banking app with authentication mechanisms, PUSH TAN and other verification methods prior to transfering money, session login which expires after a given time period of inactivity which results in automatically logging out and fingerprint authentication and so on...
QUESTIONS:
What system level architecture patterns (High Level Design) are used as the foundation for such applications?
And what (Stack) programming languages, db querying languages and APIs are typically used for such high risk applications like banking apps (maybe languages and FWs like Python and Django, MySQL, Cassandra)?
I'm very thankful for every answer so don't hesitate to throw in whatever you know.
Here are my thoughts around this:
In general, high-risk systems (banking apps, exchanges) should follow strict requirements in terms of security, availability, reliability, correctness. If one of those features is not carefully considered, bad things can happen (people lose money). This usually leads to higher costs, compared to other types of applications.
In terms of architecture, if you want to go with the current trends, you can use microservices. For example Monzo use a microservice-based approach to run their backend (this nice presentation goes into more details). This is definitely the new way of building low-latency scalable backend systems (even for high-risk apps). One downside that I see with this approach (I may be wrong) is the lack of maturity (compared to classical client-server model) and the fact that you always need to be up to date with the open-source technologies that are involved, because they are in a constant change. Big companies have dedicated teams which contribute and stay up to date with open-source technologies.
You can always go with a client-server architecture where the backend is just a fleet of nodes which process requests and talk to a database. This might be a bit simpler to start with, but probably harder to evolve. Choosing the architecture depends entirely on the development strategy that you plan for your system. There are pros and cons on any approach.
In terms of tech stack, I would go with mature languages/frameworks, like Angular, Java, C#, Spring Boot. That way you can find more easily developers to help you out, but also an ecosystem built by the industry over the years.
In terms of database, you could go with a relational solution (MySQL, SQL Server) and use transactions to provide atomic updates (to account balances, for example). This may hit the overall performance at some point, in terms of latency / throughput. A No-SQL approach may improve the speed, but may also come with data duplication, and some challenges in choosing the right data model.
As you can see, there are tradeoffs everywhere. It only depends on the initial strategy you set up, so you can avoid big future changes.
Is there an automated security scanner of any sort for Pyramid?
I've seen these for other frameworks, like Brakeman for Rails. It would be really great if there was one for Pyramid (or even Python in general).
Then again, it wouldn't surprise me if there wasn't, because it's not quite as popular as Rails.
I am not sure if Pyramid or any Python application is very good target for a security scanner. I am reading this list and I am not sure if any of these issues are very applicable to Python web applications. It is interesting idea though.
Frameworks, like Pyramid, should have safe defaults so that OWASP top 10 most common vulnerabilities should not be happening in the first place.
Usage of SQLALchemy or Django ORM prevents SQL injections
Template engines do safe HTML escape by default
Unsafe serialization (Python pickling) is non-existent as most code uses JSON
I have not seen anybody doing eval() in Python, though this doesn't mean some poor soul somewhere would not be doing it
etc.
However novice programmers surely can shoot themselves to foot with these kind of issues and it would make sense to catch handcoded SQL strings or eval() usage by an automatic scanner. But my gut feeling as being working with various Python web projects and security for over a decade is that in Python world the state of the matters is more robust and gains using a security scanner would be very small or non-existent.
We have an in house developed web-based admin console that uses a combination of C CGI and Perl scripts to administer our mail server stack. Of late we have been thinking of cleaning up the code (well, replacing most of it), making the implementation more secure, and improving the overall behavior.
I don't have much programming knowledge, but I use Ruby on and off (mainly for writing erb templates), and hence was thinking of using ruby/rails for developing such an app (off-duty for now, I also need to learn stuff !).
Before blindly picking up a language though, what would you folks suggest ? Please let me know if this is too vague a question, I'll try to supply more information, if needed.
Have you considered writing your applications as Webmin modules?
You get a lot of stuff for free when you do so (users and groups, tons of security features, a pretty big variety of helper functions related to config files, and tons of existing code for most aspects of a UNIX/Linux system). You also get a lot of stuff for nearly free, like action logging, packages and updates via wbm or apt or yum, an online help system, etc.
There are some cons, as well. It's an old codebase, so it has some clunky bits in the API among other places. A lot of the old modules can be a bit hard to grok if you're not an old-school Perl programmer. But, it's a well-maintained codebase, and it's been banged on by millions of users for over a dozen years. It's pretty robust. The UI isn't beautiful, but it is relatively theme-able, and if you're distributing a minimized version it becomes easier to customize the UI.
I suspect you can be up and running a lot faster than starting from scratch or using most existing frameworks that aren't targeted specifically to building systems management interfaces the way Webmin is.
Also, it's BSD licensed, so you can do whatever you want with it, including building a custom commercial app with it (hundreds of companies have done so over the years).
If you already know a bit of ruby, then there's no reason not to use that.
If you're interested specifically in learning another language, then what you're trying to do could be done in pretty much any language/framework, it's just a matter of which one you want to learn.
Without knowing much about your existing application I'd say that this effectively boils down to "which language do you like to work with?".
Python and Ruby are both mature languages with ample library infrastructure. They also boast popular, similar web application frameworks namely Django and Ruby-on-Rails respectively.
Since you are porting an existing Perl app(lets) it may be worthwhile to note that Ruby is relatively more similar to Perl. Not surprising given that Ruby was influenced "primarily by Perl, Smalltalk, Eiffel and Lisp".
django has a nice admin interface
I'm lucky enough to have full control over the architecture of my company's app, and I've decided to scrap our prototype written in Ruby/Rails and start afresh in Python. This is for a few reasons: I want to learn Python, I prefer the syntax and I've basically said "F**k it, let's do it."
So, baring in mind this is going to be a pretty intensive app, I'd like to hear your opinions on the following:
Generic web frameworks
ORM/Database Layer (perhaps to work with MongoDB)
RESTful API w/ oAuth/xAuth authentication
Testing/BDD support
Message queue (I'd like to keep this in Python if possible)
The API is going to need to interface with a Clojure app to handle some internal data stuff, and interface with the message queue, so if it's not Python it'd be great to have some libraries to it.
TDD/BDD is very important to me, so the more tested, the better!
It'll be really interesting to read your thoughts on this. Much appreciated.
My best,
Jamie
Frameworks
OK, so I'm a little biased here as I currently make extensive use of Django and organise the Django User Group in London so bear that in mind when reading the following.
Start with Django because it's a great gateway drug. Lots of documentation and literature, a very active community of people to talk to and lots of example code around the web.
That's a completely non-technical reason. Pylons is probably purer in terms of Python philosophy (being much more a collection of discrete bits and pieces) but lots of the technical stuff is personal preference, at least until you get into Python more. Compare the very active Django tag on Stack Overflow with that of pylons or turbogears though and I'd argue getting started is simply easier with Django irrespective of anything to do with code.
Personally I default to Django, but find that an increasing amount of time I actually opt for writing using simpler micro frameworks (think Sinatra rather than Rails). Lots of things to choose from (good list here, http://fewagainstmany.com/blog/python-micro-frameworks-are-all-the-rage). I tend to use MNML (because I wrote parts of it and it's tiny) but others are actively developed. I tend to do this for small, stupid web services which are then strung together with a Django project in the middle serving people.
Worth noting here is appengine. You have to work within it's limitations and it's not designed for everything but it's a great way to just play with Python and get something up and working quickly. It makes a great testbed for learning and experimentation.
Mongo/ORM
On the MongoDB front you'll likely want to look at the basic python mongo library ( http://api.mongodb.org/python/ ) first to see if it has everything you need. If you really do want something a little more ORM like then mongoengine (http://hmarr.com/mongoengine/) might be what you're looking for. A bunch of folks are also working on making Django specifically integrate more seamlessly with nosql backends. Some of that is for future Django releases, but django-norel ( http://www.allbuttonspressed.com/projects/django-nonrel) has code now.
For relational data SQLAlchemy ( http://www.sqlalchemy.org/) is good if you want something standalone. Django's ORM is also excellent if you're using Django.
API
The most official Oauth library is python-oauth2 ( http://github.com/simplegeo/python-oauth2), which handily has a Django example as part of it's docs.
Piston ( http://bitbucket.org/jespern/django-piston/wiki/Home) is a Django app which provides lots of tools for building APIs. It has the advantage of being pretty active and well maintained and in production all over the place. Other projects exist too, including Dagny ( http://zacharyvoase.github.com/dagny/) which is an early attempt to create something akin to RESTful resources in Rails.
In reality any Python framework (or even just raw WSGI code) should be reasonably good for this sort of task.
Testing
Python has unittest as part of it's standard library, and unittest2 is in python 2.7 (but backported to previous versions too http://pypi.python.org/pypi/unittest2/0.1.4). Some people also like Nose ( http://code.google.com/p/python-nose/), which is an alternative test runner with some additional features. Twill ( http://twill.idyll.org/) is also nice, it's a "a simple scripting language for Web browsing", so handy for some functional testing. Freshen ( http://github.com/rlisagor/freshen) is a port of cucumber to Python. I haven't yet gotten round to using this in anger, but a quick look now suggests it's much better than when I last looked.
I actually also use Ruby for high level testing of Python apps and apis because I love the combination of celerity and cucumber. But I'm weird and get funny looks from other Python people for this.
Message Queues
For a message queue, whatever language I'm using, I now always use RabbitMQ. I've had some success with stompserver in the past but Rabbit is awesome. Don't worry that it's not itself written in Python, neither is PostgresSQL, Nginx or MongoDB - all for good reason. What you care about are the libraries available. What you're looking for here is py-amqplib ( http://barryp.org/software/py-amqplib/) which is a low level library for talking amqp (the protocol for talking to rabbit as well as other message queues). I've also used Carrot ( http://github.com/ask/carrot/), which is easier to get started with and provides a nicer API. Think bunny in Ruby if you're familiar with that.
Environment
Whatever bits and pieces you decide to use from the Python ecosystem I'd recommend getting to who pip and virtualenv ( http://clemesha.org/blog/2009/jul/05/modern-python-hacker-tools-virtualenv-fabric-pip/ - note that fabric is also cool, but not essential and these docs are out of date on that tool). Think about using Ruby without gem, bundler or rvm and you'll be in the right direction.
Ok, you might be making a mistake, the same one I made when I started with python.
Before you decide on a thing like django, which is an excellent, yet atypical python web framework, spend an night cuddled up with:
This, is a good start. Make sure you do A little Werkzeug watching , Then check out
some classic WebOb. Maybe, if you feel the fire in the blood, and you might, wsgi is a bit flawed, but only to the gods, check out Flask
I'm not saying use it, Django is beautiful too, but if you don't know python, and you go through django, you run the risk of learning a framework.
WSGI is super straightforward. You'll find out about Paste, and Pastescript, and Pylons.
Then, make your decision. It'll be much easier learning stuff doing bare bones wsgi or Flask, stuff like variable assignment, using the interpreter, style concerns, testing, on 3 files for a couple of nights, instead of django. Take 2 nights. Then you'll see the great similarity between python web frameworks, instead of the differences. Hell, you might even roll with Flask.
Just some advice, I did the same thing with ruby, going in through Rails, and... well, strong words were said.
Language, then basic wsgi and testing, then pick your framework and roll
I'm new to python myself, and plan to get more in depth with it this year. I've had a few false starts at this, but always professional needs bring me back to PHP. The few times I've done some development, I've had really good experiences with web2py as a python framework. It's quite well done, and complete in features, while still being extremely lightweight. The database layer seems to be very flexible and mature.
As for TDD/BDD and the rest of your questions, I don't have any experience with python options, but would be interested to hear what others say.
I am using Twisted Framework based Nevow library for python based web app.
All your criteria fit into this single framework.
If there is truly a 'best' way, what is the best way to ship a python app and ensure people can't (easily) reverse engineer your algorithms/security/work in general?
If there isn't a 'best' way, what are the different options available?
Background:
I love coding in Python and would love to release more apps with it. One thing that I wonder about is the possibility of people circumventing any licensing code I put in, or being able to just rip off my entire source base. I've heard of Py2Exe and similar applications, but I'm curious if there are 'preferred' ways of doing it, or if this problem is just a fact of life.
Security through obscurity never works. If you must use a proprietary license, enforce it through the law, not half-baked obfuscation attempts.
If you're worried about them learning your security (e.g. cryptography) algorithm, the same applies. Real, useful, security algorithms (like AES) are secure even though the algorithm is fully known.
Even if you use a compiled language like C# or Java, people can perform reverse engineering if they are motivated and technically competent. Obfuscation is not a reliable protection against this.
You can add prohibition against reverse-engineering to your end-user license agreement for your software. Most proprietary companies do this. But that doesn't prevent violation, it only gives you legal recourse.
The best solution is to offer products and services in which the user's access to read your code does not harm your ability to sell your product or service. Base your business on service provided, or subscription to periodic updates to data, rather than the code itself.
Example: Slashdot actually makes their code for their website available. Does this harm their ability to run their website? No.
Another remedy is to set your price point such that the effort to pirate your code is more costly than simply buying legitimate licenses to use your product. Joel Spolsky has made a recommendation to this effects in his articles and podcasts.
Shipping a commercial mac desktop app in Python, we do exactly as described in the other answers; protect yourself by law with a decent EULA, not by obfuscating.
We have never had any troubles with people reverse engineering our code. And if we do, I feel confident we can take legal action. So yes, it's a fact of life. But one that is not too hard to live with. Just get a decent lawyer that writes a decent EULA.
The word you're looking for is obfuscate. A quick google reveals:
http://www.lysator.liu.se/~astrand/projects/pyobfuscate/
but:
a) If copyright infringement becomes a problem, then the law is on your side (as long as you include the appropriate copyright notices in all files).
b) It's also possible to make a profit on open source applications if you're clever about it.
c) If you want your Intellectual Property to be truly secure, then the only answer is to not let anyone have it in the first place: Write your application as a web app, (I recommend using django) and only your web hosting provider has access to your code.
py2exe
On windows py2exe is one way of shipping code to end-users, py2exe bundles the python interpreter, the necessary dlls and your code compiled to python bytecode.
Here are the python bytecode instructions to get some clue what it looks like:
http://www.python.org/doc/2.5.2/lib/bytecodes.html
Or you can use dis to disassemble some pyc/pyo files.
So, using py2exe is similar to distributing compiled python (pyc/pyo) files.
Shedskin C++ compiler
The Shedskin compiler compiles a subset of python to C++ which you can compile to native code using any compiler.
pypy
I don't know about PyPy too much. According to their docs Pypy is able to generate C code.