I have a two questions regarding best practices and technologies being used in banking apps and similar "high risk" applications.
Let's say we were trying to build a banking app on which the customer can register an account with his or her credentials and transfer and receive money. Just built like every common banking app with authentication mechanisms, PUSH TAN and other verification methods prior to transfering money, session login which expires after a given time period of inactivity which results in automatically logging out and fingerprint authentication and so on...
QUESTIONS:
What system level architecture patterns (High Level Design) are used as the foundation for such applications?
And what (Stack) programming languages, db querying languages and APIs are typically used for such high risk applications like banking apps (maybe languages and FWs like Python and Django, MySQL, Cassandra)?
I'm very thankful for every answer so don't hesitate to throw in whatever you know.
Here are my thoughts around this:
In general, high-risk systems (banking apps, exchanges) should follow strict requirements in terms of security, availability, reliability, correctness. If one of those features is not carefully considered, bad things can happen (people lose money). This usually leads to higher costs, compared to other types of applications.
In terms of architecture, if you want to go with the current trends, you can use microservices. For example Monzo use a microservice-based approach to run their backend (this nice presentation goes into more details). This is definitely the new way of building low-latency scalable backend systems (even for high-risk apps). One downside that I see with this approach (I may be wrong) is the lack of maturity (compared to classical client-server model) and the fact that you always need to be up to date with the open-source technologies that are involved, because they are in a constant change. Big companies have dedicated teams which contribute and stay up to date with open-source technologies.
You can always go with a client-server architecture where the backend is just a fleet of nodes which process requests and talk to a database. This might be a bit simpler to start with, but probably harder to evolve. Choosing the architecture depends entirely on the development strategy that you plan for your system. There are pros and cons on any approach.
In terms of tech stack, I would go with mature languages/frameworks, like Angular, Java, C#, Spring Boot. That way you can find more easily developers to help you out, but also an ecosystem built by the industry over the years.
In terms of database, you could go with a relational solution (MySQL, SQL Server) and use transactions to provide atomic updates (to account balances, for example). This may hit the overall performance at some point, in terms of latency / throughput. A No-SQL approach may improve the speed, but may also come with data duplication, and some challenges in choosing the right data model.
As you can see, there are tradeoffs everywhere. It only depends on the initial strategy you set up, so you can avoid big future changes.
Related
I'm currently about to start designing a new application.
The application will allow a user to insert some data and will provide data analysis (with reports as well), i know it's not helpful but the data-processing will be done in post-processing so that's not really interesting for the front-end.
I'd like to start with the right path to help myself when there will be the need to scale to handle more users.
I'm thinking about PostgreSQL to store the data, because I've already used it and I like it (also if a NoSQL would be a good choice -since not all data needs to have a relation- I like the Postgres support and community and I feel better knowing that there's a big community out there to help me), MySQL (innodb) is also a good choice, tbh I've not a real reason to choose it over PostgreSQL and vice-versa (is maybe MySQL easier to create shards?).
I know several programming languages but my strengths are Python, C/C++, Javascript.
I'm not sure if I should choose a sync or async approach for this task (I could scale out by running more sync applications behind a load balancer).
I've already developed another big-size project that teached me a lot of things about concurrency, but there each choice was influenced according to the (whole rest of the team, but mostly by the) sysadmin skills, so we have used python (django) + uwsgi + nginx.
For this project (since it's totally different from the other - that was an e-commerce, this is such a SaaS) I was also considering to make use of node.js, it would be a good opportunity to try it out in a serious project.
The most heavy data processing would be done by post-processes so all the front-end (user website) would be mostly I/O (+1 to use an async enviroment).
What would you suggest?
ps. I must also keep in mind that first of all the project has to start, so I cannot only think about each possible design, but I should start writing code ASAP :-)
My current thoughts are:
- start with something you know
- keep it as simple as possibile
- track everything to find bottlenecks
- scale out
So it wouldn't really matter if I deploy sync or async, but I know async has much better performances, and each thing that could help me to get better results (ergo lower costs) is evaluable as well.
I'm curious to know what are your experiences (also with other technologies)...
I'm becoming paranoid about this scalability and I fear it could lead to a wrong design (it's also the first time I'm designing alone for a commercial purpose = FUD)
If you need some more info please let me know and I'll try go give to you an answer.
Thanks.
A good resource for all of this is http://highscalability.com/. Lots of interesting case studies about handling big web loads.
You didn't mention it but you might want to think about hosting it in the cloud (Azure, Amazon, etc). Makes scaling the hardware a little easier and it's especially nice if your demand fluctuates.
Here are some basic guidelines:
Use as much async processes as possible. Or atleast design it in such a way that it can be converted to be async.
Design processes such that they can be segregated on different servers. This also goes to above. Say you have a webapp that has some intensive processes. If this process is asynch; then the main webserver could queue the job and be done with. Then a seperate server could pick the job and process it. This way your main web servers are not affected. But if you are resource constrained, you could still run the background process on same server (till you have enough clients and then you can spawn it off to a diff. server)
Design for load balancing. So if you app useses sessions, then you should factor in how you will be replicating sessions or not. You dont have to - you could send the user to a diff. server and then forward all subsequent requests to that server. But you still have to design for it.
Have the ability to route load to different servers based on some predefined criteria. So for eg: since your app is a SAAS app, you could decide that certain clients will go to Environment1 and certain other clients will go to Environment2. Lot of the SAAS players do this. For eg Salesforce.
You dont necessarily have to do this from the get go - but having this ability will go a long way to scale your app when the time comes.
Also, remember that theses approaches are not exclusive. You should design your app for all these approaches; but only implement it when required.
Take a look at the book The Art of Scalability
This book was written by guys that worked with eBay & Paypal.
Tale a look at this excellent presentation on scalability patterns and approaches.
My company is highly dependent on Java and jsf; All projects since I was hired are implemented using them. But most of those projects are facing problems related to performance and availability. So am finally considering a shift to other technologies and I have tried to research in the net, and im about to decide to try python. But before i start i would like to hear ur answer that Python would solve me the performance problems we are facing.
To make things clear the performance problems we mostly face are related to glassfish server and page loading. We are currently using ice faces and have tried wood stock back then. Additionally I can't use .net for some policy related issues. And PHP is also out of question due to some security leaks experienced in earlier projects.
So am expecting to read the pros and cons related with performance and availability in trying to convince my boss and customers in to python.
I have some doubts that you will gain performance by using Django or a Python based solution. I don't know the Glassfish server nor how it scales up but unless badly designed I don't see why it should perform badly.
From the explanation of your performance issues, it doesn't seem to be a problem of language speed but instead, server configuration and availability.
Assuming that your Java code is reasonably optimal (i.e. efficient and acceptably fast), you won't solve the problem by using some Python solution. Instead you should invest some time into studying caching mechanisms and/or proxy solutions.
Depending on how your server is setup, an additional advice would be to let all the static content be served by a dedicated server such as Apache, nginx or similar and only leave the the dynamic content on to be interpreted by your glassfish server.
Since your projects are written in Java you are in theory using a language that can potentially be faster than Python, I don't see why a Python solution would perform better unless there is something wrong with the framework you are using.
If you want to talk about prototyping or faster development, then that's a different subject discussed multiple times on stackoverflow.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
This will not be a "programming" question but more technology / platform related question. I'm trying to figure out whether Python can be a suitable Java alternative for enterprise / web applications.
Which are the ideal cases where you would prefer to use Python instead of Java? How would a typical Python web application (databases/sessions/concurrency) perform as compared to a typical Java application? How do specific Python frameworks square up against Java based frameworks (Spring, SEAM, Grails etc.)?
For businesses, switching from the Java infrastructure to a Python infrastructure .. is it too hard/expensive/resource intensive/not viable? Also shed some light on the business case for providing a Python + Google AppEngine based solution to the end customer. Will it be cost effective in an typical scenario?
Sorry if I am asking too wide a question, I would have liked to keep it specific, but I need your help to evaluate Python as a whole from the perspectives of the programmers, service providing company and end business customer.
For an SME, a Python/GoogleAppEngine based technology stack is a clear scalable and affordable platform. But what about a large MNC that already has a lot invested in Java.
Thank you so much. I am researching this myself and will gladly share my conclusions here!
Thank you,
Srirangan
An enterprise that already has a terabucks of Java investments should add jython to their mix of technologies -- it can be adopted gradually and progressively, at first for ancillary functions such as testing, "one-off" data migrations &c, prototyping of new functionality, cases in which using some existing open source Python library is obviously very handy, and so on, and so forth -- then, as the many Java developers in the company learn to use Jython, some of the prototypes will just be put in production as Jython code because there would be no advantage recoding them, some old subsystem needing recoding will be recoded in Jython, and so forth.
It's never really a wise decision to throw away a huge existing and working codebase and the ginormous investment it represents -- Python's strengths include its wealth of strong, production-level implementations, how well they "play with others", and how well Python can gradually and incrementally infiltrate most any development shop.
The larger your investment in an existing technology is, the more expensive it is to move away from it. COBOL is perhaps the best example here.
That investment isn't just in porting existing solutions, but also in retraining or hiring new staff so that you have the skill sets to build and support the new technologies even while still maintaining your legacy solutions.
Add to that the fact that for most large Multinational Corporations, software isn't their core business. As long as it functions effectively and fulfills the business need, they don't tend to care so much about the 'details'.
You need to be able to offer some pretty compelling benefits to overcome this kind of inertia.
Sad but true.
If you need to do the sort of things you can do with Django, then Django and Python is totally what you want. Google App Engine runs Django as well. So, you can do a Django app and host it on Google App Engine, and later change your mind and switch to conventional server hosting, or self-hosting if you have your own server.
I haven't tried Google App Engine but my understanding is that the price is quite reasonable for what you get. Google's IT department does a great job of keeping their data centers going; if you outsource the hosting to Google App Engine you know your data is backed up, you know the servers won't go down, and even if a backhoe takes a whole Google data center off the Internet, some other Google data center will keep serving up your application to your customers. You also know that if your application suddenly becomes hugely popular, Google App Engine wil l scale up automatically to handle the load. (I think you set a cap for the maximum you are willing to pay, and it scales until it hits the cap. But as I said I haven't used it and I'm not certain.)
I haven't used Java yet, but from what I have seen of it, Python is a much more expressive language and skilled Python coders can get more work done in a day just because the language is that much better. However, if you already have invested in Java and have in-house expertise in Java, you would be crazy to walk away from that overnight. The correct thing is to pick one new project to just try out that crazy Python thing.
And I really do recommend Django. You can get the Django book and try out the tutorial. If your first pilot project in Python is a Django project, you should have an easy time of things.
The answer to your question is yes. Python can be well suited for Enterprise because python is a language which has raw power, flexible and can be glued with other programming languages. What enterprise really requires is a language which does everything and i feel python is already enterprise ready. If you want examples then i believe there can be no bigger example than google. Google is running python internally and externally for its business critical applications. The only problem with python is that it is not very well recognized by top MNC company and we as a python programmer find hard time convincing the management team. I guess you will face the same issue. But i guarantee you once you get your feet wet in python, you will understand its true power
There is -- almost -- no usable "Business Case" for any technology choice.
"what about a large MNC that already has a lot invested in Java" Ask around. See if there's a business case for Java.
I doubt you'll find anything. Most companies drift into technology choices slowly.
There was no business case for COBOL -- it was the only game in town in the olden days.
There is rarely a business case for Java. What usually happens is that some visionary individual started building the first web site (probably in Perl). The "web thing" gained traction, and some vision individual started building web sites in Java. Eventually, the success of those small teams indicated to others that Java had advantages over COBOL.
Managers say the words "make a business case", but watch what they actually do. They listen to (1) their peers, (2) successful people.
To make the "business case" for Python, you have to be that visionary individual.
1) Use Python.
2) Be successful.
3) Share your successes.
4) Be prepared to explain that your success is due to your tools, not your personal level of genius and charisma.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 9 years ago.
Improve this question
Businessmen typically want a web application developed. They are aware of .net or J2EE by names, without much knowledge about either.
Altho' Rails and Django offer for a much better and faster development stack, it is a big task to convince businessmen to use these platforms.
The task begins with introducing Django (or Rails), quoting some blog/research. Then making a case for the use of the framework for the specific project.
Lot of the task is repetitive. What are the sources/blogs/whitepapers and other materials you use to make a case for django (or Rails)
Don't you think there should be a common brochure developed that many development agencies could use to make the same case, over and again. Are there any such ones, now?
There seems to be enough discussion on Django vs Rails. Whereas the need is (Django and Rails) vs (.net and J2EE), at least so, while making a business case. Both represent a faster pragmatic web development in a dynamic language.
It's easier to ask forgiveness than permission.
First, build the initial release in Django. Quickly. Build the model well (really well!). But use as much default admin functionality as you can.
Spend time only only reporting and display pages where the HTML might actually matter to the presentation.
Show this and they'll only want more. Once they've gotten addicted to fast turnaround and correct out-of-the box operation, you can discuss technology with them. By then it won't matter any more.
You need to speak the language of business: money.
"If we do it Rails, it will cost you 50% less than the same functionality in Java."
Your percentage may vary, and you might need to also include hosting and upkeep costs, to show how it balances out.
When you're convincing other programmers, sure, talk about development speed and automation of repetitive tasks. But talk bottom-line cost to a business person.
Before you begin making the case for Django or Rails, you have to be convinced it's the right stack first in the context of the business person's needs. If the business person is an entrepreneur, he may have other factors that go beyond how quickly can the solution be developed. For example:
If its an enterprise play that's being developed (something like SalesForce.com, SugarCRM, etc.) it may make sense to have it written in Java because this makes acquisitions and mergers easier with potential Java-based suitors.
If its an internal IT play for a custom solution in a large company, they may already have a significant amount MS infrastructure in place. It may not make sense to have your client install SQLServer or complicate their stack further with a Rails/Django friendly stack.
If you've cross this chasm and are convinced you have the client's best interest in mind, then I would look for examples on the Internet where the same application has been authored in both Java and Rails/Django. Here's an example of the Pet Store implemented in Rails.
http://www.anassina.com/projects/railspetstore/
You can download the source code and demonstrate to your client how much less code is needed to achieve the same result.
Explain to the client why less code is valuable: the less code you write, the fewer bugs you will have.
The first 2 arguments from the top of my mind:
Easier and faster development = cheaper product, less time to market.
SO optimization out of the box.
While many of you made some good suggestions, WRT the talks/resources for using these frameworks, you may also note to have a look at talk on redesigning yellow pages in ROR:
Summary from the site:
This talk explains how
YELLOWPAGES.COM, one of the
highest-traffic websites in the U.S.,
was written using Ruby on Rails, how
it was scaled to handle the traffic
and how the software architecture
evolved. Also: the reasons for
choosing Ruby on Rails.
The best case to be made for either of these frameworks is their ability to automate repetitive and time-consuming tasks. This allows developers to be faster and more productive which in turn means projects are delivered faster.
The problem with a "brochure" approach is that it doesn't address the clients needs. Putting the language/platform of choice into a presentation that addresses the clients goals is much more likely to sell them - both on the tools you want to use, as well as you as a provider. As long as you can show that your approach will solve the problem (preferably with the least amount of expense), you'll have fewer objections and less of the "but I've heard that xxx is the best".
Is Django a good choice for a security critical application?
I am asking this because most of the online banking software is built using Java. Is there any real reason for this?
Actually, the security in Java and Python is the same. Digest-only password handling, cookies that timeout rapidly, careful deletion of sessions, multi-factor authentication. None of this is unique to a Java framework or a Python framework like Django.
Django, indeed, has a security backend architecture that allows you to add your own LDAP (or AD) connection, possibly changing the digest technique used.
Django has a Profile model where you can keep additional authentication factors.
Django offers a few standard decorators for view function authorization checking. Since Python is so flexible, you can trivially write your own decorator functions to layer in different or additional authentication checking.
Security is a number of first-class features in Django.
Probably the reason behind Java is not in the in the security. I think Java is more used in large development companies and banks usually resort to them for their development needs (which probably are not only related to the web site but creep deeper in the backend).
So, I see no security reasons, mostly cultural ones.
The reasons for building banking apps in Java are not related to security, at least IMHO. They are related to:
Java is the COBOL of the 21st century, so there is a lot of legacy code that would have to be rewritten and that takes time. Basically banking apps are old apps, they were built in java some ten years ago and nobody wants to throw all the old code away (which BTW is almost always a bad decision),
some people believe that a static typed language is somewhat "safer" than the dynamic typed language. This is, IMHO, not true (take for instance collections prior to Java 5).
I find your connection between Java and banking wrong ended.
Most Banking Software has terrible security. And much banking software is written in Java. Does ths mean Java makes it more difficult to write secure software than other languages?
Probably it's not Java's fault that there is so little quality security (and safety) wise in Banking software. Actually, like the other posters mention, the choice of your Language usually has very little consequences for your security - unless you select one of the few languages where only hotshot coders can write secure code in (C and PHP come to mind).
Many huge E-Commerce sites are written in Python, Ruby and Perl using various frameworks. And I would argue that the security requirements for merchants are much higher than the requirements of the banking industry. That is because merchants have to provide security and good user experience, while banking customers are willing to put up with unusable interfaces SecureID tokens and whatever.
So yes: Django is up to the task.
You should not rely the security of the application on the framework. even though Django does come in with a pretty good number of measures against classical security issues, it can not guarantee that your application will be secure, you need much more than a programming Framework to get a security critical application.
I'd say yes, Django is a good choice as long as you know its powers and limitations and are aware of the security flaws of every application.
You can build a secure application with Django just as you can with any popular Java framework. One part where Java does shine is its extensive cryptographic library.
For the minimal encryption tasks that are required by Django, Python’s cryptographic services are sufficient, however its lack of strong block ciphers make the encryption mechanism in Django insecure for data at rest.
Python does natively support secure hashing algorithms to include SHA1, SHA224,
SHA256, SHA384, and SHA512, however Django’s authentication mechanism has yet
to be updated to use anything other than SHA1, making it potentially vulnerable to cryptographic analysis.
Are you referring to the fact that the complete application is built in Java, or just the part you see in your browser? If the latter, the reason is probably because in the context of webpages, Java applets can be downloaded and run.