Develop Reference

How to properly migrate a monolith architecture app to a microservice app in Django

Today I come with this question probably to someone who has large experience in this.
Basically what the title indicates. We have that app and we have to migrate it to microservices.
We didn't find any solid approach (or we felt it like that) about this. What we ended up doing is creating 1 project per microservice (a single functionality related to a module app, in general) but then we had some problems because we already had a database to work with since this is already a functioning app.
We had problems communicating with the existing models, so basically what we did was to point in every settings.py of the projects to the existing DB, and with python3 manage.py inspectdb, we grabbed the existing models. This approach ended up working, but we feel that is not the best approach. We had a lot of problems with circular imports and more.
Is out there good practices about microservices with Django, and how to properly do it, like in most of the cases that we want to create something with the framework?
If someone knows or has something we would really appreciate it!

you can use Django for Microservices. in this case you have only one few apps, and you start every service on own port:
first Django project
pdf generator + views generate small pdf
second Django project.
pdf generator + views generate big pdf (code can inherit other project)
orchestra:
third Django project: Autorization + call big or small pdf generator service
settings
settings.py for first and second is very easy and allows only call from internal ip's. here we don't need middleaware, template, cache, admin and other settings.
settings.py for orchestra is also very easy and used only auth and made call by internal ip ant send response to user. Here we don't need much middlaware, and don't need many other settings.
gains:
All is independent. if one server fall, other can work.
Updates are easy. One small server update is always easy than monolith update.
development is easy: three small teams works on own small projects.
Units testing is easy and fast
For complex business goals the whole system is faster.
pains:
after 100 micro-services it is completely complex to work with that all.
code style from many teams is always different. Don't matter how strict you define styleguide or settings for black-linter.
integrate Testing is difficult - If something not work, it is hard to find where.
Ecosystem Auth/services/messaging is really complex
For easy business goals the whole system is overcomplicated.
summary
Don't matter how much DB you want to use. it can be monolith or many services.
i don't see any problem to import in one Microservice something from other project: it can be model / admin or other staff. it works. probably you need to smart split monolith, but it also easy, for that we have a many experience (my own and in internet or books)

Is there a way to automatically generate a whole system for different users [duplicate]

I have developed Multi-tenant SAAS apps in PHP/Laravel but recently I had a challenge to develop one in Django/Python. I am still learning Django though and I really like Django rest framework (DRF). But I have difficulties to figure out the highlighted areas below, If someone shows some light, I will be good to go:
How to handle subdomains/domains and selecting the right tenant db
How to manage and dynamically handle different database in django
Can multi-tenant apps backend still be managed from Django admin interface
I will be using queues and other apps scalling techniques, need tips and tricks if any
Any example out there
Any challenge experience when developing SAAS through Django

Well...
django-subdomains
There are people who asked in SO questions about dynamic databases in django (including, ahem... me). I'm not entirely sure I understood what you mean by "dynamically handle different database" so I guess just go to the links I just mentioned and pick out the best one for your project. (also - perhaps this is more relevant?)
Check out django-multitenant-schemas and this answer too.
There was a video of the guys behind Disqus (one of the largest django app in the world) explaining how they handle scaling. Here another interesting article on the subject. Also, the legendary SO question Does Django Scale.
(and 6.) Check out this answer
I hope that's detailed enough. I know this might be a disappointing only-links answer, but this is the reality of it - Django is a newer framework, and web development with python is still less common than php. With that in mind, understand that for all the awesomness of django (and it is awesome), with more complex needs there's more you'll have to do yourself.
In this case, you'll have to figure out how to do each part of the way seperatly and then combine it all. You can easily find a way to create a REST django app for example, but then you'll need to figure out how to combine it with another package (such as the above subdomains).
You can find a million examples out there of people doing freaky things with django. It's really powerful (when I learned about dynamic models I was blown away). But the more complex your app, the more you'll need to do yourself.
Pick it up, one step at a time, and come back to SO with specific issues you're having (or the django users google group). Good luck!

How can I run my python script from within a web browser and process the results?

I have a written a short python script which takes a text and does a few things with it. For example it has a function which counts the words in the text and returns the number.
How can I run this script within django?
I want to take that text from the view (textfield or something) and return a result back to the view.
I want to use django only to give the script a webinterface. And it is only for me, maybe for a few people, not for a big audience. No deployment.
Edit: When I first thought the solution would be "Django", I asked for it explicitly. That was of course a mistake because of my ignorance of WSGI. Unfortunately nobody advised me of this mistake.

First off, is your heart really set on it being Django? If not I'd advise that Django, whilst an awesome framework, is a bit much for your needs. You don't really need full stack.
You might want to look at Flask instead, which is a Python micro-framework (and dead easy to use)
However, since you asked about Django...
You can create a custom Django command (docs here) that calls your script,
that can be called from a view as described in this question.
This has the added benefit of allowing you to run your script via the Django management.py script too. Which means you can keep any future scripts related to this project nice and uniform.
For getting the results of your script running, you can get them from the same bit of code that calls the command (the part described in the last link), or you can write large result sets to a file and process that file. Which you choose would really depend on the size of your result set and if you want to do anything else with it afterwards.

What nobody told me here, since I asked about Django:
What I really needed was a simple solution called WSGI. In order to make your python script accessible from the webbrowser you don't need Django, nor Flask. Much easier is a solution like Werkzeug or CherryPy.

After following the django tutorial, as suggested in a comment above, you'll want to create a view that has a text field and a submit button. On submission of the form, your view can run the script that you wrote (either imported from another file or copy and pasted; importing is probably preferable if it's complicated, but yours sounds like it's just a few lines), then return the number that you calculated. If you want to get really fancy, you could do this with some javascript and an ajax request, but if you're just starting, you should do it with a simple form first.

Do I need to use a web framework for a simple website?

The site won't be that complicated and will resemble a modern blog (users, messages, news and other similar features).
Do I need to use a framework for this, and if so, which is best?
Pyramid, Django?

You certainly don't need a webframework to create a simple website. Given that you're new to python and interested in building a python website, I imagine this implies: you're interested in learning python. If you're exclusively interested in learning django-python, there's no reason you can't jump in to django, as Ronak said, of course. He's right. It has a lot of documentation. But it will make for somewhat of an odd intro to python.
If I were in your shoes, I'd either start making some offline programs first, or consider an ultra-lightweight framework. Many would advocate web2py or pyramid for ultralightweight. I might consider going even lighter. Something like Bottle, where you're more or less just pairing functions with urls. This way you can at least do a bit of hacking/trial-and-error, instead of launching right into django.
It's not that django doesn't use python-- it will tell you many times that it is in fact 'just python.' But it's adapted at its core to be used in a large business setting (the chicago something or other online, i think). So it enforces various rules that are helpful in managing many different employees working on a project together. You may or may not wish for this kind of 'help.' It also means the scale of projects is assumed to be large and the time-horizon, limitless. If you want to see how a python dictionary works, you may not want to spend a long time configuring settings and creating the pseudo-static-typing you need for your database, and so on, just to execute your project and see a result.
I realize I will automatically get downvoted for this, but I believe it to be sound advice.

It depends on what kind of website you are planning to come up with. If the website is going to be just a set of static HTML files, then you don't really need a framework. But if your website will have lots of dynamic content that will get updated on regular basis, you should go with some framework. That will make your life maintaining the website much more simpler.
Django is the most popular framework written in Python. It has very good documentation and a strong community base too.

Go with Django - 10,000 Elvis fans can't be wrong.
Or roll your own from scratch. You'll learn a lot, know everything about how you site works, and better appreciate what a framework does for you.

As RonakG first pointed, it all depends on the kind of website you intend to have up and running. Actually, your question is too general for a single, definitive answer. There are more aspects to consider other than just being in python. For example, deadlines. This means considering the learning curve to achieve your results. If you don't have much time, a steep learning curve (time to learn it in order to develop it) is certainly something you will want to avoid. Perhaps you already develop in other languages, and need integration and/or migration support, need scalability, reusability, etc, etc, etc.
Another thing that is not so clear in your question is what you mean by "The site won't be that complicated and will resemble a modern blog (users, messages, news and other similar features)". If it really resembles just a modern blog, with users, messages and news, you could google for CMS (Content Management Systems). There are many options available, that could make you have your site up and running in almost no-time. All you'll have to learn is how to customize whatever it has to as to comply to your needs.
That said, if you prefer python, there are some good CMSs available which you can develop your site fast, like Plone. And if you prefer Django, there's Django CMS and there's the excellent Pinax project, which takes the django code reusability to deliver you sample fully customizable, complete websites.

Django, Turbo Gears, Web2Py, which is better for what?

I got a project in mind that makes it worth to finally take the plunge into programming.
After reading a lot of stuff, here and elsewhere, I'm set on making Python the one I learn for now, over C# or java. What convinced me the most was actually Paul Graham's excursions on programming languages and Lisp, though Arc is in the experimental stage, which wouldn't help me do this web app right now.
As for web app fast, I've checked out Django, Turbo Gears and Py2Web. In spite of spending a lot of time reading, I still have no clue which one I should use.
1) Django certainly has the nicest online presence, and a nicely done onsite tutorial, they sure know how to show off their thing.
2) Web2Py attracted me with its no-install-needed and the claim of making Django look complicated. But when you dig around on their website, you quickly find content that hasn't been updated in years with broken external links... There's ghosts on that website that make someone not intimately familiar with the project worry if it might be flatlining.
3) Turbo Gears ...I guess its modular too. People who wrote about it loved it... I couldn't find anything specific that might make it special over Django.
I haven't decided on an IDE yet, though I read all the answers to the Intellisense code completion post here. Showing extra code snippets would be cool too for noobs like me, but I suppose I should choose my web frame work first and then pick an editor that will work well with it.
Since probably no framework is hands down the best at everything, I will give some specifics on the app I want to build:
It will use MySQL, it needs register/sign-in, and there will be a load of simple math operations on data from input and SQL queries. I've completed a functional prototype in Excel, so I know exactly what I want to build, which I hope will help me overcome my noobness. I'll be a small app, nothing big.
And I don't want to see any HTML while building it ;-)
PS: thanks to the people running Stackoverflow, found this place just at the right moment too!

You should look at the web2py online documentation (http://web2py.com/book). It comes with a Role Based Access Control (the most general access control mechanism) and it is very granular, you can grant access for specific operation on specific records. It comes with a web based IDE but you can use WingIDE, Eclipse and PyCharm too. It comes with helper system that allows you to generate HTML without using HTML. Here is an example of a complete app that requires users to register/login/post messages:
db.define_table('message',Field('body'),Field('author',db.auth_user))
#auth.requires_login()
def index():
db.message.author.default=auth.user.id
db.message.author.writable=False
return dict(form=crud.create(db.message),
messages=db(db.message.id>0).select())
The web2py project is very active as you can see from the list of changes http://code.google.com/p/web2py/source/list
If you have web2py related questions I strongly suggest you join the web2py mailing list:
http://groups.google.com/group/web2py/topics
We are very active and your questions will be answered very quickly.

I have to say as not particularly skilled developer, the speed at which I have been able to create using web2py has blown my mind. In large part due to the amazing community and the core value Massimo has of making the framework accessible.
When I started I had written 0 lines of code in Python
Never heard of web2py
I've been at it seriously for about a month and have progressed (in my usual fashion) from asking questions that no one could answer (because they didn't make any sense) to coding for hours at a time without picking up a book or asking a question.
I'm really impressed.

I've had positive experiences with Django.
Built-In Authentication and easy to use extensions for registration
Very good documentation
You probable write your HTML templates mostly in base.html, then just use template inheritance (Note: You'll need to write at least a little bit of HTML)
In contrast to Turbogears, Django is more 'out-of-the-box'
I don't have any experience with web2py, but from my impression, it tries to do a little to much 'out-of-the-box'

If you decide to go with Django, make sure that you use its Generic Views. They will save you from writing lots of code, both Python and HTML.
Also, unless there is a very specific reason for you to use MySQL, I advise you to switch to PostgreSQL. Django is much more oriented towards PostgreSQL and it's a much better database anyway.
The online Django documentation is great, this is what put it apart from all the other frameworks. I also recommend the book Practical Django Projects by James Bennett

Django: Heard it has the best administrative
interface. But uses it's own ORM, i.e. doesn't use SQL-Alchemy.
Web2py: Didn't research this.
Turbogears2:
Uses SQL-Alchemy by default, uses Catwalk for admin
interface, but documentation isn't as
great.
I chose Turbogears2 because it uses popular components, so I didn't have to learn anything new...

I've used both web2py and RoR extensively, and while RoR has gotten a lot of popularity and support in the past few years, web2py is simpler, cleaner, less "magical", and yet also offers more (useful) out-of-the-box functionality. I'd say that web2py has more potential than RoR, but it is a relatively new framework and does yet not have the maturity of RoR. (Despite that, though, I'd choose web2py over RoR any day...)

If you "don't want to see any HTML while building it" then you can forget Django. It is not focused on "point-click-done," it is focused on pros going from concept to production in the shortest time possible. The hierarchical nature of the templating language can lead to some very clean overall site layouts. I use Django for all of my larger sites and I love it.
Although it's written in PHP, not Python, you might take a look at the major new version of WordPress that came out about 2 or 3 months ago. In 3.0 they have come a long way from being a "blogs only" environment and there are tons of ready-made templates for it. Of course if you want to tweak a template, well, there's that nasty old HTML again. I am considering using it for my smaller clients that can't deal with the admin of a dedicated server, etc., that tends to come with a Django site.
Update:
Ah, I missed the semi-joke -- I was up too early and that tends to make me tone deaf to humor. As far as using templates from existing sites, I have done this quite successfully with a couple of sites, both those that were static and those originally driven by well-written PHP scripts. I recommend a careful reading of the {% extends %} and {% include %} docs. Both take either a string literal or a variable. I have used the later method and it can be quite useful for a site that has strong hierarchy distinguished by style changes across branches.
It is also worth the time to understand the search order for templates -- it can be used to good effect, but it can be puzzling if you don't grok it. See the template-related items in the settings.py file for this and other useful goodies.