I've been looking really hard at all of the way**(s)** one can develop web applications using Python. For reference, we are using RHEL 64bit, apache, mod_wsgi.
History:
PHP + MySQL years ago
PHP + Python 2.x + MySQL recently and current
Python + PostgreSQL working on it
We use a great library for communicating between PHP and Python (interface in PHP, backend in Python)... However, with a larger upcoming project starting, using 100% python may be very advantagous.
We typically prefer not to have a monolithic framework dictating how things are done. A collection of useful helpers and utilities are much preferred (be it PHP or Python).
Question 1:
In reading a number of answers from experienced Python users, I've seen Werkzeug recommended a number of times. I would love it if several people with direct experience using Werkzeug to develop professional web applications could comment (in as much detail as their fingers feel like) why they use it, why they like it, and anything to watch out for.
Question 2:
Is there a version of Werkzeug that supports Python 3.1.1. I've succefully installed mod_wsgi on Apache 2.2 with Python 3.1.1.
If there is not a version, what would it take to upgrade it to work on Python 3.1?
Note: I've run 2to3 on the Werkzeug source code, and it does python-compile without
Edit:
The project that we are starting is not slated to be finished until nearly a year from now. At which point, I'm guessing Python 3.X will be a lot more mainstream. Furthermore, considering that we are running the App (not distributing it), can anyone comment on the viability of bashing through some of the Python 3 issues now, so that when a year from now arrives, we are more-or-less already there?
Thoughts appreciated!
mod_wsgi for Python 3.x is also not ready. There is no satisfactory definition of WSGI for Python 3.x yet; the WEB-SIG are still bashing out the issues. mod_wsgi targets a guess at what might be in it, but there are very likely to be changes to both the spec and to standard libraries. Any web application you write today in Python 3.1 is likely to break in the future.
It's a bit of a shambles. Today, for webapps you can only realistically use Python 2.x.
I haven't used Werkzeug, so I can only answer question 2:
No, Werkzeug does not work on Python 3. In fact, very little works on Python 3 as of today. Porting is not difficult, but you can't port until all your third-party libraries have been ported, so progress is slow.
One big stopper has been setuptools, which is a very popular package to use. Setuptools is unmaintained, but there is a maintained fork called Distribute. Distribute was released with Python 3 support just a week or two ago. I hope package support for Python 3 will pick up now. But it will still be a long time, at least months probably a year or so, before any major project like Werkzeug will be ported to Python 3.
I can only answer question one:
I started using it for some small webstuff but now moved on to rework larger apps with it. Why Werkzeug? The modular concept is really helpful. You can hook in modules as you like, make stuff easily context aware and you get good request file handling for free which is able to cope with 300mb+ files by not storing it in memory.
Disadvantages... Well sometimes modularity needs some upfront thought (django f.ex. gives you everything all at once, stripping stuff out is hard to do there though) but for me it works fine.
Related
Currently I am working in big firm where we need to convert python2 old big Django project into python3 version so I have done lots of research related but still not able to find any perfect answer related to which version of Python and Django best suited for conversion.
Currently I am using Python : 2.7.16 & Django : 1.9.13 in my old version.
Anyone can suggest me best suited version of Python & Django for above old version for python2 to python3 conversion.
I thought I'd add a bit to the strategy advocated by Wim's answer - get the appropriate version of Django working on both 2.7 and 3.x first - and outline some tactics that worked for me.
Python 2.7 is your escape pod, until you pull the trigger on 3.x
your tests should run on both
don't use any 3.x specific features, like f-strings
first Python 3.x, then only later Django 2.x which doesn't run on 2.7
start early, don't over analyze, but avoid the big bang approach
file by file at first.
start with the lowest level code, like utility libraries, that you have test suites for.
if possible, try to gradually merge your changes to the 2.7 production branches and keep your 3.x porting code up to date with prod changes.
Which minor version of Django to start with?
My criteria here is that Django migrations can be fairly involved (and actually require more thinking than 2=>3 work). So I would move to the latest and greatest 1.11 that way you're already providing some value to your 2.7 users. There's probably a good number of pre-2.x compatibility shims on 1.11 and you'll be getting its 2.x deprecation warnings.
Which minor version of Python 3.x to start with?
Best to consider all angles, such as the availability of your 3rd party libs, support from your CI/devops suite and availability on your chosen server OS images. You could always install 3.8 and try a pip install of your requirements.txt by itself, for example.
Leverage git (or whatever scm you use) and virtualenv.
separate requirement.txt files, but...
if you have a file-based, git repo, you can point each venv at the same codeline with a pip install -e <your directory>. that means that, in 2 different terminals you can run 2.7 and 3.x against the same unittest(s).
you could even run 2.7 and 3.x Django servers side-by-side on different ports and point say Firefox and Chrome at them.
commit often (on the porting branch at least) and learn about git bisect.
make use of 2to3
Yes, it will break 2.7 code and Django if you let it. So...
run it in preview mode or against a single file. see what it breaks but also see what it did right.
throttle it to only certain conversions that don't break 2.7 or Django. print x=> print (x) and except(Exception) as e are 2 no-brainers.
This is what my throttled command looked like:
2to3 $tgt -w -f except -f raise -f next -f funcattrs -f print
run it file-by-file until you are really confident.
use sed or awk rather than your editor for bulk conversions.
The advantage is that, as you become more aware of your apps' specifics concerns, you can build a suite of changes that can be run on either 1 file or many files and do most of the work without breaking 2.7 or Django. Apply this after your suitably-throttled 2to3 pass. That leaves you with residual cleanups in your editor and getting your tests to pass.
(optional) start running black on 2.7 code.
black which is a code formatter, uses Python 3 ASTs to run its analysis. It doesn't try to run the code, but it will flag syntax errors that prevent it from getting to the AST stage. You will have to work some pip install global magic to get there though and you have to buy into black's usefulness.
Other people have done it - learn from them.
Listening to #155 Practical steps for moving to Python 3 should give you some ideas of the work. Look at the show links for it. They love to talk up the Instagram(?) move which involved a gradual adjustment of running 2.7 code to 3.x syntax on a common codebase, and on the same git branch, until pull-the-trigger day.
See also The Conservative Python 3 Porting Guide
and Instagram Makes a Smooth Move to Python 3 - The New Stack
Conclusion
Your time to Django 1.11 EOL (April 2020) is rather short, so if you have 2+ dev resources to throw at it, I'd consider doing the following in parallel:
DEV#1: start out on a Django 1.11 bump (the theory being that Django 1.11 is probably best positioned as a jump off point to Django 2.x), using 2.7.
DEV#2: get started on Python 3.6/3.7 of your non-Django utility code. Since the code is 2.7 compatible at this point, merge it into #1 as you go.
See how both tasks proceed, assess what the Django related project risk is and what the Python 3 pain looks like. You're already missing the Python 2.7 EOL, but an obsolete web framework is probably more dangerous than legacy Python 2.7, at least for a few months. So I wouldn't wait too long to start migrating off Django 1.9 and your work doing so won't be wasted. As you see the progress, you'll start seeing the project risks better.
Your initial 2to3 progress will be slow, but the tooling and guidance is good enough that you'll quickly pick up speed so don't overthink it before starting to gather experience. The Django side depends on your exposure to breaking changes in the framework which is why I think it's best to start early.
P.S. (controversial/personal opinion) I didn't use six or other canned 2-to-3 bridge libraries much.
It's not because I don't trust it - it's brilliant for 3rd party libs - but rather that I didn't want to add a complex permanent dependency (and I was too lazy to read its doc). I'd been writing 2.7 code in 3.x compatible syntax for a long time so I didn't really feel the need to use them. Your mileage may vary and don't set out on this path if it seems like a lot of work.
Instead, I created a py223.py (57 LOC incl. comments) with this type of content, most of which is concerned with workarounds for deprecations and name changes in the standard library.
try:
basestring_ = basestring
except (NameError,) as e:
basestring_ = str
try:
cmp_ = cmp
except (NameError,) as e:
# from http://portingguide.readthedocs.io/en/latest/comparisons.html
def cmp_(x, y):
"""
Replacement for built-in function cmp that was removed in Python 3
"""
return (x > y) - (x < y)
Then import from that py223 to work around those specific concerns. Later on I will just ditch the import and move those weird isinstance(x, basestr_) to isinstance(x, str) but I know in advance there is little to worry about.
My suggestion is to first upgrade to Django==1.11.26, which is the most recent version of Django that is supporting both Python 2 and Python 3. Stay on your current version of Python 2.7 for now.
Read carefully the release notes for 1.10.x and 1.11.x, checking for deprecations and fixing anything that stopped working from your 1.9.x code. Things WILL break. Django moves fast. For a large Django project, there may be many code changes required, and if you're using a lot of 3rd-party plugins or libraries you may have to juggle their versions around. Some of your 3rd-party dependencies will probably have been abandoned entirely, so you have to find replacements or remove the features.
To find the release notes for each version upgrade, just google "What's new in Django ". The hits will meticulously document all the deprecations and changes:
https://docs.djangoproject.com/en/2.2/releases/1.10/
https://docs.djangoproject.com/en/2.2/releases/1.11/
Once the webapp appears to be working fine on Django 1.11, with all tests passing (you do have a test suite, right?) then you can do the Python 3 conversion, whilst keeping the Django version the same. Django 1.11 supports up to Python 3.7, so that would be a good version to target. Expect unicode all over the place, since the implicit conversions between bytes and text is gone now and many Python 2 webapps relied upon that.
Once the project appears to be working fine on Django 1.11 and Python 3.7, then you can think about upgrading to Django 3.0, following the same process as before - reading the release notes, making the necessary changes, running the test suite, and checking out the webapp in a dev server manually.
I would upgrade to py3 first. You'll need to look at setup.py in the Django repo on the stable/1.9.x branch (https://github.com/django/django/blob/stable/1.9.x/setup.py) to figure out that the py3 versions supported are 3.4 (dead) and 3.5.
Once you're on py3.5 and Django 1.9 you can upgrade one at a time until you get to the version you want to end at. E.g. Django 1.11 supports py3.5 and py3.7, so
py27/dj19 -> py35/dj19 -> py35/dj1.11 -> py37/dj1.11 ... -> py37/dj2.2
dj2.2 is the first version supporting py3.8, but I would probably stop at py37/dj2.2 if you're working in a normally conservative environment.
If you have other packages you'll need to find version combinations that will work together on each step. Having a plan is key, and upgrading only one component at a time will usually end up saving you time.
The future library (https://python-future.org/) will help you with many icky situations while you need code to run on both py27 and 3.x. six is great too. I would avoid rolling your own compatibility layer (why reinvent the wheel?)
If at all possible, try to get your unit test coverage up to 75-85% before starting, and definitely set up automatic testing on both "from" and "to" versions for each upgrade step. Make sure you read and fix all warnings from Django before upgrading to the next version -- Django cares very little about backward compatibility, so I would normally suggest hitting every minor version on the upgrade path (or at least make sure you read the "backwards incompatibilities" and deprecation lists for each minor version).
Good luck (we're upgrading a 300+Kloc code base from py27/dj1.7 right now, so I feel your pain ;-)
I have same kind of issue with my project and I have tried python 3.7.5 with Django version 2.2.7.
You should not go with python latest version 3.8 or Django latest version 3.0 because you there may have been chances that for any kind of bug you may not able to get proper solution for latest versions.
You should try to shoot for the current versions. Python 3.8 and Django 3.0.The Six library will help with some convention changes. Either way you are going to have to do some refactoring so you might as well make it current.
I'm currently testing frameworks to create a big multiplayers game. I choose Django.
But I have a question about the version of Python. I should to create that project from scratch with Python 3.x or Python 2.x?
Python 3.x and Django compatibly is ok, or not production usable for now?
I wouldn't highly suggest going production with Python 3 with Django or for that matter any other framework that requires you to depend on many third party applications, although many have been ported to Python 3, you still may find bugs, which you will likely have to fix or wait awhile before maintainers get to it.
Also, there aren't many compelling reasons to move to Python 3 at the moment, but that I suspect that will change soon with all the asynchronous work being put into Python 3.
Django is compatible with Python 3.
There's at least one issue with Python3 + Django (1.6) + MySQL.
MySQLdb hasn't been ported to Python3 yet.
The other python-only mysql connector mentioned in the Django documentation (MySQL Connector/Python) has a bug in it which might stop it working with fixtures.
http://bugs.mysql.com/bug.php?id=72001
It looks like Oracle has closed this bug report by documenting that the problem may occur; so a real fix doesn't look likely any time soon.
So if you rely on fixtures and use MySQL, you'll likely have a problem.
Try using the latest GA version of mysql-connector-python.
Is 2.x still the norm or would you recommend just coding in v3 at this point?
Python 3 is still a long way off having universal support from tools, libraries and distros, so its use in production would depend very much on whether the bits you need (or are likely to need in the near future) have been ported.
For exploratory, educational and other uses, it depends very much on your own proclivity for living on the bleeding edge. If you are happy building from source and debugging and hacking third-party libraries to get things working, then you'll probably have no issues with Python 3. Otherwise, stick to the latest your distro offers, and if it is stuck on a really old Python — CentOS is still on 2.4! — you have my commiserations. Personally, I steer clear of CentOS for precisely this reason.
Google App Engine documentation states it uses Python 2.5
Today I happened to notice that Python Imaging Library is till not released for 3.x.
So, if you need those libraries or services, I guess you should wait.
Do the frameworks and libraries you use have Python 3 versions? Libraries you use for development, and does your deploy stack support Python 3?
Many Python projects have a lot of dependancies, especially web based projects, most of which aren't Py3K ready yet.
If your stack is good, sure - Python 3 is the future, might as well embrace it now.
My main use of Python is Django. Support for 3.x for this framework still lies in the future, unfortunately, and who knows about any related modules - so no, it's not quite time for many people.
I actually bought Python 3 books last year when I started learning Python, thinking "I'll just start with 3 from the beginning!". That didn't work out, though.
I've always used v3 primarily. "Hacking 3rd party libraries" to me is just like importing any other module. The only thing is since most stuff still uses v2 you have to know both versions and keep them straight when looking at others code.
One of the feature I like in RoR is the db management, it can hide all the sql statement, also, it is very easy to change different db in RoR, is there any similar framework in Python 3000?
This answer was awfully outdated. The current state of afairs is:
Django is close to supporting Python 3
CherryPy supports Python 3 since version 3.2
Pyramid has Python 3 support since 1.3
Bottle, which is a lightweight WSGI micro web-framework, supports Python 3
I'm sure this list will keep growing every coming month, specially considering that there will never be a Python 2.8.
2.7 will be the end of the line for Python 2 development, and now the official upgrade path from 2.7 is Python 3.x. I'm sure that with this state of affairs, Python 3 support from web frameworks is only going to get better and better.
[OUTDATED]
Python 3 is not yet in high deployment. It's still lacking a lot of third party libraries.
The recommended Python version is 2.6.x, as it's the most current, it's backwards compatible, and has many backported features from 3.1.
For Python 2.6 you will find quite a few frameworks:
Django
Turbogears
CherryPy
Zope
and many more
I believe CherryPy is on the verge of being released for Python 3.X.
Python 3 isn't ready for web applications right now. The WSGI 1.0 specification isn't suitable for Py3k and the related standard libraries are 2to3 hacks that don't work consistently faced with bytes vs. unicode. It's a real mess.
WEB-SIG are bashing out proposals for a WSGI revision; hopefully it can move forward soon, because although Python 3 isn't mainstream yet it's certainly heading that way, and the brokenness of webdev is rather embarrassing.
Python 3 is not ready for practical use, because there is not yet enough libraries that have been updated to support Python 3. So the answer is: No.
But there are LOADS of them on Python 2. Tens, at least.
Django, Turbogears, BFG and of course the old man of the game: Zope. To tell which is best for you, you need to expand your requirements a lot.
There's Django but it works with Python 2.3+ only for now.
I found http://www.iseriespython.com/, which is a version of Python for the iSeries apparently including some system specific data access classes. I am keen to try this out, but will have to get approval at work to do so. My questions are:
Does the port work well, or are there limits to what the interpreter can handle compared with standard Python implementations?
Does the iSeries database access layer work well, creating usable objects from table definitions?
From what I have seen so far, it works pretty well. Note that I'm using iSeries Python 2.3.3. The fact that strings are natively EBCDIC can be a problem; it's definitely one of the reasons many third-party packages won't work as-is, even if they are pure Python. (In some cases they can be tweaked and massaged into working with judicious use of encoding and decoding.) Supposedly 2.5 uses ASCII natively, which would in principle improve compatibility, but I have no way to test this because I'm on a too-old version of OS/400.
Partly because of EBCDIC and partly because OS/400 and the QSYS file system are neither Unix-like nor Windows-like, there are some pieces of the standard library that are not implemented or are imperfectly implemented. How badly this would affect you depends on what you're trying to do.
On the plus side, the iSeries-specific features work quite well. It's very easy to work with physical files as well as stream files. Calling CL or RPG programs from Python is fairly painless. On balance, I find iSeries Python to be highly usable and very worthwhile.
Update (2012): A lot of work has gone into iSeries Python since this question was asked. Version 2.7 is now available, meaning it's up-to-date as far as 2.x versions go. A few participants of the forum are reasonably active and provide amazing support. One of them has gotten Django working on the i. As expected, the move to native ASCII strings solves a lot of the EBCDIC problems and greatly increases compatibility with third-party packages. I enthusiastically recommend iSeries Python 2.7 for anyone on V5R3 or later. (I still strongly recommend iSeries Python 2.3.3 for those who are on earlier versions of the operating system.)
Update (2021): Unfortunately, iSeriesPython is no longer maintained, and the old website and forum are gone. You can still get the software from its SourceForge repository, and it is still an amazingly useful and worthwhile asset for those who are stuck on old (pre-7.2) versions of the operating system. For those who are on 7.2 or newer, there is a Python for PASE from IBM, which should be considered the preferred way to run Python on the midrange platform. This version of Python is part of a growing ecosystem of open source software on IBM i.
It sounds like it is would work as expected. Support for other libraries might be pretty limited, though.
Timothy Prickett talks about some Python ports for the iSeries in this article:
http://www.itjungle.com/tfh/tfh041706-story02.html
Also, some discussion popped up in the Python mailing archives:
http://mail.python.org/pipermail/python-list/2004-January/245276.html
iSeriesPython is working very well.
We are usning it since 2005 (or earlier) in our Development and Production Environments as an utility language, for generating of COBOL source code, generating of PCML interfaces, sending SMS, validating/correcting some data ... etc.
With iSeriesPython you can access the iSeries database at 2 ways: using File400 and/or db2 module. You can execute OS/400 commands and you can work with both QSYS.LIB members and IFS stream files.
IMHO, iSeries Python is very powerful tool, more better than REXX included with iSeries.
Try it!
I got permission to install iSeries Python on a box about 3 years ago. I found that it worked pretty much as advertised. I contacted the developer and he was very good about answering questions. However, before I could think about using it in production, I had to approach the developer regarding a support contract. That really isn't his gig, so he said no and we scrapped the idea. The main limitation I found is that it is several releases behind Python on other platforms.
I have also had very good experience with Jython on the iSeries. Java is completely supported on the iSeries. Theoretically, everything you can do in RPG on the iSeries, you can do in Java, which means you can do it in Jython. I was sending email from an AS/400 (old name for iSeries) via JPython (old name for Jython) and smtplib.py in 1999 or 2000.
Another place to look is on the mailing list MIDRANGE-L or search the archives for the list at midrange.com. I know they have talked about this a while back.