Develop Reference

is PyMySQL well supported? and is it slower than than MySQLdb?

I'm building a python3 app which requires high-speed connections. Being a pure python library, will PyMySQL be significantly slower than the C based MySQLdb in connecting and executive queries?
Is PyMySQL well supported for future versions of python?
If not, can anyone suggest a reliable alternative?

PyMySQL is implemented in pure Python while MySQLdb is C extension. PyMySQL is easy to install (and some times the only way) in some system while MySQLdb sometimes give problems.
But both are just MySQL python connectors.
Why PyMySQL? MySQL is an immensely popular RDBMS, and you usually need
to talk to it when writing a web application in Python. The defacto
standard, MySQLdb, is a C extension module that has a reputation of
being difficult to compile, especially if you're on a Mac (like I am).
Additionally, end-users need to wait for new binaries to be compiled
for each new release of Python, and MySQLdb will never run on Jython,
IronPython, or PyPy (without something like cpyext or IronClad). We
also maintain 100% compatibility between Python 2 and Python 3, so all
advancements made on the 2.x trunk will be immediately available on
Python 3.
We are developing a drop-in replacement for MySQLdb that "just works"
without the hassle of compiling and installing C extensions and
without worrying what platform you're on.
Source https://code.google.com/p/pymysql/wiki/Goals

Many projects use PyMySQL because it supports Python 3 (3.3 and up) as well as Python 2. At the time of writing it has over 650 forks on Github and over 60 contributors. There are regular updates and patches to it every couple of months or so and it's popular in the Python community for it's relatively easy installation and Python 3 support. So all that said, yes, I would say it's well supported.
Now for speed: since it's written pure Python whereas MySQLdb is C, you can expect PyMySQL to be slower. For an in depth analysis I suggest a look at the openstack link at the bottom of this post. Based on their testing, PyMySQL was roughly 10x slower than MySQLdb. However, with DB calls you are mostly IO bound anyways so the advantage of PyMySQL supporting gevent or something similar for asynchronous IO may make up for the speed depending on your application.
Sources:
https://github.com/PyMySQL/PyMySQL
https://wiki.openstack.org/wiki/PyMySQL_evaluation

Sqlite3 on Python 2.5 is too slow

When I access an Sqlite database using Python 2.5, it takes too long, but if I access the same database by other ways (including Python 3.2) it takes much shorter. What's going on? (I need Python 2.5 and can't switch to Python 3.2)

It should go without saying that if you use an old version of a program, you don't get the performance improvements, bug fixes, and feature additions that were added later. Python 2.5 is slower, buggier, and less powerful than 2.7 (or 3.2) in many ways, and for the most part, the only solution is "Stop using 2.5", unless you want to track down the specific improvement in the changelogs and backport it to the 2.5 codebase.
But in this case, it's a lot easier, because sqlite3 is developed independently of Python (in fact, before 2.5, it didn't even come built in), as pysqlite. Here's some version history (as seen in the What's New documentation for 2.6 and 2.7):
Python 2.5: pysqlite 2.3.2
Python 2.6: pysqlite 2.4.1
Python 2.7: pysqlite 2.6.0
The latest version is 2.6.3. You can install it by using any of the usual means (pip, easy_install, downloading it from the website and following the instructions, downloading the Windows binary installers from the website and running them, etc.).
In fact, if you're building a package with setuptools/distribute, you may just be able to put in a requirement for >= 2.6.0, and pip install mypackage will automatically get the new version for 2.5 (and 2.6 users).
It's also possible that your problem is with the underlying C sqlite3 library, not the pysqlite wrapper. If you install sqlite3 and then build pysqlite from source, you can solve that too.
If you're not sure which version you have, you can check at runtime, because the module has a human-readable version attribute (and a version_info tuple just like the one in sys, so you can check version_info >= (2, 6)), and likewise sqlite_version and sqlite_version_info for the underlying C library.

In addition to abarnert's excellent answer, look at how often you are committing. Comitting is slow in SQLite and generally performance benefits if it is put off. I encountered this myself during some other testing I did and the performance difference of reducing the number of commits is enormous.

How can I upgrade the sqlite3 package in Python 2.6?

I was using Python 2.6.5 to build my application, which came with sqlite3 3.5.9. Apparently though, as I found out in another question of mine, foreign key support wasn't introduced in sqlite3 until version 3.6.19. However, Python 2.7 comes with sqlite3 3.6.21, so this work -- I decided I wanted to use foreign keys in my application, so I tried upgrading to python 2.7.
I'm using twisted, and I couldn't for the life of me get it to build. Twisted relies on zope.interface and I can't find zope.interface for python 2.7 -- I thought it might just "work" anyway, but I'd have to just copy all the files over myself, and get everything working myself, rather than just using the self-installing packages.
So I thought it might be wiser to just re-build python 2.6 and link it against a new version of sqlite3. But I don't know how--
How would I do this?
I have Visual Studio 2008 installed as a compiler, I read that that is the only one that is really supported for Windows, and I am running a 64 bit operating system

download the latest version of sqlite3.dll from sqlite website and replace the the sqlite3.dll in the python dir.

sqlite3 is not a built-in module; it's an extension module (the binary is C:\Python26\DLLs_sqlite3.pyd (on my machine)). A pyd is a DLL with a different filename extension and only 1 entry point. There's also a sqlite3.dll, which contains the SQLite code. python.exe is not linked against either of those, and thus rebuilding python.exe has no point.
The next idea is to go to the pysqlite2 download site, and get the latest Windows installer for Python 2.6. Unfortunately there's no docs about which version of SQLite it contains; one needs to install it and then muck about:
>>> import sqlite3 as standard
>>> from pysqlite2 import dbapi2 as latest
>>> for m in (standard, latest):
... print m.sqlite_version
...
3.5.9
3.6.2
>>>
So, it contains only SQLite version 3.6.2, which doesn't have the real foreign key support that you want.
I suggest that you check the mailing list to see if your question is answered there, and if not ask about the possibility of a Python 2.6 installer containing a later SQLite (e.g. the one included with Python 2.7).

I decided I'd just give this a shot when I realized that every library I've ever installed in python 2.6 resided in my site-packages folder. I just... copied site-packages to my 2.7 installation, and it works so far. This is by far the easiest route for me if this works -- I'll look further into it but at least I can continue to develop now.
I won't accept this answer, because it doesn't even answer my question, but it does solve my problem, as far as I can tell so far.

Which version of python is currently best for os x?

After going through hell trying to install the latest version of postgresql and psycopg2 today I'm going for a complete reinstall of Leopard.
I've been sticking with macpython 2.5 for the past year but now I'm considering macports even 2.6
For me it's most important for Twisted, PIL and psycopg2 to be working without a problem.
Can anyone give some guidelines for what version I should choose, based on experience?
Edit:
Ok I've decided to go without reinstalling the os. Hacked around to clean up the bad PostgresPlus installation and installed another one. The official python 2.6.1 package works great, no problem installing it alongside 2.5.2. Psycopg2 works. But as expected PIL wont compile.
I guess I'll be switching between the 2.5 from macports and the official 2.6 for different tasks, since I know the macports python has it's issues with some packages.
Another Edit:
I've now compiled PIL. Had to hide the whole macports directory and half the xcode libraries, so it would find the right ones. It wouldn't accept the paths I was feeding it. PIL is notorious for this on leopard.

You can install them side-by-side. If you've encounter problems just set python 2.5 as the standard python and use e.g. python26 for a newer version.

Read this
http://farmdev.com/thoughts/66/python-3-0-on-mac-os-x-alongside-2-6-2-5-etc-/

I still use macports python25, because so many other packages depend on it, and have not updated to use python26.
$ port dependents python25
gnome-doc-utils depends on python25
mod_python25 depends on python25
postgresql83 depends on python25
gtk-doc depends on python25
at-spi depends on python25
gnome-desktop depends on python25
mercurial depends on python25
And that's excluding the py25-* packages I have installed.

I wrote something today on this very subject, my recommendation? Run multiple version, and slap virtualenv down to compartmentalize things.
http://jessenoller.com/2009/03/16/so-you-want-to-use-python-on-the-mac/
I also wouldn't both with macports. I don't see a need for it.

I've updated my macbook running leopard to python 2.6 and haven't had any problems with psycopg2. For that matter, I haven't had any compatibility issues anywhere with 2.6, but obviously switching to python3k isn't exactly recommended if you're concerned about backwards compatibility.

I would stick with the MacPython version 2.5.x (I believe 2.5.4 currently). Here's my rationale:
Snow Leopard may still be on the 2.5 series, so you might as well be consistent with the future OS (i.e. no point in going too far ahead).
For most production apps, nobody is going to want to use 2.6 for another year.
No frameworks/programs are going to leave 2.5 behind for at least 2 years.
In other words, my approach is that the only reason to do 2.6 is for fun. If you're looking to have fun, just go for 3.0.

I use both Twisted and Psycopg2 extensively on OSX, and both work fine with Python 2.6. Neither has been ported to Python 3.0, as far as I know.
Several of Python 3.0's features have been back-ported to 2.6, so you gain quite a bit by moving from 2.5 to 2.6. But I wouldn't switch to 3.0 until all of your thirdparty libraries support it; and this may not happen for some time.

I had some trouble installing PIL. I compiled it and it worked with the modification explained on this post http://passingcuriosity.com/2009/installing-pil-on-mac-os-x-leopard/
After that it worked fine.

I am using Python 2.5.1. It's working great for me for general scripting and some CherryPy web projects.

If your using Macports, I recommend downloading the python_select package, which facilitates easy switching between different versions including the built in apple versions. Makes life a lot easier.

SQLite in Python 2.2.3

I've written a web-app in python using SQLite and it runs fine on my server at home (with apache and python 2.5.2). I'm now trying to upload it to my web host and there servers use python 2.2.3 without SQLite.
Anyone know of a way to use SQLite in python 2.2.3 e.g. a module that I can upload and import? I've tried butchering the module from newer versions of python, but they don't seem to be compatible.
Thanks,
Mike

There is no out-of-the-box solution; you either have to backport the SQLlite module from Python 2.5 to Python 2.2 or ask your web hoster to upgrade to the latest Python version.
Python 2.2 is really ancient! At least for security reasons, they should upgrade (no more security fixes for 2.2 since May 30, 2003!).
Note that you can install several versions of Python in parallel. Just make sure you use "/usr/bin/python25" instead of "/usr/bin/python" in your scripts. To make sure all the old stuff is still working, after installing Python 2.5, you just have to fix the two symbolic links "/usr/bin/python" and "/usr/lib/python" which should now point to 2.5. Bend them back to 2.2 and you're good.

Look here: http://oss.itsystementwicklung.de/download/pysqlite/
From the release notes (http://oss.itsystementwicklung.de/trac/pysqlite/browser/doc/install-source.txt)
Python:
Python 2.3 or later
You may not be able to do what you're trying to do.

If you have shell access to your web server, you can probably build you're own version of Python and SQLite. This will let you use the latest version. Download the source code, then when you configure it, do something like "./configure --prefix=$HOME/packages".
Next, fiddle around with your .profile, or .bashrc or whatever it is to make sure $HOME/packages/bin comes first in your path. This will cause your private Python to override the one installed by your web server.
This page might give you a little more information for how to do this on a server like Dreamhost: http://wiki.dreamhost.com/Python

In case anyone comes across this question, the reason why neither pysqlite nor APSW are available for Python 2.2 is because Python 2.3 added the simplified GIL API. Prior to Python 2.3 it required a lot of code to keep track of the GIL. (The GIL is the lock used by Python to ensure correct behaviour while multi-threading.)
Doing a backport to 2.2 would require ripping out all the threading code. Trying to make it also be thread safe under 2.2 would be a nightmare. There was a reason they introduced the simplified GIL API!
I am still astonished at just how popular older Python versions are. APSW for Python 2.3 is still regularly downloaded.