Which Distributed Hash Table (DHT) is easiest to implement in Python? Any good example that is not bloated?
I not am looking for a definition of DHT because I am more oriented and focused on design and implementation of such.
In my job I'm working with entagled. I can't say it's great code, but it seems to be the only Kademlia implementation for Python around.
I think Kademlia has become the default DHT for most applications today, because it's quite simple and has fast lookups. At least in the academic world that I've seen so far.
If you are focused on implementation, rather than looking for an out-of-the-box solution, this article might help a bit: http://www.linuxjournal.com/article/6797
You might want to check out DHTBot. It's a python implementation of the BitTorrent MDHT and is written using twisted, a high-level python networking library.
(**Disclaimer: I am the author of DHTBot)
You may also check btdht that doesn't use twisted and is useful to sniff Bittorrent DHT.
Related
I am looking high and low for anything related to KNX implementation in Python, especially KNXnet/IP. So far I couldn't find anything. I know there are few projects written in Java but I thought I'd give it a shot asking here before I start porting code from other languages.
Also, does anyone know of any free documentation of the KNX standard, its data structures and KNXnet/IP? The official documentation from the KNX association is ridiculously expensive, especially for something supposed to be an "open" standard.
thanks
The KNX association have released the Falcon Runtime Developer kit but I think you can only get it if you are a KNX member. it doesnt supports Python but hey, its something i guess.
in regards to the documentation I could probably help.
is there a PM function on this site or something to give me your emails address?
There is the open-source BCUSDK software stack for KNX which has a C-based system daemon (eibd) with clients (Unix and TCP-socket based) for multiple languages (Python is one). I've written the Ruby and Lua client libraries as well, and I'm using it for day-to-day hacking with KNX.
Also, there is OpenRemote which has lots of information on KNXnet/IP and some Java classes that could get you started, if you wanted to, from scratch.
You might want to have a look a this (if by that time you are still looking for such a library) : https://github.com/leadrien/knxnet
This allows me to command KNX devices through an IP/KNX interface pretty well.
Alexandre
Meanwhile there is an open source Python 3 implementation of KNX IP https://github.com/XKNX/xknx
The documentation of the protocol can be downloaded after (free) registration from https://my.knx.org
I was wondering what good networking libraries/frameworks there are for Python.
Please provide a link to the standard API documentation for the library, and perhaps a link to a decent tutorial to get started with it.
A comment or two about its advantages/disadvantages would be nice as well.
The standard library has asyncore which is good for very simple stuff as well as the SocketServer stuff if you'd prefer something that does threads. There's also Twisted but the barrier of entry to that is a bit high if you're not used to event-driven IO. If you're after web frameworks, CherryPy is a good start or there's Django and TurboGears if you're looking for something more full-featured.
Consider the Twisted framework. The advantage:
solid reactor implementation
support for almost all network protocols found in the wild
well documented
Disadvantages:
it's huge
the asynchronous APIs need some time to get used to (but once you are familiar, things are actually pretty usable)
CPython itself ships with a tiny reactor/socket package. Never used it myself, though.
Twisted is the most complete, and complex, of all Python networking frameworks.
It's well-established and very complete, but it has a steep learning curve.
Documentation here; FAQ here.
In case you want to build/manipulate your own packets there is Scapy too :)
The usage is pretty straight forward, it lets you do whatever you want with the packets
and it's multi-Platform.
Project Page: http://www.secdev.org/projects/scapy/
Docs: http://www.secdev.org/projects/scapy/doc/
Example: http://www.secdev.org/projects/scapy/demo.html
I am considering programming the network related features of my application in Python instead of the C/C++ API. The intended use of networking is to pass text messages between two instances of my application, similar to a game passing player positions as often as possible over the network.
Although the python socket modules seems sufficient and mature, I want to check if there are limitations of the python module which can be a problem at a later stage of the development.
What do you think of the python socket module :
Is it reliable and fast enough for production quality software ?
Are there any known limitations which can be a problem if my app. needs more complex networking other than regular client-server messaging ?
Thanks in advance,
Paul
Check out Twisted, a Python engine for Networking. Has built-in support for TCP, UDP, SSL/TLS, multicast, Unix sockets, a large number of protocols (including HTTP, NNTP, IMAP, SSH, IRC, FTP, and others)
Python is a mature language that can do almost anything that you can do in C/C++ (even direct memory access if you really want to hurt yourself).
You'll find that you can write beautiful code in it in a very short time, that this code is readable from the start and that it will stay readable (you will still know what it does even after returning one year later).
The drawback of Python is that your code will be somewhat slow. "Somewhat" as in "might be too slow for certain cases". So the usual approach is to write as much as possible in Python because it will make your app maintainable. Eventually, you might run into speed issues. That would be the time to consider to rewrite a part of your app in C.
The main advantages of this approach are:
You already have a running application. Translating the code from Python to C is much more simple than write it from scratch.
You already have a running application. After the translation of a small part of Python to C, you just have to test that small part and you can use the rest of the app (that didn't change) to do it.
You don't pay a price upfront. If Python is fast enough for you, you'll never have to do the optional optimization.
Python is much, much more powerful than C. Every line of Python can do the same as 100 or even 1000 lines of C.
To answer #1, I know that among other things, EVE Online (the MMO) uses a variant of Python for their server code.
The python that EVE online uses is StacklessPython (http://www.stackless.com/), and as far as i understand they use it for how it implements threading through using tasklets and whatnot. But since python itself can handle stuff like MMO with 40k people online i think it can do anything.
This bad answer and not really an answer to your question, rather addition to previous answer.
Alan.
I want to start writing a http proxy that will modify responses according to some rules/filters I will configure. However, before I start coding it, I want to make sure I'm making the right choice in going with Python. Later, this tool would have to be able to process a lot of requests, so, I would like to know I can count on it later on to be able to perform when "push comes to shove".
As long as the bulk of the processing uses Python's built-in modules it should be fine as far as performance. The biggest strength of Python is its clear syntax and ease of testing/maintainability. If you find that one section of your code is slowing down the process, you can rewrite that section and use it as a C module, while keeping the bulk of your control code in Python.
However if you're looking to make the most optimized Python Code you may want to check out this SO post.
Yes, I think you will find Python to be perfectly adequate for your needs. There's a huge number of web frameworks, WSGI libraries, etc. to choose from, or learn from when building your own.
There's an interesting post on the Python History blog about how Python was supporting high performance websites in 1996.
This will depend on the library you use more than the language itself. The twisted framework is known to scale well.
Here's a proxy server example in python/twisted to get you started.
Bottomline: choose your third party tools wisely and I'm sure you'll be fine.
Python performs pretty well for most tasks, but you'll need to change the way you program if you're used to other languages. See Python is not Java for more info.
If plain old CPython doesn't give the performance you need, you have other options as well.
As has been mentioned, you can extend it in C (using a tool like swig or Pyrex). I also hear good things about PyPy as well, but bear in mind that it uses a restricted subset of Python. Lastly, a lot of people use psyco to speed up performance.
The ruby folks have Ferret. Someone know of any similar initiative for Python? We're using PyLucene at current, but I'd like to investigate moving to pure Python searching.
Whoosh is a new project which is similar to lucene, but is pure python.
The only one pure-python (not involving even C extension) search solution I know of is Nucular. It's slow (much slower than PyLucene) and unstable yet.
We moved from PyLucene-based home baked search and indexing to Solr but YMMV.
I recently found pyndexter. It provides abstract interface to various different backend full-text search engines/indexers. And it ships with a default pure-python implementation.
These things can be disastrously slow though in Python.
For some applications pure Python is overrated. Take a look at Xapian.
lupy was a lucene port to pure python.The lupy people suggest that you use PyLucene. Sorry. Maybe you can use the Java sources in combination with Jython.
+1 to the Xapian and Pyndexter answers.
Ferret is actually written in C with Ruby bindings on top. A pure Ruby search engine would be even slower than a pure Python one. I would love to see "someone else" write a Cython/Pyrex layer for Python interface to Ferret, but won't do it myself because why bother when there are Python bindings for Xapian.
For non-pure Python, Sphinx Search with Python API works the fastest. From the benchmarks from multiple blogs, Sphinx Search is way faster than Lucene, uses way less memory and it is in C.
I am developing a multi-document search engine based on it, using python and web2py as framework.
After weeks of searching for this, I found a nice Python solution: repoze.catalog. It's not strictly Python-only because it uses ZODB for storage, but it seems a better dependency to me than something like SOLR.