So i been readying for a awhile now. And it seems like asynchronous socket handling would be a better approach to dealing with what I'm trying to do.
Right now I'm working on a gaming server. At the moment socket server will do ok with about 3 clients or so. Sending data at the same exact time.
But my problem is, after that things start to get laggy. So if i do a asynchronous server in the same manner to what i'm already doing. Would it make the game data transfer more smoothly?
This is in python by the way.
Asynchronous sockets are more effective then synchronous. But if the game is lagging for 4+ clients, then your server/client system is badly written and it is not the matter of sockets imho.
Related
I'm getting started with websockets. Trying to write a python server for a browser based (javascript) client.
I have also never really done asynchronous programming before (except "events"). I was trying to avoid it - I have searched and searched for an example of websocket use that did not involve importing tornado or asyncio. But I've found nothing, even the "most basic examples" do it.
So now I'm internalising it, but clear it up for me - is "full duplex" server code necessarily asynchronous?
Full-duplex servers are necessarily concurrent. In Tornado and asyncio, concurrency is based on the asynchronous programming model, so if you use a websocket library based on one of those packages, your code will need to be asynchronous.
But that's not the only way: full-duplex websockets could be implemented in a synchronous way by dedicating a thread to reading from the connection (in addition to whatever other threads you're using). I don't know if there are any python websocket implementations that support this kind of multithreading model for full-duplex, but that's how Go's websocket implementation works for example.
That said, the asynchronous/event-driven model is a natural fit for websockets (it's how everything works on the javascript side), so I would encourage you to get comfortable with that model instead of trying to find a way to work with websockets synchronously.
I have a program that sniffs network data and stores it in a database using pcapy (based on this). I need to make the data available in realtime over a network connection.
Right now when i run the program it will start a second thread for the sniffer and a Twisted server on the main thread, however i have no idea how to get clients to 'tap into' the sniffer that's running in the background.
The end result should be that a client enters an url and the connection will be kept open until the client disconnects (even when there's nothing to send), whenever the server has network activity the sniffer will sniff it and send it to the clients.
I'm a beginner with Python so i'm quite overwhelmed so if anyone could point me in the right direction it would be greatly appreciated.
Without more information (a simple code sample that doesn't work as you expect, perhaps) it's tough to give a thorough answer.
However, here are two pointers which may help you:
Twisted Pair, a (unfortunately very rudimentary and poorly documented) low-level/raw sockets networking library within Twisted itself, which may be able to implement the packet capture directly in a Twisted-friendly way, or
The recently-released Crochet, which will allow you to manage the background Twisted thread and its interactions with your pcapy-based capture code.
I'm making an action game in Python. It has to support at least 2 players playing together over the Internet. I've chosen UDP since it seems to be the natural choice for low-latency games. I'm going with a client-server model and I want one player to host the game. This means one player is both hosting the game (server) and running the game (client) at the same time.
Should I be using threads for this or is there another approach? The problem is that the Python documentation says to use serve_forever but I need a non-blocking solution, ideally something I can just call every game loop.
I assume, by your reference to "the Python documentation says to use serve_forever" you are planning to use SocketServer. The module implements synchronous (forking or threading) servers, and you seem to be looking for asynchronous solutions.
For async, non-blocking solutions you might take a look to twisted http://twistedmatrix.com/trac/ (specifically twisted.internet).
Or, it you need something very specific and don't mind writing an event loop, just use poll or select for sockets and other resources...
UDP is not the "natural choice". It's an optimization for a specific type of problem (packet loss) for a specific type of data (typically position/velocity data).
You should stick with TCP until you can demonstrate that your game has a problem with it.
To integrate non-blocking networking into a game in Python, take a look at the approach taken by Game, which invokes the PyGame mainloop from Twisted. This will work equally well with TCP, UDP, or any other protocol that uses sockets.
I'm trying to figure out how to make a server that can accept multiple clients at one time. While doing so, I need the client able to send and receive data from the server at the same time.
Would i have to make a threaded server? And have a thread for listening for data.
And then another thread for sending out information to the client?
Then for the client side, do i need use threads to send/get info?
Use async IO. There are dozen of async IO socket libs for python. Here is a brief benchmark.
I also tested gevent, eventlet, asyncore, twisted, pyev, pycurl, tornado.
Twsited
is stable but most slow and also not easy to start with.
gevent, eventlet (libevent)
easy to start and fast (code looks like blocking) but have some issues with forking.
pycurl (libcurl)
fast and easy (if you ok to do flags magic.. but there are example) but only http.
pyev (libev)
you must understand what you are doing almost like polling yourself.
tornado (polling in python)
fast enough and i think stable and also easy to start.
asyncore
really fast.. but don't use it.. it is ugly-ugly.
Don't use threads in python unless you are really know what you are doing.
Python and threads not really big friends (unless version <3.2 in 3.2 there must be a new gil).
On server-side you clearly need a Socket Server. This server creates a new thread for every incoming client connection.
Once a connection is established, both the client and the thread that was instantiated for the communication require an additional thread if they have to do other business in parallel than listening to the socket if the communication is synchronous. In case an asynchronous communication is what you need, then Python provides an excellent Asynchronous Socket Handler.
Use a asynchronous socket. Example server could be found here and the client code here. No direct hassle with threads. Depending on your needs, you probably don't need the asynchronous client.
You don't need threads for either client or server; you can instead select() to multiplex all the I/O inside a single thread.
I'm looking for some general information on how I should approach a problem that I think Twisted is a great fit for. (I'm new to Twisted but not Python)
I have a home automation controller that can support a single TCP socket connection, sending and receiving binary data. I'd like to use XMPP as a bridge to the socket so a user can send commands and receive events.
I got a rudimentary socket connection working with Twisted that was able to send and receive commands from one of the examples in the O'Reilly book. I also have a fully working Python XMPP bot written with the SleekXMPP library that I'm happy with. I'm just not sure how to bring these together.
The basic scenario is:
User sends message to XMPP bot, which figures out what command to send to the socket
ASCII Socket command is converted to binary and sent to socket
Socket receives command and sends binary response
Binary response converted to ASCII
XMPP bot sends response back to user.
Network events (independent from user action) can also be received by network socket and should be sent to user
It's #6 that is presenting the challenge, otherwise I'd just open/close the socket on demand when in need to write something.
The part that I'm having trouble wrapping my head around with Twisted is the best approach to make these two event loops communicate. I've seen lots of info on using Queues, deferred, threads, select, etc. I have a feeling that Twisted can handle much of the complexity if I just learn to use the tool properly.
If someone can point me in the right direction, I'll take the ball and run with it. As I mentioned, I'm happy with my XMPP bot and I'd like to use the existing code. I think my problem now comes down to creating the socket in the background, then sending and receiving data from that socket in the foreground.
By the way, I'm very happy to share back my code once it's working so someone else can benefit from the help I'm asking for.
-- Scott
One of the problems with a non-blocking IO engine is that its pretty much all-or-nothing. As soon as you introduce blocking code, you can quickly lose most of the benefits of the event-driven asynch approach. Wherever possible (as a rule of thumb), its best to have the entire app running off the same reactor.
As i see it, you have two options:
Twisted is not thread safe. That said, you can use mechanisms like deferToThread and callFromThread to interact with other threads. This is by far the most confusing and needlessly complex approach for your application design. It's particularly painful if you're new to twisted.
Use twisted.words.protocols.jabber, and implement your XMPP stuff in a non-blocking manner using the twisted reactor. That way it will happily exist alongside all your other twisted code. and allow you to cleanly interact between protocols. It will result in less code, and a robust implementation that is easy to extend, maintain, and test.