I am writing python code on top of the openstack shade library.
Connecting to a stack is pretty straight forward:
return shade.openstack_cloud(cloud='mycloud', **auth_data)
Now I am simply wondering: is there a canonical way to disconnect when I am done?
Or is the assumption that my script ending will do a "graceful" shutdown of that connection; not leaving anything behind?
OpenStack works on a RESTful api model. This means the connections are stateless, i.e. it makes a HTTP connection when you do your request, and closes that connection when the request is finished.
The above code simply initialises things by reading your config, authentication data, etc. A connection is not made until you do something with that object, e.g. create an image:
image = cloud.create_image( 'ubuntu-trusty',
filename='ubuntu-trusty.qcow2', wait=True)
In summary, no, you don't need to disconnect, shade's underlying code will take care of closing connections.
Related
I am using kubernetes-client library in python and looking at the various examples, it appears we don't need to explicitly close the client connection to the API server. Does the client connection gets terminated automatically or are the examples missing the call to close the connection? I also found the docs page for the APIs (AppsV1 for example) and the examples shown there use context manager for the calls so the connection gets disconnected automatically there but I still have questions for the scripts that don't use the context manager approach.
Kubernetes's API is HTTP-based, so you can often get away without explicitly closing a connection. If you have a short script, things should get cleaned up automatically at the end of the script and it's okay to not explicitly close things.
The specific documentation page you link to shows a safe way to do it:
with kubernetes.client.ApiClient(configuration) as api_client:
api_instance = kubernetes.client.AppsV1Api(api_client)
api_instance.create_namespaced_controller_revision(...)
The per-API-version client object is stateless if you pass in an ApiClient to its constructor, so it's safe to create these objects as needed.
The ApiClient class includes an explicit close method, so you could also do this (less safely) without the context-manager syntax:
api_client = kubernetes.client.ApiClient(configuration)
apps_client = kubernetes.client.AppsV1Api(api_client)
...
api_client.close()
The library client front-page README suggests a path that doesn't explicitly create an ApiClient. Looking at one of the generated models' code, if you don't pass an ApiClient option explicitly, a new one will be created for each API-version client object; that includes a connection pool as well. That can leak local memory and cause extra connections to the cluster, but this might not matter to you for small scripts.
I am using SolrClient for python with Solr 6.6.2. It works as expected but I cannot find anything in the documentation for closing the connection after opening it.
def getdocbyid(docidlist):
for id in docidlist:
solr = SolrClient('http://localhost:8983/solr', auth=("solradmin", "Admin098"))
doc = solr.get('Collection_Test',doc_id=id)
print(doc)
I do not know if the client closes it automatically or not. If it doesn't, wouldn't it be a problem if several connections are left open? I just want to know if it there is any way to close the connection. Here is the link to the documentation:
https://solrclient.readthedocs.io/en/latest/
The connections are not kept around indefinitely. The standard timeout for any persistent http connection in Jetty is five seconds as far as I remember, so you do not have to worry about the number of connections being kept alive exploding.
The Jetty server will also just drop the connection if required, as it's not required to keep it around as a guarantee for the client. solrclient uses a requests session internally, so it should do pipelining for subsequent queries. If you run into issues with this you can keep a set of clients available as a pool in your application instead, then request an available client instead of creating a new one each time.
I'm however pretty sure you won't run into any issues with the default settings.
I use python socket to send data to server, and the code like:
When I close the server, it will send the data twice, and then, it will go to the "except" code. If I set the SEND_INTERVAL too long, it will be a disaster. So, how to get the error immediately when the server is closed or downtime?
Nothing happens immediatly over the network. That's one thing.
Secondly the underlying OS will detect broken connections (and Python gets that info in the form of an exception), but usually this takes time. And that's why you still send messages even though the connection is already dead. But since OS operates on network layer (as opposed to the application layer) then there's an issue: the connection may be dead but OS may never detect this. For example this will happen when the server is dead but behind alive proxy.
Thirdly the most reliable way to know that a server is alive is when it sends something back to the client. So you should always .recv() (with timeout) after .sendall() call and the server should always .sendall() after .recv() (the request-response pattern, even when the response is a simple "I received message"). If you can't modify the server side and (in worst case) if the server doesn't send anything back to the client then there's no reliable way to know this.
That's why you need some form of framing/correctness protocol. Simple .sendall() won't do.
I have a Flask app that accepts HTTP requests. When certain HTTP requests come in, I want to trigger a message on a zeromq stream. I'd like to keep the zeromq stream open all the time. I'm wondering what the appropriate way to do this is. Since it is recommended to use gunicorn with Flask in production, doesn't that mean that there will be multiple instances of the Flask app, and if I put the zeromq connection in the same place as the Flask app, only one of those will be able to connect, and the others will fail.
I use a threading.local() object to store the zeromq context and socket objects.
That way I can re-use the already connected sockets inside a thread, while ensuring each thread will have its own socket objects.
Is the ZMQ socket in your app connect()-ing, or is it bind()-ing? If your app is considered the client and it's connecting, then multiple instances should be able to connect without issue. If it's considered the server and it's binding, then yes, you'll have problems... but in your case, it seems like you should consider your Flask app to be more transient, and thus the client, and the other end to be more reliable, and thus the server.
But it's hard to really give any concrete advice without code, there's only so much I can intuit from the little information you've given.
ZeroMQ shall not reuse context across different threads. The same applies to sockets.
If you manage to keep the socket used exclusively by one thread in worker, you might reuse the
socket.
Anyway, I would start with creating new context and socket with every request and see, if there is
any need to go into complexities of sharing a ZeroMQ connection. Set up ZeroMQ is often rather fast.
so I'm implementing a log server with twisted (python-loggingserver) and I added simple authentication to the server. If the authentication fails, I wanna close the connection to the client. The class in the log server code already has a function called handle_quit(). Is that the right way to close the connection? Here's a code snippet:
if password != log_password:
self._logger.warning("Authentication failed. Connection closed.")
self.handle_quit()
If the handle_quit message you're referring to is this one, then that should work fine. The only thing the method does is self.transport.loseConnection(), which closes the connection. You could also just do self.transport.loseConnection() yourself, which will accomplish the same thing (since it is, of course, the same thing). I would select between these two options by thinking about whether failed authentication should just close the connection or if it should always be treated the same way a quit command is treated. In the current code this makes no difference, but you might imagine the quit command having extra processing at some future point (cleaning up some resources or something).