Develop Reference

How can I inherit from classes I didn't make like requests in python

I am building an object-oriented app that utilizes the requests lib for sending and receiving data from an API, but I can't manage to find out a way to inherit from the requests module. not just for requests but for Tkinter also.
the general problem is that I can't really inherit from classes I didn't make personally like third party lib
All I want to do is just simply say "self.get" or "self.post" .
here is a code example of what I want to do: -
import requests
class Create(requests):
def __init__(self):
super().__init__()
def create(self):
self.post(url,headers,data)
I tried several methods even including modifying the original code of the lib to make classes I can inherit from but it didn't work either

There's nothing special about inheriting from 3rd-party classes, inheritance works just like with your own classes:
>>> import requests
>>> requests.Request
<class 'requests.models.Request'>
>>> class MyRequest(requests.Request):
... def say_hello(self):
... print("Hello!")
...
>>> MyRequest()
<Request [None]>
>>> MyRequest().say_hello()
Hello!
>>>

With requests, you want to derive from requests.Session. (The requests.post, etc. helpers just build an one-shot session and call the method by the same name.)
Like so:
import requests
class MySession(requests.Session):
base_url = "https://example.com"
def create(self, data):
return self.post(f"{self.base_url}/create", json=data)
sess = MySession()
sess.create("hello") # does a POST to https://example.com/create

How to define a global error handler in gRPC python

Im trying to catch any exception that is raised in any servicer so I can make sure that I only propagate known exceptions and not unexpected ones like ValueError, TypeError etc.
I'd like to be able to catch any raised error, and format them or convert them to other errors to better control the info that is exposed.
I don't want to have to enclose every servicer method with try/except.
I've tried with an interceptor, but im not able to catch the errors there.
Is there a way of specifying an error handler for the grpc Server? like what you do with flask or any other http server?

gRPC Python currently don't support server-side global error handler. The interceptor won't execute the server handler inside the intercept_service function, so there is no way to try/except.
Also, I found the gRPC Python server interceptor implementation is different from what they proposed original at L13-Python-Interceptors.md#server-interceptors. If the implementation stick to the original design, we can use interceptor as global error handler easily with handler and request/request_iterator.
# Current Implementation
intercept_service(self, continuation, handler_call_details)
# Original Design
intercept_unary_unary_handler(self, handler, method, request, servicer_context)
intercept_unary_stream_handler(self, handler, method, request, servicer_context)
intercept_stream_unary_handler(self, handler, method, request_iterator, servicer_context)
intercept_stream_stream_handler(self, handler, method, request_iterator, servicer_context)
Please submit a feature request issue to https://github.com/grpc/grpc/issues.

Maybe this will help you :)
def _wrap_rpc_behavior(handler, fn):
if handler is None:
return None
if handler.request_streaming and handler.response_streaming:
behavior_fn = handler.stream_stream
handler_factory = grpc.stream_stream_rpc_method_handler
elif handler.request_streaming and not handler.response_streaming:
behavior_fn = handler.stream_unary
handler_factory = grpc.stream_unary_rpc_method_handler
elif not handler.request_streaming and handler.response_streaming:
behavior_fn = handler.unary_stream
handler_factory = grpc.unary_stream_rpc_method_handler
else:
behavior_fn = handler.unary_unary
handler_factory = grpc.unary_unary_rpc_method_handler
return handler_factory(fn(behavior_fn,
handler.request_streaming,
handler.response_streaming),
request_deserializer=handler.request_deserializer,
response_serializer=handler.response_serializer)
class TracebackLoggerInterceptor(grpc.ServerInterceptor):
def intercept_service(self, continuation, handler_call_details):
def latency_wrapper(behavior, request_streaming, response_streaming):
def new_behavior(request_or_iterator, servicer_context):
try:
return behavior(request_or_iterator, servicer_context)
except Exception as err:
logger.exception(err, exc_info=True)
return new_behavior
return _wrap_rpc_behavior(continuation(handler_call_details), latency_wrapper)

As some of the previous comments suggested, I tried the meta-class approach which works quite well.
Attached is a simple example to demonstrate how to intercept the grpc calls.
You could extend this by providing the metaclass a list of decorators which you could apply on each function.
Also, it would be wise to be more selective regarding the methods you apply the wrapper to. A good option would be to list the methods of the autogenerated base class and only wrap those.
from types import FunctionType
from functools import wraps
def wrapper(method):
#wraps(method)
def wrapped(*args, **kwargs):
# do stuff here
return method(*args, **kwargs)
return wrapped
class ServicerMiddlewareClass(type):
def __new__(meta, classname, bases, class_dict):
new_class_dict = {}
for attribute_name, attribute in class_dict.items():
if isinstance(attribute, FunctionType):
# replace it with a wrapped version
attribute = wrapper(attribute)
new_class_dict[attribute_name] = attribute
return type.__new__(meta, classname, bases, new_class_dict)
# In order to use
class MyGrpcService(grpc.MyGrpcServicer, metaclass=ServicerMiddlewareClass):
...

modify a function of a class from another class

In pymodbus library in server.sync, SocketServer.BaseRequestHandler is used, and defines as follow:
class ModbusBaseRequestHandler(socketserver.BaseRequestHandler):
""" Implements the modbus server protocol
This uses the socketserver.BaseRequestHandler to implement
the client handler.
"""
running = False
framer = None
def setup(self):
""" Callback for when a client connects
"""
_logger.debug("Client Connected [%s:%s]" % self.client_address)
self.running = True
self.framer = self.server.framer(self.server.decoder, client=None)
self.server.threads.append(self)
def finish(self):
""" Callback for when a client disconnects
"""
_logger.debug("Client Disconnected [%s:%s]" % self.client_address)
self.server.threads.remove(self)
def execute(self, request):
""" The callback to call with the resulting message
:param request: The decoded request message
"""
try:
context = self.server.context[request.unit_id]
response = request.execute(context)
except NoSuchSlaveException as ex:
_logger.debug("requested slave does not exist: %s" % request.unit_id )
if self.server.ignore_missing_slaves:
return # the client will simply timeout waiting for a response
response = request.doException(merror.GatewayNoResponse)
except Exception as ex:
_logger.debug("Datastore unable to fulfill request: %s; %s", ex, traceback.format_exc() )
response = request.doException(merror.SlaveFailure)
response.transaction_id = request.transaction_id
response.unit_id = request.unit_id
self.send(response)
# ----------------------------------------------------------------------- #
# Base class implementations
# ----------------------------------------------------------------------- #
def handle(self):
""" Callback when we receive any data
"""
raise NotImplementedException("Method not implemented by derived class")
def send(self, message):
""" Send a request (string) to the network
:param message: The unencoded modbus response
"""
raise NotImplementedException("Method not implemented by derived class")
setup() is called when a client is connected to the server, and finish() is called when a client is disconnected. I want to manipulate these methods (setup() and finish()) in another class in another file which use the library (pymodbus) and add some code to setup and finish functions. I do not intend to modify the library, since it may cause strange behavior in specific situation.
---Edited ----
To clarify, I want setup function in ModbusBaseRequestHandler class to work as before and remain untouched, but add sth else to it, but this modification should be done in my code not in the library.

The simplest, and usually best, thing to do is to not manipulate the methods of ModbusBaseRequestHandler, but instead inherit from it and override those methods in your subclass, then just use the subclass wherever you would have used the base class:
class SoupedUpModbusBaseRequestHandler(ModbusBaseRequestHandler):
def setup(self):
# do different stuff
# call super().setup() if you want
# or call socketserver.BaseRequestHandler.setup() to skip over it
# or call neither
Notice that a class statement is just a normal statement, and can go anywhere any other statement can, even in the middle of a method. So, even if you need to dynamically create the subclass because you won't know what you want setup to do until runtime, that's not a problem.
If you actually need to monkeypatch the class, that isn't very hard—although it is easy to screw things up if you aren't careful.
def setup(self):
# do different stuff
ModbusBaseRequestHandler.setup = setup
If you want to be able to call the normal implementation, you have to stash it somewhere:
_setup = ModbusBaseRequestHandler.setup
def setup(self):
# do different stuff
# call _setup whenever you want
ModbusBaseRequestHandler.setup = setup
If you want to make sure you copy over the name, docstring, etc., you can use `wraps:
#functools.wraps(ModbusBaseRequestHandler.setup)
def setup(self):
# do different stuff
ModbusBaseRequestHandler.setup = setup
Again, you can do this anywhere in your code, even in the middle of a method.
If you need to monkeypatch one instance of ModbusBaseRequestHandler while leaving any other instances untouched, you can even do that. You just have to manually bind the method:
def setup(self):
# do different stuff
myModbusBaseRequestHandler.setup = setup.__get__(myModbusBaseRequestHandler)
If you want to call the original method, or wraps it, or do this in the middle of some other method, etc., it's otherwise basically the same as the last version.

It can be done by Interceptor
from functools import wraps
def iterceptor(func):
print('this is executed at function definition time (def my_func)')
#wraps(func)
def wrapper(*args, **kwargs):
print('this is executed before function call')
result = func(*args, **kwargs)
print('this is executed after function call')
return result
return wrapper
#iterceptor
def my_func(n):
print('this is my_func')
print('n =', n)
my_func(4)
more explanation can be found here

Given a method name as string, how to get the method instance?

I've a class Client which has many methods:
class Client:
def compute(self, arg):
#code
#more methods
All the methods of this class runs synchronously. I want to run them asynchronously. There are too many ways to accomplish this. But I'm thinking along these lines:
AsyncClient = make_async(Client) #make all methods of Client async, etc!
client = AsyncClient() #create an instance of AsyncClient
client.async_compute(arg) #compute asynchronously
client.compute(arg) #synchronous method should still exist!
Alright, that looks too ambitious, and I feel it can be done.
So far I've written this:
def make_async(cls):
class async_cls(cls): #derive from the given class
def __getattr__(self, attr):
for i in dir(cls):
if ("async_" + i) == attr:
#THE PROBLEM IS HERE
#how to get the method with name <i>?
return cls.__getattr__(i) # DOES NOT WORK
return async_cls
As you see the comment in the code above, the problem is to get the method given its name as string. How to do that? Once I get the method, I would wrap it in async_caller method, etc — the rest of the work I hope I can do myself.

The function __getattr__ just works with class instance, not class. Use getattr(cls, method_name) instead, it will solve the problem.
getattr(cls, method_name)

pythonic way to wrap xmlrpclib calls in similar multicalls

I'm writing a class that interfaces to a MoinMoin wiki via xmlrpc (simplified code follows):
class MoinMoin(object):
token = None
def __init__(self, url, username=None, password=None):
self.wiki = xmlrpclib.ServerProxy(url + '/?action=xmlrpc2')
if username and password:
self.token = self.wiki.getAuthToken(username, password)
# some sample methods:
def searchPages(self, regexp):
def getPage(self, page):
def putPage(self, page):
now each of my methods needs to call the relevant xmlrpc method alone if there isn't authentication involved, or to wrap it in a multicall if there's auth. Example:
def getPage(self, page):
if not self.token:
result = self.wiki.getPage(page)
else:
mc = xmlrpclib.MultiCall(self.wiki) # build an XML-RPC multicall
mc.applyAuthToken(self.token) # call 1
mc.getPage(page) # call 2
result = mc()[-1] # run both, keep result of the latter
return result
is there any nicer way to do it other than repeating that stuff for each and every method?
Since I have to call arbitrary methods, wrap them with stuff, then call the identically named method on another class, select relevant results and give them back, I suspect the solution would involve meta-classes or similar esoteric (for me) stuff. I should probably look at xmlrpclib sources and see how it's done, then maybe subclass their MultiCall to add my stuff...
But maybe I'm missing something easier. The best I've come out with is something like:
def _getMultiCall(self):
mc = xmlrpclib.MultiCall(self.wiki)
if self.token:
mc.applyAuthToken(self.token)
return mc
def fooMethod(self, x):
mc = self._getMultiCall()
mc.fooMethod(x)
return mc()[-1]
but it still repeats the same three lines of code for each and every method I need to implement, just changing the called method name. Any better?

Python function are objects so they can be passed quite easily to other function.
def HandleAuthAndReturnResult(self, method, arg):
mc = xmlrpclib.MultiCall(self.wiki)
if self.token:
mc.applyAuthToken(self.token)
method(mc, arg)
return mc()[-1]
def fooMethod(self, x):
HandleAuthAndReturnResult(xmlrpclib.MultiCall.fooMethod, x)
There may be other way but I think it should work. Of course, the arg part needs to be aligned with what is needed for the method but all your methods take one argument.
Edit: I didn't understand that MultiCall was a proxy object. Even if the real method call ultimately is the one in your ServerProxy, you should not pass this method object in case MultiCall ever overrides(define) it. In this case, you could use the getattribute method with the method name you want to call and then call the returned function object. Take care to handle the AttributeError exception.
Methods would now look like:
def HandleAuthAndReturnResult(self, methodName, arg):
mc = xmlrpclib.MultiCall(self.wiki)
if self.token:
mc.applyAuthToken(self.token)
try:
methodToCall = getattr(mc, methodName)
except AttributeError:
return None
methodToCall(arg)
return mc()[-1]
def fooMethod(self, x):
HandleAuthAndReturnResult('fooMethod', x)