For a third party library* I have to provide a function which consumes some data. My implementation of the consumption requires the data to be posted to an API. So I came up with this structure below:
def consumer_provider():
with HttpClient() as http_client:
def consumer(data):
http_client.post(data)
return consumer
So I can present the function to the third party lib like so:
third_party_lib.add(consumer=consumer_provider())
In my tests it works quite well, but is this legit? When do I have to expect that the context manager releases the resource (in this case the connection pool)?
* loguru in this case, but it should not really matter for the question
It depends on the context manager. In the code you wrote, the HTTPClient you created stays alive because the function it returns maintains a reference to it, even though the variable http_client defined in consumer_provider goes out of scope.
However, HTTPClient.__exit__ is still called before consumer_provider returns, so the consumer function may not work as intended.
You may want to do something like
def consumer_provider():
http_client = HttpClient()
def consumer(data):
with http_client:
http_client.post(data)
return consumer
which ensures that the HttpClient object stays "hidden" inside the closure, but its __enter__ and __exit__ methods aren't called until the function gets called. (Whether the client can be used by multiple function calls also depends on the definition of HttpClient.)
Related
I have an architecture, where I use wrapper for calling functions from package module. Inside the module there is a function that calls another three. I need to override one of them in run-time. Exactly I need to change parameters that are forwarded to another set of functions being called.
Here is a case sample:
a.py
import b_wrapper as wrapper
def foo():
if wrapper.bar(parameter):
"""some more code goes here"""
b_wrapper.py
import some.package.module as module
def bar(parameter):
return module.baz(veryImportantParameter, parameter)
file.py
def functionThree(par): # needs to be overwritten
"""more functions called forwarding par as a parameter"""
def baz(veryImportantParameter, parameter)
functionOne(veryImportantParameter, otherParameters)
functionTwo(veryImportantParameter, someMoreParameters)
functionThree(veryImportantParameter, parameterToChange, evenMoreParameters)
What I tried to do is overriding in wrapper file, didn't work out, as other functions are interfering with it. As reference used this post.
I'm not quite sure that this is doable, because of unique functions that are called inside this module, also looking for alternatives that won't require overriding portion of module.
Edit: mixing up arguments and parameters is intentional for demonstration purpose only.
In pkg_resources module you have this weird function:
#_call_aside
def _initialize_master_working_set():
# A bunch of ugly code, and then finally:
globals().update(locals())
The _call_aside is a decorator defined like this, it calls the function once at definition time (why to do this with a decorator rather than simply calling the function explicitly, I can't tell):
def _call_aside(f, *args, **kwargs):
f(*args, **kwargs)
return f
The function _initialize_master_working_set is not called anywhere else and the underscore on the name suggests it's not intended for public re-use. The docstring further warns against calling that:
This function ... is intended to be invoked once at the initialization of this module.
Invocation by other packages is unsupported
I don't get it. Isn't this just an obfuscated way of executing "A bunch of ugly code" at the module scope? How is this pattern any different than executing code directly at the global scope?
Git blame turns up a commit message with a link to an issue that motivated this function's introduction. Someone wanted a way to rerun this initialization, so they extracted it into a function that can be called repeatedly.
This function was not intended to behave any differently from the old version, where its code was at module level. While there are cases where a function with globals().update(locals()) at the end would behave differently from running the code in the global scope directly (for example, if something else rebinds the same global names in the middle of the function, the function will stomp over those changes at the end), that was not the motivation for introducing this function. They just wanted to be able to rerun it on demand.
I am struggling to understand the scope of request and request.db in the following decorated python function (this function is part of the Pyramid "Todo List Application in One File" tutorial):
#subscriber(NewRequest)
def new_request_subscriber(event):
request = event.request
settings = request.registry.settings
request.db = sqlite3.connect(settings['db'])
I thought assignments in a function were limited in scope to that function unless declared as a global; so according to my flawed understanding, request and request.db would go out of scope when the function completes. But in this case I am clearly mistaken because request.db is accessed subsequently within other functions. Could somebody explain the genesis and scope of the magic object's request and request.db please?
request is really just a local alias to event.request. That function could be rewritten as follows:
def new_request_subscriber(event):
event.request.db = sqlite3.connect(event.request.registry.settings['db'])
So all we're doing is modifying the attributes of the event object that's passed in. Since Python passes the actual object, the modifications will be seen by whatever called the function.
I have a function called get_account(param1,param2)
in run time I need to replace this function with the function mock_get_account(param1,param2)
so when the system calls get_account(param1,param2) I need the mock_get_account(param1,param2) to be called instead.
I tried this code:
package.get_account=self.mock_get_account
package.get_account(x,y)
but still the get_account runs instead of the mock_get_account
I'm new to python and I don't know if this is even possible but I have seen the lamda function and I know that function programming is possible in python. Thanks
Edit:
if i do the following:
package.get_account=self.mock_get_account
package.get_account(x,y)
then every thing is ok, meaning the mock_get_account is called, but in mu code I the following code i do a post self.client.post(url, data=data, follow=True) that triggers the package.get_account and this is not working:
package.get_account=self.mock_get_account
package.get_account(x,y)
#the folowing call will trigger the package.get_account(x,y) function in a django url #callback
self.client.post(url, data=data, follow=True)
meaning it calls the old function, also get_account(param1,param2) is defined in side a file, and is not a child function of a class and mock_get_account(self,param1,param2) is defined in a class Test and is called inside the Test.test_account - function
This is very opinionated and does not (directly) answer your question, but hopefully solves your problem.
A better practice is to use a subclass with your mock_get_account's implementation override the parent get_account method, example below:
class A(object):
def get_account(self):
return 1
def post(self):
return self.get_account()
class B(A):
def get_account(self):
return 2 # your original mock_get_account implementation
a = A()
print(a.get_account())
b = B()
print(b.post()) # this .post will trigger the overridden implementation of get_account
My guess is that the code implementing self.client.post has access to get_account through an import statement that looks like from package import get_account.
from package import get_account will first load package if it hasn't been already imported. Then it will look for a name get_account in that module, and whatever object that was bound to will be bound in the importing package's namespace, also under the name get_account. Thereafter the two names refer to the same object, but they are not the same name.
So if your mocking code comes along after this point, it sets the name get_account in package to instead refer to mock_get_account. But that'll only affect code that reads get_account from package again; anything that's already imported that name specially won't be affected.
If the code behind self.client.post instead had access only to package through import package, and was calling package.get_account it would work, because it's then only the object representing the package module that has been bound in the importing module's namespace. package.get_account would be reading an attribute of that object, and so would get whatever the current value is. If the from package import get_account appeared at function local scope rather than module scope, then this would behave similarly.
If I'm correct and your code is structured this way, then it's unfortunately not really package.get_account you need to rebind to a mock, but the get_account name in the module where self.client.post comes from (as well as any other modules which may call it).
I would like to create a Pyramid app with an orm which I am writing (currently in deep alpha status). I want to plug the orm into the app sanely and thus I want to know how global objects are handled in multithreading.
In the file:
https://www.megiforge.pl/p/elephantoplasty/source/tree/0.0.1/src/eplasty/ctx.py
you can see, there is a global object called ctx which contains a default session. What if I run set_context() and start_session() in middleware at ingress? Can I expect then to have a separate session in ctx in every thread? Or is there a risk that two threads will use the same session?
Global variables are shared between all threads, so if you run those functions the threads will conflict with each other in unpredictable ways.
To do what you want you can use thread local data, using threading.local. You need to remove the global definition of ctx and then create the following function.
def get_ctx():
thread_data = threading.local()
if not hasattr(thread_data, "ctx"):
thread_data.ctx = Ctx()
return thread_data.ctx
Then, everywhere you reference ctx call get_ctx() instead. This will ensure that your context is not shared between threads.