Develop Reference

Is it safe to define a context manager around a nested function?

For a third party library* I have to provide a function which consumes some data. My implementation of the consumption requires the data to be posted to an API. So I came up with this structure below:
def consumer_provider():
with HttpClient() as http_client:
def consumer(data):
http_client.post(data)
return consumer
So I can present the function to the third party lib like so:
third_party_lib.add(consumer=consumer_provider())
In my tests it works quite well, but is this legit? When do I have to expect that the context manager releases the resource (in this case the connection pool)?
* loguru in this case, but it should not really matter for the question

It depends on the context manager. In the code you wrote, the HTTPClient you created stays alive because the function it returns maintains a reference to it, even though the variable http_client defined in consumer_provider goes out of scope.
However, HTTPClient.__exit__ is still called before consumer_provider returns, so the consumer function may not work as intended.
You may want to do something like
def consumer_provider():
http_client = HttpClient()
def consumer(data):
with http_client:
http_client.post(data)
return consumer
which ensures that the HttpClient object stays "hidden" inside the closure, but its __enter__ and __exit__ methods aren't called until the function gets called. (Whether the client can be used by multiple function calls also depends on the definition of HttpClient.)

Function closures and meaning of <locals> syntax in object name in Python

Suppose I have the following code:
def outer(information):
print(locals())
def inner():
print("The information given to me is: ", information)
return inner
func1 = outer("info1")
print(func1)
It returns:
{'information': 'info1'}
<function outer.<locals>.inner at 0x1004d9d30>
Of course, if I call func1, it will print with info1 in the statement. So, from printing the locals() in the outer function, I can see that there is some relationship between the local scope and the storage of the argument.
I was expecting func1 to simply be outer.inner, why does the syntax instead say outer.<locals>.inner? Is this a syntactical way of clarifying that there are different local scopes associated to each of these functions - imagine I made another one func2 = outer("info2") - I return using the outer function?
Also, is there something special about the enclosing <> syntax when used around a name? I see it around both the object and locals.

See PEP 3155 -- Qualified name for classes and functions and the example with nested functions.
For nested classes, methods, and nested functions, the __qualname__ attribute contains a dotted path leading to the object from the module top-level. A function's local namespace is represented in that dotted path by a component named <locals>.
Since the __repr__ of a function uses the __qualname__ attribute, you see this extra component in the output when printing a nested function.
I was expecting func1 to simply be outer.inner
That's not a fully qualified name. With this repr you might mistakenly assume you could import the name outer and dynamically access the attribute inner. Remember the qualname is a "dotted path leading to the object", but in this case attribute access is not possible because inner is a local variable.
Also, is there something special about the enclosing <> syntax when used around a name?
There is nothing special about it, but it gives a pretty strong hint to the programmer that you can't access this namespace directly, because the name is not a valid identifier.

You can think of outer.<locals>.inner as saying that inner is a local variable created by the function. inner is what is referred to a closure in computer science. Roughly speaking a closure is like a lambda in that it acts as a function, but it requires non-global data be bundled with it to operate. In memory it acts as a tuple between information and a reference to the function being called.
foo = outer("foo")
bar = outer("bar")
# In memory these more or less looks like the following:
("foo", outer.inner)
("bar", outer.inner)
# And since it was created from a local namespace and can not be accessed
# from a static context local variables bundled with the function, it
# represents that by adding <local> when printed.
# While something like this looks a whole lot more convenient, it gets way
# more annoying to work with when the local variables used are the length of
# your entire terminal screen.
<function outer."foo".inner at 0x1004d9d30>
There is nothing inherently special about the <> other than informing you that <local> has some special meaning.
Edit:
I was not completely sure when writing my answer, but after seeing #wim's answer <local> not only applies to closures created consuming variables within a local context. It can be applied more broadly to all functions (or anything else) created within a local namespace. So in summary foo.<local>.bar just means that "bar was created within the local namespace of foo".

How do variables from an outer scope get evaluated in nested functions?

Code first:
def another_func(func):
func()
def outer_func(pa=1, pb=2):
def inner_func():
print(pa)
print(type(inner_func))
another_func(inner_func)
if __name__ == '__main__':
outer_func()
#print "1"
I am not sure that, "inner_func" calls a parameter of "outer_func", but that is in the body of "outer_func". How can it "know" there is a "pa" when called by another_func?
I mean, what is actually passed to another_func when its called in "outer_func"? It seems there is something more than a reference of the function object.

Function objects in Python aren't just functions, they're closures:1 they carry around a reference to the local environment where the def statement was executed.
In particular, local variables from inside outer_func can be accessed from inside inner_func. (Even if you return inner_func, those values are kept alive so the closure will still work, for as long as inner_func is alive.)
If you add a nonlocal statement inside inner_func, it can even reassign the local variables from the body of outer_func.
How does this work?
Well, a def statement2 is just a statement, like any other. What it does is something like this:
inner_func = _make_closure(<code from compiling inner_func body>, locals())
That <code from compiling inner_func body> is actually a constant value—the compiler compiles the body of every function in your module into constant code objects at import time.
But the function object that comes back from that _make_closure is a new thing that's created on the fly, and it has a reference to the local variables baked into it. Every time you run outer_func, it creates a new inner_func closure from the same <code>, each one capturing the current local environment.
The details are a little more complicated—and, to some extent, they differ between implementations, so this will be CPython-specific.
Part of the compiler's job is figuring out what kind of variable each name in a function is. You may have read the rules on globals vs. locals (a variable is local if and only if you have an assignment for the name somewhere in the function body, and there's no global statement). But closures make things more complicated.
If a variable would have been local, but a nested function references the variable without assigning to it, or has a nonlocal statement, then it's a cell variable in the outer function, and a free variable in the inner function.3
When the interpreter calls a function, it creates a frame object that holds the local namespace—the references to all of the function's local variables.
But cell variables are special: the interpreter creates a special cell object for each one, and a reference to that cell goes into the namespace, so there's an extra dereference in front of the value every time you access or change it.
And what that _make_closure pseudo-code above does is to copy the cells from the outer function's frame to a special attribute on the nested function called __closure__.
Then, when you call the inner function, the interpreter copies those cells from the __closure__ into the frame for that function.
So, the outer function's frame and the inner function's frame both have references to the same cells, which is how they can share variables.
From more on this, see the inspect module's docs, which show you how to find things like __closure__ and co_freevars in your interactive interpreter, and the dis module which lets you look at the actual bytecode that your functions get compiled to.
1. This is one of those words that has a slew of related but different meanings. "Closure" can mean the technique of capturing the local namespace in a function, or it can mean the captured namespace, or it can mean the function with a captured namespace attached to it, or it can mean one of the variables in the captured namespace. Usually it's obvious which one you mean from context. If not, you have to say something like "closure capture" or "closure function" or "closure variable".
2. In case you're wondering, lambda expressions work exactly the same way as def statements. And class definitions are not identical, but similar.
3. It's actually still more complicated if you have multiple layers of nesting, but let's ignore that.

You seem to be confusing the code if the function with the function object. The code object is evaluated only once, when the source file is read. However, a new function object called inner_func is created every time outer_func is called. This happens because a def statement is a type of assignment: it associates a function object with the specified name.
The function object contains a reference to its code as a matter of course, along with references to all the namespaces it will need to operate, including it's parent's nonlocal namespace and the global module namespace.
So the value of pa in inner_func is going to be whatever it is in outer_func at the time of calling. The reference is to the namespace, not the name itself. If outer_func returns (think decorators), the namespace will be fixed, and only accessible through inner_func's special reference to it.

Why would you execute code in a local scope just to update the global scope?

In pkg_resources module you have this weird function:
#_call_aside
def _initialize_master_working_set():
# A bunch of ugly code, and then finally:
globals().update(locals())
The _call_aside is a decorator defined like this, it calls the function once at definition time (why to do this with a decorator rather than simply calling the function explicitly, I can't tell):
def _call_aside(f, *args, **kwargs):
f(*args, **kwargs)
return f
The function _initialize_master_working_set is not called anywhere else and the underscore on the name suggests it's not intended for public re-use. The docstring further warns against calling that:
This function ... is intended to be invoked once at the initialization of this module.
Invocation by other packages is unsupported
I don't get it. Isn't this just an obfuscated way of executing "A bunch of ugly code" at the module scope? How is this pattern any different than executing code directly at the global scope?

Git blame turns up a commit message with a link to an issue that motivated this function's introduction. Someone wanted a way to rerun this initialization, so they extracted it into a function that can be called repeatedly.
This function was not intended to behave any differently from the old version, where its code was at module level. While there are cases where a function with globals().update(locals()) at the end would behave differently from running the code in the global scope directly (for example, if something else rebinds the same global names in the middle of the function, the function will stomp over those changes at the end), that was not the motivation for introducing this function. They just wanted to be able to rerun it on demand.

replace functions with a different function in python

I have a function called get_account(param1,param2)
in run time I need to replace this function with the function mock_get_account(param1,param2)
so when the system calls get_account(param1,param2) I need the mock_get_account(param1,param2) to be called instead.
I tried this code:
package.get_account=self.mock_get_account
package.get_account(x,y)
but still the get_account runs instead of the mock_get_account
I'm new to python and I don't know if this is even possible but I have seen the lamda function and I know that function programming is possible in python. Thanks
Edit:
if i do the following:
package.get_account=self.mock_get_account
package.get_account(x,y)
then every thing is ok, meaning the mock_get_account is called, but in mu code I the following code i do a post self.client.post(url, data=data, follow=True) that triggers the package.get_account and this is not working:
package.get_account=self.mock_get_account
package.get_account(x,y)
#the folowing call will trigger the package.get_account(x,y) function in a django url #callback
self.client.post(url, data=data, follow=True)
meaning it calls the old function, also get_account(param1,param2) is defined in side a file, and is not a child function of a class and mock_get_account(self,param1,param2) is defined in a class Test and is called inside the Test.test_account - function

This is very opinionated and does not (directly) answer your question, but hopefully solves your problem.
A better practice is to use a subclass with your mock_get_account's implementation override the parent get_account method, example below:
class A(object):
def get_account(self):
return 1
def post(self):
return self.get_account()
class B(A):
def get_account(self):
return 2 # your original mock_get_account implementation
a = A()
print(a.get_account())
b = B()
print(b.post()) # this .post will trigger the overridden implementation of get_account

My guess is that the code implementing self.client.post has access to get_account through an import statement that looks like from package import get_account.
from package import get_account will first load package if it hasn't been already imported. Then it will look for a name get_account in that module, and whatever object that was bound to will be bound in the importing package's namespace, also under the name get_account. Thereafter the two names refer to the same object, but they are not the same name.
So if your mocking code comes along after this point, it sets the name get_account in package to instead refer to mock_get_account. But that'll only affect code that reads get_account from package again; anything that's already imported that name specially won't be affected.
If the code behind self.client.post instead had access only to package through import package, and was calling package.get_account it would work, because it's then only the object representing the package module that has been bound in the importing module's namespace. package.get_account would be reading an attribute of that object, and so would get whatever the current value is. If the from package import get_account appeared at function local scope rather than module scope, then this would behave similarly.
If I'm correct and your code is structured this way, then it's unfortunately not really package.get_account you need to rebind to a mock, but the get_account name in the module where self.client.post comes from (as well as any other modules which may call it).