I inherited some large and unwieldy python code. In one file its using a list of commands imported from another file. Looking at it with pdb this commands variable ends up in the global namespace. However there's another file that doesn't look like its even being used that also has a commands variable in it and for some reason on certain machines that variable is used instead.
My question is, is there a way in pdb or just code to show the source of the commands variable? I'm hoping for some concrete evidence that shows it's pointing to that file for some reason.
It's a nice demonstration on the dangers of global variables I guess, and I can clean up the code but I'd like to fully understand it first.
To get the module of the commands object, you could try:
import inspect
inspect.getmodule(commands)
I have heard a lot about namespace as a feature for programming language. Thus, can not help thinking:
Does Python have so called namespace feature?
Quote from Python documentation:
A namespace is a mapping from names to objects. Most namespaces are currently implemented as Python dictionaries, but that’s normally not noticeable in any way (except for performance), and it may change in the future. Examples of namespaces are: the set of built-in names (containing functions such as abs(), and built-in exception names); the global names in a module; and the local names in a function invocation. In a sense the set of attributes of an object also form a namespace. The important thing to know about namespaces is that there is absolutely no relation between names in different namespaces; for instance, two different modules may both define a function maximize without confusion — users of the modules must prefix it with the module name.
For a good overview, see the "Python Scopes and Namespaces" section of the Python tutorial at https://docs.python.org/2/tutorial/classes.html#python-scopes-and-namespaces.
I wasn't looking what I had previously written on the line so I accidently declared a variable in ipython as:
np.zerosn=10
Surprisingly this was allowed. So I thought that maybe it was because you can name use periods in your variable names, but that is not the case. So I'm wondering what is actually happening. Is this adding a new variable to the numpy module?
Yes.
In general, (most/many) python objects have dynamic attribute spaces, and you can stick whatever you want onto them whenever you want. And modules are just objects. Their attribute space is essentially the same as their global scope.
Pure python functions are another (perhaps surprising) example of something onto which you can stick arbitrary attributes, though these are not associated with the function's local scope.
Most 'builtin' types (i.e. those which are implemented in extension modules, rather than those that are found in the __builtins__ module) and their instances, do not have dynamic attribute spaces. Neither do pure python types with __slots__.
I tried to look around but I couldn't find anything clear about this topic.
Are built-in functions implemented in a module that is automatically imported every time Python is launched? In the case which is the module?
Or are built-in functions just embedded functions inside the Python interpreter?
For CPython, the built-in functions are (for the most part) implemented in the bltinmodule.c file.
The exceptions are mostly the types; things like str and dict and list have their own C files in the Objects directory of the C source; these are listed as a table in the bltinmodule source.
Technically speaking, this is treated as a separate module object by the implementation, but one that is automatically searched when the current global namespace does not contain a name. So when you use abs() in your code, and there is no abs object in the global namespace, the built-ins module is also searched for that name.
It is also exposed as the __builtin__ module (or builtins in Python 3) so you can access the built-in names even if you shadowed any in your code. Like the sys module, however, it is compiled into the Python binary, and is not available as a separate dynamically loaded file.
Ruby uses require, Python uses import. They're substantially different models, and while I'm more used to the require model, I can see a few places where I think I like import more. I'm curious what things people find particularly easy — or more interestingly, harder than they should be — with each of these models.
In particular, if you were writing a new programming language, how would you design a code-loading mechanism? Which "pros" and "cons" would weigh most heavily on your design choice?
The Python import has a major feature in that it ties two things together -- how to find the import and under what namespace to include it.
This creates very explicit code:
import xml.sax
This specifies where to find the code we want to use, by the rules of the Python search path.
At the same time, all objects that we want to access live under this exact namespace, for example xml.sax.ContentHandler.
I regard this as an advantage to Ruby's require. require 'xml' might in fact make objects inside the namespace XML or any other namespace available in the module, without this being directly evident from the require line.
If xml.sax.ContentHandler is too long, you may specify a different name when importing:
import xml.sax as X
And it is now avalable under X.ContentHandler.
This way Python requires you to explicitly build the namespace of each module. Python namespaces are thus very "physical", and I'll explain what I mean:
By default, only names directly defined in the module are available in its namespace: functions, classes and so.
To add to a module's namespace, you explicitly import the names you wish to add, placing them (by reference) "physically" in the current module.
For example, if we have the small Python package "process" with internal submodules machine and interface, and we wish to present this as one convenient namespace directly under the package name, this is and example of what we could write in the "package definition" file process/__init__.py:
from process.interface import *
from process.machine import Machine, HelperMachine
Thus we lift up what would normally be accessible as process.machine.Machine up to process.Machine. And we add all names from process.interface to process namespace, in a very explicit fashion.
The advantages of Python's import that I wrote about were simply two:
Clear what you include when using import
Explicit how you modify your own module's namespace (for the program or for others to import)
A nice property of require is that it is actually a method defined in Kernel. Thus you can override it and implement your own packaging system for Ruby, which is what e.g. Rubygems does!
PS: I am not selling monkey patching here, but the fact that Ruby's package system can be rewritten by the user (even to work like python's system). When you write a new programming language, you cannot get everything right. Thus if your import mechanism is fully extensible (into totally all directions) from within the language, you do your future users the best service. A language that is not fully extensible from within itself is an evolutionary dead-end. I'd say this is one of the things Matz got right with Ruby.
Python's import provides a very explicit kind of namespace: the namespace is the path, you don't have to look into files to know what namespace they do their definitions in, and your file is not cluttered with namespace definitions. This makes the namespace scheme of an application simple and fast to understand (just look at the source tree), and avoids simple mistakes like mistyping a namespace declaration.
A nice side effect is every file has its own private namespace, so you don't have to worry about conflicts when naming things.
Sometimes namespaces can get annoying too, having things like some.module.far.far.away.TheClass() everywhere can quickly make your code very long and boring to type. In these cases you can import ... from ... and inject bits of another namespace in the current one. If the injection causes a conflict with the module you are importing in, you can simply rename the thing you imported: from some.other.module import Bar as BarFromOtherModule.
Python is still vulnerable to problems like circular imports, but it's the application design more than the language that has to be blamed in these cases.
So python took C++ namespace and #include and largely extended on it. On the other hand I don't see in which way ruby's module and require add anything new to these, and you have the exact same horrible problems like global namespace cluttering.
Disclaimer, I am by no means a Python expert.
The biggest advantage I see to require over import is simply that you don't have to worry about understanding the mapping between namespaces and file paths. It's obvious: it's just a standard file path.
I really like the emphasis on namespacing that import has, but can't help but wonder if this particular approach isn't too inflexible. As far as I can tell, the only means of controlling a module's naming in Python is by altering the filename of the module being imported or using an as rename. Additionally, with explicit namespacing, you have a means by which you can refer to something by its fully-qualified identifier, but with implicit namespacing, you have no means to do this inside the module itself, and that can lead to potential ambiguities that are difficult to resolve without renaming.
i.e., in foo.py:
class Bar:
def myself(self):
return foo.Bar
This fails with:
Traceback (most recent call last):
File "", line 1, in ?
File "foo.py", line 3, in myself
return foo.Bar
NameError: global name 'foo' is not defined
Both implementations use a list of locations to search from, which strikes me as a critically important component, regardless of the model you choose.
What if a code-loading mechanism like require was used, but the language simply didn't have a global namespace? i.e., everything, everywhere must be namespaced, but the developer has full control over which namespace the class is defined in, and that namespace declaration occurs explicitly in the code rather than via the filename. Alternatively, defining something in the global namespace generates a warning. Is that a best-of-both-worlds approach, or is there an obvious downside to it that I'm missing?