Develop Reference

Why wrapper and wrapped function are the same for some python codes.

I was reading Ian Goodfellow's GAN source code in Github (link https://github.com/goodfeli/adversarial/blob/master/deconv.py). In particular, at line 40/41, the code is:
#functools.wraps(Model.get_lr_scalers)
def get_lr_scalers(self):
It's a rather unfamiliar way of using wraps, and it seems the goal is to replace the get_lr_scalers with a user defined function. But in that case, we don't really need a wrapper for that, right? I don't really know the purpose of wraps in this case.

wraps copies a number of attributes from another function onto this function—by default, __module__, __name__, __qualname__, __annotations__ and __doc__.
The most obviously useful one to copy over is the __doc__. Consider this simpler example:1
class Base:
def spam(self, breakfast):
"""spam(self, breakfast) -> breakfast with added spam
<29 lines of detailed information here>
"""
class Child:
#functools.wraps(Base.spam)
def spam(self, breakfast):
newbreakfast = breakfast.copy()
newbreakfast.meats['spam'] + 30
return newbreakfast
Now if someone wants to use help(mychild.spam), they'll get the 29 lines of useful information. (Or, if they autocomplete mychild.spam in PyCharm, it'll pop up the overlay with the documentation, etc.) All without me having to manually copy and paste it. And, even better, if Base came from some framework that I didn't write, and my user upgrades from 1.2.3 to 1.2.4 of that framework, and there's a better docstring, they'll see that better docstring.
In the most common case, Child would be a subclass of Base, and spam would be an override.2 But that isn't actually required—wraps doesn't care whether you're subtyping via inheritance, or duck typing by just implementing an implicit protocol; it's equally useful for both cases. As long as Child is intended to implement the spam protocol from Base, it makes sense for Child.spam to have the same docstring (and maybe other metadata attributes).
Others attributes probably aren't quite as useful as docstrings. For example, if you're using type annotations, their benefit in reading the code is probably at least as high as their benefit in being able to run Mypy for static type checking, so just copying them over dynamically from another method often isn't all that useful. And __module__ and __qualname__ are primarily used for reflection/inspection, and are more likely to be misleading than helpful in this case (although you could probably come up with an example of a framework where you'd want people to read the code in Base instead of the code in Child, that isn't true for the default obvious example). But, unless they're actively harmful, the readability cost of using #functools.wraps(Base.spam, assigned=('__doc__',)) instead of just the defaults may not be worth it.
1. If you're using Python 2, change these classes to inherit from object; otherwise they'll be old-style classes, which just complicates things in an irrelevant way. If Python 3, there are no old-style classes, so this issue can't even arise.
2. Or maybe a "virtual subclass" of an ABC, declared via a register call, or via a subclass hook.

The purpose of #wraps is to copy meta information of one function to another function. This is usually done when replacing the original function by wrapping it, which is often done by decorators.
But in general case, here is what it does in an example:
def f1():
"""Function named f1. Prints 'f1'."""
print('f1')
#functools.wraps(f1)
def f2():
print('f2')
Now, you can test what happened:
>>> f1
<function f1 at 0x006AD8E8>
>>> f2
<function f1 at 0x006AD978>
>>> f1()
f1
>>> f2()
f2
>>> f1.__doc__
"Function named f1. Prints 'f1'."
>>> f2.__doc__
"Function named f1. Prints 'f1'."
When you call f2, it is obvious that it is actually f2, but when you inspect it, it behaves like f1 - it has the same doc string and the same name.
What is that good for? For this:
f1 = f2
Now the original f1 is replaced with a new functionality, but it still looks like f1 from the outside.
It is usually done in a decorator:
def replace(func):
#functools.wraps(func)
def replacement():
print('replacement')
return replacement
#replace
def f1():
"""Function named f1. Prints 'f1'."""
print('f1')
And it behaves like this:
>>> f1()
replacement
>>> f1
<function f1 at 0x006AD930>
>>> f1.__name__
'f1'
>>> f1.__doc__
"Function named f1. Prints 'f1'."

Function visible to the module classes but not outside [duplicate]

According to http://www.faqs.org/docs/diveintopython/fileinfo_private.html:
Like most languages, Python has the
concept of private elements:
Private
functions, which can't be called from
outside their module
However, if I define two files:
#a.py
__num=1
and:
#b.py
import a
print a.__num
when i run b.py it prints out 1 without giving any exception. Is diveintopython wrong, or did I misunderstand something? And is there some way to do define a module's function as private?

In Python, "privacy" depends on "consenting adults'" levels of agreement - you can't force it (any more than you can in real life;-). A single leading underscore means you're not supposed to access it "from the outside" -- two leading underscores (w/o trailing underscores) carry the message even more forcefully... but, in the end, it still depends on social convention and consensus: Python's introspection is forceful enough that you can't handcuff every other programmer in the world to respect your wishes.
((Btw, though it's a closely held secret, much the same holds for C++: with most compilers, a simple #define private public line before #includeing your .h file is all it takes for wily coders to make hash of your "privacy"...!-))

There may be confusion between class privates and module privates.
A module private starts with one underscore
Such a element is not copied along when using the from <module_name> import * form of the import command; it is however imported if using the import <moudule_name> syntax (see Ben Wilhelm's answer)
Simply remove one underscore from the a.__num of the question's example and it won't show in modules that import a.py using the from a import * syntax.
A class private starts with two underscores (aka dunder i.e. d-ouble under-score)
Such a variable has its name "mangled" to include the classname etc.
It can still be accessed outside of the class logic, through the mangled name.
Although the name mangling can serve as a mild prevention device against unauthorized access, its main purpose is to prevent possible name collisions with class members of the ancestor classes.
See Alex Martelli's funny but accurate reference to consenting adults as he describes the convention used in regards to these variables.
>>> class Foo(object):
... __bar = 99
... def PrintBar(self):
... print(self.__bar)
...
>>> myFoo = Foo()
>>> myFoo.__bar #direct attempt no go
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'Foo' object has no attribute '__bar'
>>> myFoo.PrintBar() # the class itself of course can access it
99
>>> dir(Foo) # yet can see it
['PrintBar', '_Foo__bar', '__class__', '__delattr__', '__dict__', '__doc__', '__
format__', '__getattribute__', '__hash__', '__init__', '__module__', '__new__',
'__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__
', '__subclasshook__', '__weakref__']
>>> myFoo._Foo__bar #and get to it by its mangled name ! (but I shouldn't!!!)
99
>>>

This question was not fully answered, since module privacy is not purely conventional, and since using import may or may not recognize module privacy, depending on how it is used.
If you define private names in a module, those names will be imported into any script that uses the syntax, 'import module_name'. Thus, assuming you had correctly defined in your example the module private, _num, in a.py, like so..
#a.py
_num=1
..you would be able to access it in b.py with the module name symbol:
#b.py
import a
...
foo = a._num # 1
To import only non-privates from a.py, you must use the from syntax:
#b.py
from a import *
...
foo = _num # throws NameError: name '_num' is not defined
For the sake of clarity, however, it is better to be explicit when importing names from modules, rather than importing them all with a '*':
#b.py
from a import name1
from a import name2
...

Python allows for private class members with the double underscore prefix. This technique doesn't work at a module level so I am thinking this is a mistake in Dive Into Python.
Here is an example of private class functions:
class foo():
def bar(self): pass
def __bar(self): pass
f = foo()
f.bar() # this call succeeds
f.__bar() # this call fails

You can add an inner function:
def public(self, args):
def private(self.root, data):
if (self.root != None):
pass #do something with data
Something like that if you really need that level of privacy.

This is an ancient question, but both module private (one underscore) and class-private (two underscores) mangled variables are now covered in the standard documentation:
The Python Tutorial » Classes » Private Variables

embedded with closures or functions is one way. This is common in JS although not required for non-browser platforms or browser workers.
In Python it seems a bit strange, but if something really needs to be hidden than that might be the way. More to the point using the python API and keeping things that require to be hidden in the C (or other language) is probably the best way. Failing that I would go for putting the code inside a function, calling that and having it return the items you want to export.

Sorry if I'm late to answer, but in a module, you can define the packages to "export" like this:
mymodule
__init__.py
library.py
main.py
mymodule/library.py
# 'private' function
def _hello(name):
return f"Hello {name}!"
# 'public' function which is supposed to be used instead of _hello
def hello():
name = input('name: ')
print(_hello(name))
mymodule/__init__.py
# only imports certain functions from library
from .library import hello
main.py
import mymodule
mymodule.hello()
Nevertheless, functions can still be accessed,
from mymodule.library import _hello
print(_hello('world'))
But this approach makes it less obvious

For methods: (I am not sure if this exactly what you want)
print_thrice.py
def private(method):
def methodist(string):
if __name__ == "__main__":
method(string)
return methodist
#private
def private_print3(string):
print(string * 3)
private_print3("Hello ") # output: Hello Hello Hello
other_file.py
from print_thrice import private_print3
private_print3("Hello From Another File? ") # no output
This is probably not a perfect solution, as you can still "see" and/or "call" the method. Regardless, it doesn't execute.

See PEP8 guideline:
Method Names and Instance Variables
Use the function naming rules: lowercase with words separated by underscores as necessary to improve
readability.
Use one leading underscore only for non-public methods and instance
variables.
To avoid name clashes with subclasses, use two leading underscores to
invoke Python’s name mangling rules.
Python mangles these names with the class name: if class Foo has an
attribute named __a, it cannot be accessed by Foo.__a. (An insistent
user could still gain access by calling Foo._Foo__a.) Generally,
double leading underscores should be used only to avoid name conflicts
with attributes in classes designed to be subclassed.
Designing for Inheritance
Always decide whether a class’s methods and
instance variables (collectively: “attributes”) should be public or
non-public. If in doubt, choose non-public; it’s easier to make it
public later than to make a public attribute non-public.
Public attributes are those that you expect unrelated clients of your
class to use, with your commitment to avoid backwards incompatible
changes. Non-public attributes are those that are not intended to be
used by third parties; you make no guarantees that non-public
attributes won’t change or even be removed.
We don’t use the term “private” here, since no attribute is really
private in Python (without a generally unnecessary amount of work).

Python has three modes via., private, public and protected .While importing a module only public mode is accessible .So private and protected modules cannot be called from outside of the module i.e., when it is imported .

Synthetic functions in python

In python I can create a class without class statement:
MyClass = type('X', (object,), dict(a=1))
Is there a way to create a function without 'def'?
Thats as far as i got...
d={} # func from string
exec'''\
def synthetics(s):
return s*s+1
''' in d
>>> d.keys()
['__builtins__', 'synthetics']
>>> d['synthetics']
<function synthetics at 0x00D09E70>
>>> foo = d['synthetics']
>>> foo(1)
2

Technically, yes, this is possible. The type of a function is, like all other types, a constructor for instances of that type:
FunctionType = type(lambda: 0)
help(FunctionType)
As you can see from the help, you need at minimum code and globals. The former is a compiled bytecode object; the latter is a dictionary.
To make the code object, you can use the code type's constructor:
CodeType = type((lambda: 0).func_code)
help(CodeType)
The help says this is "not for the faint of heart" and that's true. You need to pass bytecode and a bunch of other stuff to this constructor. So the easiest way to get a code object is from another function, or using the compile() function. But it is technically possible to generate code objects completely synthetically if you understand Python bytecode well enough. (I have done this, on a very limited basis, to construct signature-preserving wrapper functions for use in decorators.)
PS -- FunctionType and CodeType are also available via the types module.

There might be a more direct way than the following, but here's a full-blown function without def. First, use a trivial lambda expression to get a function object:
>>> func = lambda: None
Then, compile some source code to get a code object and use that to replace the lambda's code:
>>> func.__code__ = compile("print('Hello, world!')", "<no file>", "exec")
>>> func()
Hello, world!

How is a Python project set up?

I am doing some heavy commandline stuff (not really web based) and am new to Python, so I was wondering how to set up my files/folders/etc. Are there "header" files where I can keep all the DB connection stuff?
How/where do I define classes and objects?

Just to give you an example of a typical Python module's source, here's something with some explanation. This is a file named "Dims.py". This is not the whole file, just some parts to give an idea what's going on.
#!/usr/bin/env python
This is the standard first line telling the shell how to execute this file. Saying /usr/bin/env python instead of /usr/bin/python tells the shell to find Python via the user's PATH; the desired Python may well be in ~/bin or /usr/local/bin.
"""Library for dealing with lengths and locations."""
If the first thing in the file is a string, it is the docstring for the module. A docstring is a string that appears immediately after the start of an item, which can be accessed via its __doc__ property. In this case, since it is the module's docstring, if a user imports this file with import Dims, then Dims.__doc__ will return this string.
# Units
MM_BASIC = 1500000
MILS_BASIC = 38100
IN_BASIC = MILS_BASIC * 1000
There are a lot of good guidelines for formatting and naming conventions in a document known as PEP (Python Enhancement Proposal) 8. These are module-level variables (constants, really) so they are written in all caps with underscores. No, I don't follow all the rules; old habits die hard. Since you're starting fresh, follow PEP 8 unless you can't.
_SCALING = 1
_SCALES = {
mm_basic: MM_BASIC,
"mm": MM_BASIC,
mils_basic: MILS_BASIC,
"mil": MILS_BASIC,
"mils": MILS_BASIC,
"basic": 1,
1: 1
}
These module-level variables have leading underscores in their names. This gives them a limited amount of "privacy", in that import Dims will not let you access Dims._SCALING. However, if you need to mess with it, you can explicitly say something like import Dims._SCALING as scaling.
def UnitsToScale(units=None):
"""Scales the given units to the current scaling."""
if units is None:
return _SCALING
elif units not in _SCALES:
raise ValueError("unrecognized units: '%s'." % units)
return _SCALES[units]
UnitsToScale is a module-level function. Note the docstring and the use of default values and exceptions. No spaces around the = in default value declarations.
class Length(object):
"""A length. Makes unit conversions easier.
The basic, mm, and mils properties can be used to get or set the length
in the desired units.
>>> x = Length(mils=1000)
>>> x.mils
1000.0
>>> x.mm
25.399999999999999
>>> x.basic
38100000L
>>> x.mils = 100
>>> x.mm
2.54
"""
The class declaration. Note the docstring has things in it that look like Python command line commands. These care called doctests, in that they are test code in the docstring. More on this later.
def __init__(self, unscaled=0, basic=None, mm=None, mils=None, units=None):
"""Constructs a Length.
Default contructor creates a length of 0.
>>> Length()
Length(basic=0)
Length(<float>) or Length(<string>) creates a length with the given
value at the current scale factor.
>>> Length(1500)
Length(basic=1500)
>>> Length("1500")
Length(basic=1500)
"""
# Straight copy
if isinstance(unscaled, Length):
self._x = unscaled._x
return
# rest omitted
This is the initializer. Unlike C++, you only get one, but you can use default arguments to make it look like several different constructors are available.
def _GetBasic(self): return self._x
def _SetBasic(self, x): self._x = x
basic = property(_GetBasic, _SetBasic, doc="""
This returns the length in basic units.""")
This is a property. It allows you to have getter/setter functions while using the same syntax as you would for accessing any other data member, in this case, myLength.basic = 10 does the same thing as myLength._SetBasic(10). Because you can do this, you should not write getter/setter functions for your data members by default. Just operate directly on the data members. If you need to have getter/setter functions later, you can convert the data member to a property and your module's users won't need to change their code. Note that the docstring is on the property, not the getter/setter functions.
If you have a property that is read-only, you can use property as a decorator to declare it. For example, if the above property was to be read-only, I would write:
#property
def basic(self):
"""This returns the length in basic units."""
return self._x
Note that the name of the property is the name of the getter method. You can also use decorators to declare setter methods in Python 2.6 or later.
def __mul__(self, other):
"""Multiplies a Length by a scalar.
>>> Length(10)*10
Length(basic=100)
>>> 10*Length(10)
Length(basic=100)
"""
if type(other) not in _NumericTypes:
return NotImplemented
return Length(basic=self._x * other)
This overrides the * operator. Note that you can return the special value NotImplemented to tell Python that this operation isn't implemented (in this case, if you try to multiply by a non-numeric type like a string).
__rmul__ = __mul__
Since code is just a value like anything else, you can assign the code of one method to another. This line tells Python that the something * Length operation uses the same code as Length * something. Don't Repeat Yourself.
Now that the class is declared, I can get back to module code. In this case, I have some code that I want to run only if this file is executed by itself, not if it's imported as a module. So I use the following test:
if __name__ == "__main__":
Then the code in the if is executed only if this is being run directly. In this file, I have the code:
import doctest
doctest.testmod()
This goes through all the docstrings in the module and looks for lines that look like Python prompts with commands after them. The lines following are assumed to be the output of the command. If the commands output something else, the test is considered to have failed and the actual output is printed. Read the doctest module documentation for all the details.
One final note about doctests: They're useful, but they're not the most versatile or thorough tests available. For those, you'll want to read up on unittests (the unittest module).

Each python source file is a module. There are no "header" files. The basic idea is that when you import "foo" it'll load the code from "foo.py" (or a previously compiled version of it). You can then access the stuff from the foo module by saying foo.whatever.
There seem to be two ways for arranging things in Python code. Some projects use a flat layout, where all of the modules are at the top-level. Others use a hierarchy. You can import foo/bar/baz.py by importing "foo.bar.baz". The big gotcha with hierarchical layout is to have __init__.py in the appropriate directories (it can even be empty, but it should exist).
Classes are defined like this:
class MyClass(object):
def __init__(self, x):
self.x = x
def printX(self):
print self.x
To create an instance:
z = MyObject(5)

You can organize it in whatever way makes the most sense for your application. I don't exactly know what you're doing so I can't be certain what the best organization would be for you, but you can pretty much split it up as you see fit and just import what you need.
You can define classes in any file, and you can define as many classes as you would like in a script (unlike Java). There are no official header files (not like C or C++), but you can use config files to store info about connecting to a DB, whatever, and use configparser (a standard library function) to organize them.
It makes sense to keep like things in the same file, so if you have a GUI, you might have one file for the interface, and if you have a CLI, you might keep that in a file by itself. It's less important how your files are organized and more important how the source is organized into classes and functions.

This would be the place to look for that: http://docs.python.org/reference/.
First of all, compile and install pip: http://pypi.python.org/pypi/pip. It is like Ubuntu's apt-get. You run it via a Terminal by typing in pip install package-name. It has a database of packages, so you can install/uninstall stuff quite easily with it.
As for importing and "header" files, from what I can tell, if you run import foo, Python looks for foo.py in the current folder. If it's not there, it looks for eggs (folders unzipped in the Python module directory) and imports those.
As for defining classes and objects, here's a basic example:
class foo(foobar2): # I am extending a class, in this case 'foobar2'. I take no arguments.
__init__(self, the, list, of, args = True): # Instead, the arguments get passed to me. This still lets you define a 'foo()' objects with three arguments, only you let '__init__' take them.
self.var = 'foo'
def bar(self, args):
self.var = 'bar'
def foobar(self): # Even if you don't need arguments, never leave out the self argument. It's required for classes.
print self.var
foobar = foo('the', 'class', 'args') # This is how you initialize me!
Read more on this in the Python Reference, but my only tip is to never forget the self argument in class functions. It will save you a lot of debugging headaches...
Good luck!

There's no some fixed structure for Python programs, but you can take Django project as an example. Django project consists of one settings.py module, where global settings (like your example with DB connection properties) are stored and pluggable applications. Each application has it's own models.py module, which stores database models and, possibly, other domain specific objects. All the rest is up to you.
Note, that these advices are not specific to Python. In C/C++ you probably used similar structure and kept settings in XML. Just forget about headers and put settings in plain in .py file, that's all.

In python django how do you print out an object's introspection? The list of all public methods of that object (variable and/or functions)?

In python django how do you print out an object's inrospection? The list of all public methods of that object (variable and/or functions)?
e.g.:
def Factotum(models.Model):
id_ref = models.IntegerField()
def calculateSeniorityFactor():
return (1000 - id_ref) * 1000
I want to be able to run a command line in the Django shell to tell me all of the public methods of a Django model. The output of running on above would be:
>> introspect Factotoum
--> Variable: id_ref
--> Methods: calculateSeniorityFactor

Well, things you can introspect are many, not just one.
Good things to start with are:
>>> help(object)
>>> dir(object)
>>> object.__dict__
Also take a look at the inspect module in the standard library.
That should make 99% of all the bases belong to you.

Use inspect:
import inspect
def introspect(something):
methods = inspect.getmembers(something, inspect.ismethod)
others = inspect.getmembers(something, lambda x: not inspect.ismethod(x))
print 'Variable:', # ?! what a WEIRD heading you want -- ah well, w/ever
for name, _ in others: print name,
print
print 'Methods:',
for name, _ in methods: print name,
print
There's no way you can invoke this without parentheses in a normal Python shell, you'll have to use introspect(Factotum) ((with class Factotum property imported in the current namespace of course)) and not introspect Factotum with a space. If this irks you terribly, you may want to look at IPython.