Develop Reference

python thousands separator formatting with different character without using locale

I am looking to format thousands with an apostrophe (but could be something else) without using locale or a replace method.
The comma that is used by default in the string formatting must be defined somewhere in the Python source code but I can't find where. Is it possible to access it and change it once so that all formatting in the same session uses the new character?
To be clear (edit after #Matiss comment), I do not want to change the formatting syntax. It should stay as f"{value:,}". However, instead of inserting commas, I would like to insert something else.
Hopefully the comma is declared as a constant somewhere in Python source code (for example string._THOUSANDS_SEPARATOR), and just re-assigning it to soemthing else will do the trick.
I can not find that constant definition.
For example, one of the ways to do it now is:
>>> val = 123456
>>> print(f"{val:,}".replace(",", "'"))
123'456
The replace method is cumbersome and has to be repeated every time.
I also want to avoid the locale environment.
If the comma , is defined as a constant somewhere in Python source code, say for example in a module called python_formatting_constants, and the name of the constant was for example THOUSANDS_SEPARATOR, one could do:
>>> from python_formatting_constants import THOUSANDS_SEPARATOR
>>> THOUSANDS_SEPARATOR = "'"
>>> print(f"{val:,}") # the "," in the formatting string stays the same
123'456
>>> # but the formatted string uses the apostrophe instead of the comma

If you want to do the changes from the core of python, then you might need help from python language developers. But You can create your own format specs by creating your own class and override any provided format specs does not work for you.
class MyInt(int):
def __new__(cls, number):
return super(MyInt, cls).__new__(cls, number)
def __init__(self, value):
self.value = value
def __format__(self, format_spec):
if format_spec == ",":
return format(self.value, ",").replace(",", "'")
return format(self.value, format_spec)
Here I have created my own class which is a subclass of int and I have overloaded the __format__ built-in method of int according to your need.
Example:
# Input
test_value = TestClass(1000)
"{:,}".format(test_value) # alternative: f"{test_value:,}"
# Output
"1'000"
It is just a dummy example, You can do more deep dive into format_spec argument of __format__ method to have your own and replace any existing identifier with your own.
My suggestion will be, it will be quite easier to create your own format_spec than waiting to get some help from core python development team.
Few caveats:
If you are planning to do some add, subtract ... etc with integer and your new class, then you also have to overload all those functions for example __add__, __sub__ because any numeric operation will return pure int but not MyInt which has its own formatting.
Example:
# input
another_value = test_value + 2500
"{:,}".format(another_value)
# output
'3,500'
As you can see, the return value of + is a int but not MyInt so it has used , in the output instead. So you have to do all numeric operation beforehand and then wrap the final numeric result with the MyInt for just to have proper formatting.
More details about the format specs here.

Using f-strings to format class attribute [duplicate]

I am using template strings to generate some files and I love the conciseness of the new f-strings for this purpose, for reducing my previous template code from something like this:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print (template_a.format(**locals()))
Now I can do this, directly replacing variables:
names = ["foo", "bar"]
for name in names:
print (f"The current name is {name}")
However, sometimes it makes sense to have the template defined elsewhere — higher up in the code, or imported from a file or something. This means the template is a static string with formatting tags in it. Something would have to happen to the string to tell the interpreter to interpret the string as a new f-string, but I don't know if there is such a thing.
Is there any way to bring in a string and have it interpreted as an f-string to avoid using the .format(**locals()) call?
Ideally I want to be able to code like this... (where magic_fstring_function is where the part I don't understand comes in):
template_a = f"The current name is {name}"
# OR [Ideal2] template_a = magic_fstring_function(open('template.txt').read())
names = ["foo", "bar"]
for name in names:
print (template_a)
...with this desired output (without reading the file twice):
The current name is foo
The current name is bar
...but the actual output I get is:
The current name is {name}
The current name is {name}

Here's a complete "Ideal 2".
It's not an f-string—it doesn't even use f-strings—but it does as requested. Syntax exactly as specified. No security headaches since we are not using eval().
It uses a little class and implements __str__ which is automatically called by print. To escape the limited scope of the class we use the inspect module to hop one frame up and see the variables the caller has access to.
import inspect
class magic_fstring_function:
def __init__(self, payload):
self.payload = payload
def __str__(self):
vars = inspect.currentframe().f_back.f_globals.copy()
vars.update(inspect.currentframe().f_back.f_locals)
return self.payload.format(**vars)
template = "The current name is {name}"
template_a = magic_fstring_function(template)
# use it inside a function to demonstrate it gets the scoping right
def new_scope():
names = ["foo", "bar"]
for name in names:
print(template_a)
new_scope()
# The current name is foo
# The current name is bar

A concise way to have a string evaluated as an f-string (with its full capabilities) is using following function:
def fstr(template):
return eval(f"f'{template}'")
Then you can do:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print(fstr(template_a))
# The current name is foo
# The current name is bar
And, in contrast to many other proposed solutions, you can also do:
template_b = "The current name is {name.upper() * 2}"
for name in names:
print(fstr(template_b))
# The current name is FOOFOO
# The current name is BARBAR

This means the template is a static string with formatting tags in it
Yes, that's exactly why we have literals with replacement fields and .format, so we can replace the fields whenever we like by calling format on it.
Something would have to happen to the string to tell the interpreter to interpret the string as a new f-string
That's the prefix f/F. You could wrap it in a function and postpone the evaluation during call time but of course that incurs extra overhead:
def template_a():
return f"The current name is {name}"
names = ["foo", "bar"]
for name in names:
print(template_a())
Which prints out:
The current name is foo
The current name is bar
but feels wrong and is limited by the fact that you can only peek at the global namespace in your replacements. Trying to use it in a situation which requires local names will fail miserably unless passed to the string as arguments (which totally beats the point).
Is there any way to bring in a string and have it interpreted as an f-string to avoid using the .format(**locals()) call?
Other than a function (limitations included), nope, so might as well stick with .format.

How about:
s = 'Hi, {foo}!'
s
> 'Hi, {foo}!'
s.format(foo='Bar')
> 'Hi, Bar!'

Using .format is not a correct answer to this question. Python f-strings are very different from str.format() templates ... they can contain code or other expensive operations - hence the need for deferral.
Here's an example of a deferred logger. This uses the normal preamble of logging.getLogger, but then adds new functions that interpret the f-string only if the log level is correct.
log = logging.getLogger(__name__)
def __deferred_flog(log, fstr, level, *args):
if log.isEnabledFor(level):
import inspect
frame = inspect.currentframe().f_back.f_back
try:
fstr = 'f"' + fstr + '"'
log.log(level, eval(fstr, frame.f_globals, frame.f_locals))
finally:
del frame
log.fdebug = lambda fstr, *args: __deferred_flog(log, fstr, logging.DEBUG, *args)
log.finfo = lambda fstr, *args: __deferred_flog(log, fstr, logging.INFO, *args)
This has the advantage of being able to do things like: log.fdebug("{obj.dump()}") .... without dumping the object unless debugging is enabled.
IMHO: This should have been the default operation of f-strings, however now it's too late. F-string evaluation can have massive and unintended side-effects, and having that happen in a deferred manner will change program execution.
In order to make f-strings properly deferred, python would need some way of explicitly switching behavior. Maybe use the letter 'g'? ;)
It has been pointed out that deferred logging shouldn't crash if there's a bug in the string converter. The above solution can do this as well, change the finally: to except:, and stick a log.exception in there.

An f-string is simply a more concise way of creating a formatted string, replacing .format(**names) with f. If you don't want a string to be immediately evaluated in such a manner, don't make it an f-string. Save it as an ordinary string literal, and then call format on it later when you want to perform the interpolation, as you have been doing.
Of course, there is an alternative with eval.
template.txt:
f'The current name is {name}'
Code:
>>> template_a = open('template.txt').read()
>>> names = 'foo', 'bar'
>>> for name in names:
... print(eval(template_a))
...
The current name is foo
The current name is bar
But then all you've managed to do is replace str.format with eval, which is surely not worth it. Just keep using regular strings with a format call.

What you want appears to be being considered as a Python enhancement.
Meanwhile — from the linked discussion — the following seems like it would be a reasonable workaround that doesn't require using eval():
class FL:
def __init__(self, func):
self.func = func
def __str__(self):
return self.func()
template_a = FL(lambda: f"The current name, number is {name!r}, {number+1}")
names = "foo", "bar"
numbers = 40, 41
for name, number in zip(names, numbers):
print(template_a)
Output:
The current name, number is 'foo', 41
The current name, number is 'bar', 42

inspired by the answer by kadee, the following can be used to define a deferred-f-string class.
class FStr:
def __init__(self, s):
self._s = s
def __repr__(self):
return eval(f"f'{self._s}'")
...
template_a = FStr('The current name is {name}')
names = ["foo", "bar"]
for name in names:
print (template_a)
which is exactly what the question asked for

Or maybe do not use f-strings, just format:
fun = "The curent name is {name}".format
names = ["foo", "bar"]
for name in names:
print(fun(name=name))
In version without names:
fun = "The curent name is {}".format
names = ["foo", "bar"]
for name in names:
print(fun(name))

Most of these answers will get you something that behaves sort of like f-strings some of the time, but they will all go wrong in some cases.
There is a package on pypi f-yeah that does all this, only costing you two extra characters! (full disclosure, I am the author)
from fyeah import f
print(f("""'{'"all" the quotes'}'"""))
There are a lot of differences between f-strings and format calls, here is a probably incomplete list
f-strings allow for arbitrary eval of python code
f-strings cannot contain a backslash in the expression (since formatted strings don't have an expression, so I suppose you could say this isn't a difference, but it does differ from what a raw eval() can do)
dict lookups in formatted strings must not be quoted. dict lookups in f-strings can be quoted, and so non-string keys can also be looked up
f-strings have a debug format that format() does not: f"The argument is {spam=}"
f-string expressions cannot be empty
The suggestions to use eval will get you full f-string format support, but they don't work on all string types.
def f_template(the_string):
return eval(f"f'{the_string}'")
print(f_template('some "quoted" string'))
print(f_template("some 'quoted' string"))
some "quoted" string
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in f_template
File "<string>", line 1
f'some 'quoted' string'
^
SyntaxError: invalid syntax
This example will also get variable scoping wrong in some cases.

to do that I prefer to use fstring inside a lambda function like:
s = lambda x: f'this is your string template to embed {x} in it.'
n = ['a' , 'b' , 'c']
for i in n:
print( s(i) )

There's a lot of talk about using str.format(), but as noted it doesn't allow most of the expressions that are allowed in f-strings such as arithmetic or slices. Using eval() obviously also has it's downsides.
I'd recommend looking into a templating language such as Jinja. For my use-case it works quite well. See the example below where I have overridden the variable annotation syntax with a single curly brace to match the f-string syntax. I didn't fully review the differences between f-strings and Jinja invoked like this.
from jinja2 import Environment, BaseLoader
a, b, c = 1, 2, "345"
templ = "{a or b}{c[1:]}"
env = Environment(loader=BaseLoader, variable_start_string="{", variable_end_string="}")
env.from_string(templ).render(**locals())
results in
'145'

A suggestion that uses f-strings. Do your evaluation on the
logical level where the templating is occurring and pass it as a generator.
You can unwind it at whatever point you choose, using f-strings
In [46]: names = (i for i in ('The CIO, Reed', 'The homeless guy, Arnot', 'The security guard Spencer'))
In [47]: po = (f'Strangely, {next(names)} has a nice {i}' for i in (" nice house", " fast car", " big boat"))
In [48]: while True:
...: try:
...: print(next(po))
...: except StopIteration:
...: break
...:
Strangely, The CIO, Reed has a nice nice house
Strangely, The homeless guy, Arnot has a nice fast car
Strangely, The security guard Spencer has a nice big boat

You could use a .format styled replacement and explicitly define the replaced variable name:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print (template_a.format(name=name))
Output
The current name is foo
The current name is bar

How to stack multiple calls? [duplicate]

I'm trying to create a function that chains results from multiple arguments.
def hi(string):
print(string)<p>
return hi
Calling hi("Hello")("World") works and becomes Hello \n World as expected.
the problem is when I want to append the result as a single string, but
return string + hi produces an error since hi is a function.
I've tried using __str__ and __repr__ to change how hi behaves when it has not input. But this only creates a different problem elsewhere.
hi("Hello")("World") = "Hello"("World") -> Naturally produces an error.
I understand why the program cannot solve it, but I cannot find a solution to it.

You're running into difficulty here because the result of each call to the function must itself be callable (so you can chain another function call), while at the same time also being a legitimate string (in case you don't chain another function call and just use the return value as-is).
Fortunately Python has you covered: any type can be made to be callable like a function by defining a __call__ method on it. Built-in types like str don't have such a method, but you can define a subclass of str that does.
class hi(str):
def __call__(self, string):
return hi(self + '\n' + string)
This isn't very pretty and is sorta fragile (i.e. you will end up with regular str objects when you do almost any operation with your special string, unless you override all methods of str to return hi instances instead) and so isn't considered very Pythonic.
In this particular case it wouldn't much matter if you end up with regular str instances when you start using the result, because at that point you're done chaining function calls, or should be in any sane world. However, this is often an issue in the general case where you're adding functionality to a built-in type via subclassing.
To a first approximation, the question in your title can be answered similarly:
class add(int): # could also subclass float
def __call__(self, value):
return add(self + value)
To really do add() right, though, you want to be able to return a callable subclass of the result type, whatever type it may be; it could be something besides int or float. Rather than trying to catalog these types and manually write the necessary subclasses, we can dynamically create them based on the result type. Here's a quick-and-dirty version:
class AddMixIn(object):
def __call__(self, value):
return add(self + value)
def add(value, _classes={}):
t = type(value)
if t not in _classes:
_classes[t] = type("add_" + t.__name__, (t, AddMixIn), {})
return _classes[t](value)
Happily, this implementation works fine for strings, since they can be concatenated using +.
Once you've started down this path, you'll probably want to do this for other operations too. It's a drag copying and pasting basically the same code for every operation, so let's write a function that writes the functions for you! Just specify a function that actually does the work, i.e., takes two values and does something to them, and it gives you back a function that does all the class munging for you. You can specify the operation with a lambda (anonymous function) or a predefined function, such as one from the operator module. Since it's a function that takes a function and returns a function (well, a callable object), it can also be used as a decorator!
def chainable(operation):
class CallMixIn(object):
def __call__(self, value):
return do(operation(self, value))
def do(value, _classes={}):
t = type(value)
if t not in _classes:
_classes[t] = type(t.__name__, (t, CallMixIn), {})
return _classes[t](value)
return do
add = chainable(lambda a, b: a + b)
# or...
import operator
add = chainable(operator.add)
# or as a decorator...
#chainable
def add(a, b): return a + b
In the end it's still not very pretty and is still sorta fragile and still wouldn't be considered very Pythonic.
If you're willing to use an additional (empty) call to signal the end of the chain, things get a lot simpler, because you just need to return functions until you're called with no argument:
def add(x):
return lambda y=None: x if y is None else add(x+y)
You call it like this:
add(3)(4)(5)() # 12

You are getting into some deep, Haskell-style, type-theoretical issues by having hi return a reference to itself. Instead, just accept multiple arguments and concatenate them in the function.
def hi(*args):
return "\n".join(args)
Some example usages:
print(hi("Hello", "World"))
print("Hello\n" + hi("World"))

How to document python function parameter types?

I know that the parameters can be any object but for the documentation it is quite important to specify what you would expect.
First is how to specify a parameter types like these below?
str (or use String or string?)
int
list
dict
function()
tuple
object instance of class MyClass
Second, how to specify params that can be of multiple types like a function that can handle a single parameter than can be int or str?
Please use the below example to demonstrate the syntax needed for documenting this with your proposed solution. Mind that it is desired to be able to hyperlink reference to the "Image" class from inside the documentation.
def myMethod(self, name, image):
"""
Does something ...
name String: name of the image
image Image: instance of Image Class or a string indicating the filename.
Return True if operation succeeded or False.
"""
return True
Note, you are welcome to suggest the usage of any documentation tool (sphinx, oxygen, ...) as long it is able to deal with the requirements.
Update:
It seams that there is some kind of support for documenting parameter types in doxygen in. general. The code below works but adds an annoying $ to the param name (because it was initially made for php).
#param str $arg description
#param str|int $arg description

There is a better way. We use
def my_method(x, y):
"""
my_method description
#type x: int
#param x: An integer
#type y: int|string
#param y: An integer or string
#rtype: string
#return: Returns a sentence with your variables in it
"""
return "Hello World! %s, %s" % (x,y)
That's it. In the PyCharm IDE this helps a lot. It works like a charm ;-)

You need to add an exclamation mark at the start of the Python docstring for Doxygen to parse it correctly.
def myMethod(self, name, image):
"""!
Does something ...
#param name String: name of the image
#param image Image: instance of Image Class or a string indicating the filename.
#return Return True if operation succeeded or False.
"""
return True

If using Python 3, you can use the function annotations described in PEP 3107.
def compile(
source: "something compilable",
filename: "where the compilable thing comes from",
mode: "is this a single statement or a suite?"):
See also function definitions.

Figured I'd post this little tidbit here since IDEA showed me this was possible, and I was never told nor read about this.
>>> def test( arg: bool = False ) -> None: print( arg )
>>> test(10)
10
When you type test(, IDLE's doc-tip appears with (arg: bool=False) -> None Which was something I thought only Visual Studio did.
It's not exactly doxygen material, but it's good for documenting parameter-types for those using your code.

Yup, #docu is right - this is the (IMHO best) way to combine both documentation schemes more or less seamlessly. If, on the other hand, you also want to do something like putting text on the doxygen-generated index page, you would add
##
# #mainpage (Sub)Heading for the doxygen-generated index page
# Text that goes right onto the doxygen-generated index page
somewhere at the beginning of your Python code.
In other words, where doxygen does not expect Python comments, use ## to alert it that there are tags for it. Where it expects Python comments (e.g. at the beginning of functions or classes), use """!.

Doxygen is great for C++, but if you are working with mostly python code you should give sphinx a try. If you choose sphinx then all you need to do is follow pep8.

Passing a list to eval()

Is there a way to pass a list as a function argument to eval() Or do I have to convert it to a string and then parse it as a list in the function?
My simple example looks like:
eval("func1(\'" + fArgs + "\')")
I'm just not sure if there is a better way of taking fArgs as a list instead of a string
Note:
The list is provided from a JSON response
EDIT: Ok here's a bit more of my class so there's a better understanding of how I'm using eval
def test(arg):
print arg
#Add all allowed functions to this list to be mapped to a dictionary
safe_list = ['test']
safe_dict = dict([ (k, locals().get(k, None)) for k in safe_list ])
class Validate:
def __init__(self, Value, fName, fArgs):
eval(fName + "(\'" + fArgs + "\')", {"__builtins__":None},safe_dict)
I may be wrong in thinking this, but to my understanding this is a safe use of eval because the only functions that can be called are the ones that are listed in the safe_list dictionary. The function to be run and the arguments for that function are being extracted out of a JSON object. The arguments are to be structured as a list, Will joining the list together with ", " be interpreted as actual arguments or just a single argument?

If you're using Python 2.6.x, then you should be able to use the json module (see py doc 19.2). If not, then there is python-json available through the python package index. Both of these packages will provide a reader for parsing JSON data into an appropriate Python data type.
For your second problem of calling a function determined by a message, you can do the following:
def foo():
print 'I am foo!'
def bar():
pass
def baz():
pass
funcs = {'func_a':foo, 'func_b':bar, 'func_c':baz}
funcs['func_a']()
This approach can be a bit more secure than eval because it prevents 'unsafe' python library functions from being injected into the JSON. However, you still need to be cautious that the data supplied to your functions can't be manipulated to cause problems.

Specifying parameters the following way works:
root#parrot$ more test.py
def func1(*args):
for i in args:
print i
l = [1,'a',9.1]
func1(*l)
root#parrot$ python test.py
1
a
9.1
so, no direct need for eval(), unless I'm misunderstanding something.

Using a library to parse JSON input may be a better approach than eval, something like:
import json
func1(json.loads(fArgs))
Assert-ing that user input is correct would be a good idea, too.

The others have a good point, that you shouldn't be using eval. But, if you must:
eval("func1(%s)" % ", ".join(fArgs))
will call the function with all the arguments in the list. This:
eval("func1([%s])" % ", ".join(fArgs))
will call it with the list of arguments in just one argument. Maybe you even want this?
eval("func1([%s])" % ", ".join(map(eval, fArgs)))
which would eval the arguments as well?