Develop Reference

Arbitrary type conversion in python

An arbitrary typecasting function (shown below as cast) seems like a fairly straightforward function:
print(type(variable))
variable = cast(variable,type) # where type is any type included in __builtins__
print(type(variable))
And the result:
>>> <original_type>
>>> <type>
Does such a function exist in python? I can't seem to find any reference to it if it does. If this function does not exist, please explain the rationale for why it does not.
As one example usage, I have a config with arbitrarily many values, and a schema with the desired type of each. I want to check that specified value for each config variable can be cast as corresponding type specified in the schema. Treating each as a dict below for convenience:
for variable in config.keys():
val = config[variable]
type_name = schema[variable]
try:
config[variable] = cast(val,type_name)
except TypeError:
print("Schema checking failed for variable {}".format(variable))

Ok, I think the comments have covered the matter in enough detail so I'll just try to summarize my best understanding of them here. Most of this is by way of #juanpa.arrivillaga.
A standard python casting operation like int(x) (or more precisely, a type conversion operation), is actually a call to the __call__() function of an object. Types like int, float, str, etc are all object classes and are all instances of the metaclass type. A call to one of these instance of type e.g. int.__call__() calls the int object constructor which creates a new instance of that type and initializes it with the inputs to __call__().
In short, there is nothing special or different about the common python "type conversions" (e.g. int(x), str(40)) other than that the int and str objects are included in __builtins__.
And to answer the original question, if type_name is an instance of the type class then the type_name.__call__() function simply declares and initializes a new instance of that type. Thus, one can simply do:
# convert x to type type_name
x = type_name(x)
however this may cause an exception if x is not a valid input to the type_name constructor.

To cast a value in another type you can use the type itself, you can pass the type as an argument and call it into a function and you can get it from the builtins module if you sure that the type is a builtin:
value = "1"
value = int(value) # set value to 1
value = 1
value = str(value) # set value to "1"
def cast(value, type_):
return type_(value)
import buitlins
def builtin_cast(value, type_):
type_ = getattr(buitlins, type_, None)
if isinstance(type_, type):
return type_(value)
raise ValueError(f"{type_!r} is not a builtins type.")
value = cast("1", int) # set value to 1
value = cast(1, str) # set value to "1"
value = builtin_cast("1", "int") # set value to 1
value = builtin_cast(1, "str") # set value to "1"

Programmatic generation of Literal options

MyPy's Literal type can be super useful for defining available options. Is it possible to generate a literal type programmatically, e.g. from a canonical registry?
e.g.
class Dispatcher():
func_reg = {
'f1': my_func,
'f2': new_func,
'f3': shoe_func,
}
def dispatch(cls, func_name: Literal[*func_reg.keys()]) -> Whatever:
pass

Unfortunately, the answer is no.
According to the mypy documentation:
Literal types may contain one or more literal bools, ints, strs, bytes, and enum values. However, literal types cannot contain arbitrary expressions: types like Literal[my_string.trim()], Literal[x > 3], or Literal[3j + 4] are all illegal.

As #BrokenBenchmark notes, it is not possible to auto-generate Literal types. However, if the end goal is just to require specific values generated from some kind of function registry, we can hack it with enum.Enum.
To quote PEP 586
rather than entirely special-casing enums, we can instead treat them
as being approximately equivalent to the union of their values...
the Status enum could be treated as being approximately equivalent to Literal[Status.SUCCESS, Status.INVALID_DATA, Status.FATAL_ERROR]
Here, functions are "registered" by adding an enum value that is an exact uppercasing of the function name to the FuncNames enum. This is not a pretty or robust solution, but it runs, it supports single-location registration of a function for type-checked dispatch, and mypy handles the required enum values as expected.
from enum import Enum, auto
def f():
return "f"
def g():
return "g"
def h():
return "h"
class Dispatcher():
# Build the enum used to register the functions
class FuncNames(Enum):
"""
The enum names here _must_ be exact uppercase-ings of the function
names. The names will be lowercased and evaluated to register their
associated functions
"""
F = auto()
G = auto()
H = auto()
# NOTE: The functional syntax works just as well
# FuncNames = Enum('FuncNames', 'F G H')
# Comprehensions can't access names defined in the class block,
# so use a standard for loop
func_reg = dict()
for name in list(FuncNames):
func_reg[eval(f"FuncNames.{name.name}")] = eval(str(name.name).lower())
#classmethod
def dispatch(cls, func_name: FuncNames):
"""
Prints the return from a registered function.
Can only be called with an item from FuncNames
"""
print(cls.func_reg[func_name]())
Dispatcher.dispatch(Dispatcher.FuncNames.F)
Dispatcher.dispatch(Dispatcher.FuncNames.G)
Dispatcher.dispatch(Dispatcher.FuncNames.H)
# Dispatcher.dispatch(Dispatcher.FuncNames.I) -> "FuncNames has no attribute I"
# Dispatcher.dispatch(Dispatcher2.FuncNames) -> "incompatible type"
# Dispatcher.dispatch('MyPy hates me!') -> "incompatible type"
Interestingly, though it feels cleaner to generate the enum itself from a list of the functions themselves, MyPy chokes on this.
class Dispatcher2():
# Build an enum used to register these (the actual functions)
funcs_to_register = [f, g, h]
enum_names = [func.__name__.upper() for func in funcs_to_register]
joined = ' '.join(enum_names)
FuncNames = Enum('FuncNames', joined)
func_reg = dict()
for name in enum_names:
func_reg[eval(f"FuncNames.{name}")] = eval(name.lower())
#classmethod
def dispatch(cls, func_name: FuncNames):
"""
Prints the return from a registered function.
Can only be called with an item from FuncNames
"""
print(cls.func_reg[func_name]())
Dispatcher2.dispatch(Dispatcher2.FuncNames.F)
Dispatcher2.dispatch(Dispatcher2.FuncNames.G)
Dispatcher2.dispatch(Dispatcher2.FuncNames.H)
The above runs as expected, but mypy presumably can't infer the values present in the enum unless it is statically defined, so it errors.
> mypy enums_typing.py
enums_typing.py:19: error: Enum() expects a string, tuple, list or dict literal as the second argument
enums_typing.py:36: error: "Type[FuncNames]" has no attribute "F"
enums_typing.py:37: error: "Type[FuncNames]" has no attribute "G"
enums_typing.py:38: error: "Type[FuncNames]" has no attribute "H"
Found 4 errors in 1 file (checked 1 source file)
TLDR:
In order to define a fixed set of choices that MyPy can check against, you must define them statically. It may then be possible to use those statically-defined choices to programmatically build your function registry.

strange error using AttrDict on Python 2.7

I am getting a weird recurring error using AttrDict 2.0 on Python 2.7. The weird part is that transitive assignment seems to break, but only when using AttrDict.
What's happening is that I want to instantiate a new list on an object if it doesn't exist and then append data to it.
If I use AttrDict, the list somehow gets transformed into a tuple and I get an exception.
from attrdict import AttrDict
class Test(object):
pass
try:
for cls_ in [Test,AttrDict]:
foo = cls_()
print ("\ntesting with class %s" % (cls_))
#this
chk = foo.li = getattr(foo, "li", None) or []
print(" type(chk):%s, id(chk):%s" % (type(chk),id(chk)))
print(" type(foo.li):%s, id(foo.li):%s" % (type(foo.li),id(foo.li)))
foo.li.append(3)
print (" success appending with class %s: foo.li:%s" % (cls_, foo.li))
except (Exception,) as e:
# pdb.set_trace()
raise
Now check out the output, when I use the Test class vs when I use AttrDict.
testing with class <class '__main__.Test'>
type(chk):<type 'list'>, id(chk):4465207704
type(foo.li):<type 'list'>, id(foo.li):4465207704
success appending with class <class '__main__.Test'>: foo.li:[3]
With the custom Test class, as expected, chk and foo.li are both lists and have the same id. append works.
Looking at the pass using AttrDict, id does not match and foo.li is a tuple rather than a list.
testing with class <class 'attrdict.dictionary.AttrDict'>
type(chk):<type 'list'>, id(chk):4465207848
type(foo.li):<type 'tuple'>, id(foo.li):4464595080
Traceback (most recent call last):
File "test_attrdict2.py", line 25, in <module>
test()
File "test_attrdict2.py", line 18, in test
foo.li.append(3)
AttributeError: 'tuple' object has no attribute 'append'
Is attrdict assignment actually returning some kind of property/accessor object that gets changed the 2nd time you access it?
Took #abartnet's suggestion:
from attrdict import AttrDict
a = AttrDict()
a.li = []
print(a.li)
output:
()
OK, but even if that points to some weird behavior on AttrDict's end, how is it the transitive assignment does not assign the tuple as well?
reworked:
from attrdict import AttrDict
a = AttrDict()
b = a.li = []
print("a.li:", a.li)
print("b:",b)
output:
('a.li:', ())
('b:', [])

This is part of the automatic recursiveness of AttrDict. Which is explained better in the inline help (which you can find here in the source) than in the README:
If a values which is accessed as an attribute is a Sequence-type (and is not a string/bytes), it will be converted to a _sequence_type with any mappings within it converted to Attrs.
In other words, in order to auto-convert any dict or other mappings recursively inside your AttrDict to AttrDict values when doing attribute access, it also converts all sequences to (by default) tuple. This is a little weird, but appears to be intentional and somewhat-documented behavior, not a bug.
>>> a = AttrDict()
>>> a._sequence_type
tuple
>>> a.li = []
>>> a.li
()
The more flexible AttrMap type lets you specify the sequence type, and documents that you can disable this recursive remapping stuff by passing None:
>>> a = AttrMap(sequence_type=None)
>>> a.li = []
>>> a.li
[]
But of course AttrMap isn't a dict (although it is a collections.abc.MutableMapping, and more generally it duck-types as a dict-like type).
OK, but even if that points to some weird behavior on AttrDict's end, how is it the transitive assignment does not assign the tuple as well?
Because that's not how chained assignment works. Oversimplifying a bit:
target1 = target2 = value
… is not equivalent to this:
target2 = value
target1 = target2
… but to this:
target2 = value
target1 = value
The best way to understand why that's true: targets aren't expressions, and therefore don't have values. Sure, often the exact same sequence of tokens would be valid as an expression elsewhere in the grammar, but that sequence of tokens never gets evaluated as an expression anywhere in an assignment statement—otherwise, simple things like d['spam'] = 'eggs' would have to raise an exception if d['spam'] didn't exist.
Also, a.li = [] doesn't actually assign tuple([]) anywhere; it actually stores the [] internally, and does the tuple(…) later, when you try to access a.li. You can't really tell that for sure without reading the source, but when you consider that a['li'] gives you [] rather than (), it pretty much has to be true. And, in fact:
>>> li = []
>>> a.li = li
>>> a['li'] is li
True

How to get back name of the enum element?

I have an enum defined like this:
def enum(**enums):
return type('Enum', (), enums)
Status = enum(
STATUS_OK=0,
STATUS_ERR_NULL_POINTER=1,
STATUS_ERR_INVALID_PARAMETER=2)
I have a function that returns status as Status enum.
How can I get the name of the enum value, and not just value?
>>> cur_status = get_Status()
>>> print(cur_status)
1
I would like to get STATUS_ERR_NULL_POINTER, instead of 1

You'd have to loop through the class attributes to find the matching name:
name = next(name for name, value in vars(Status).items() if value == 1)
The generator expression loops over the attributes and their values (taken from the dictionary produced by the vars() function) then returns the first one that matches the value 1.
Enumerations are better modelled by the enum library, available in Python 3.4 or as a backport for earlier versions:
from enum import Enum
class Status(Enum):
STATUS_OK = 0
STATUS_ERR_NULL_POINTER = 1
STATUS_ERR_INVALID_PARAMETER = 2
giving you access to the name and value:
name = Status(1).name # gives 'STATUS_ERR_NULL_POINTER'
value = Status.STATUS_ERR_NULL_POINTER.value # gives 1

2021 update:
These answers are out of date. Using Python's standard Enum class,
cur_status.name
will return the name. (STATUS_ERR_NULL_POINTER)
To look up the enum knowing the name:
s = Status['STATUS_ERR_NULL_POINTER']

Not sure which python version it was introduced, but the hidden attribute _value2member_map_ gives you what you want.
class Status(Enum):
STATUS_OK=0
STATUS_ERR_NULL_POINTER=1
STATUS_ERR_INVALID_PARAMETER=2
str(Status._value2member_map_[1])
Out:
'Status.STATUS_ERR_NULL_POINTER'

You don't need to loop through the Enum class but just access _member_map_.
>>> Status._member_map_['STATUS_OK']
<Status.STATUS_OK: 0>

For some reason, most of the methods above did not work for me. All methods return the Enum type as an integer. I'm working with Python 3.7.
In my solution, I defined class function to handle this. It's not purely pythonic, but worked well enough for my case.
from enum import Enum
class Status(Enum):
STATUS_OK = 0
STATUS_ERR_NULL_POINTER = 1
STATUS_ERR_INVALID_PARAMETER = 2
#classmethod
def name(cls,val):
return { v:k for k,v in dict(vars(cls)).items() if isinstance(v,int)}.get(val,None)
# test it
stat = Status.STATUS_OK
print(Status.name(stat))
Prints: 'STATUS_OK'
It may seem obvious that we asked for the status after giving it the status, but in my case, this is set programmatically elsewhere

How to convert int to Enum in python?

Using the new Enum feature (via backport enum34) with python 2.7.6.
Given the following definition, how can I convert an int to the corresponding Enum value?
from enum import Enum
class Fruit(Enum):
Apple = 4
Orange = 5
Pear = 6
I know I can hand craft a series of if-statements to do the conversion but is there an easy pythonic way to convert? Basically, I'd like a function ConvertIntToFruit(int) that returns an enum value.
My use case is I have a csv file of records where I'm reading each record into an object. One of the file fields is an integer field that represents an enumeration. As I'm populating the object I'd like to convert that integer field from the file into the corresponding Enum value in the object.

You 'call' the Enum class:
Fruit(5)
to turn 5 into Fruit.Orange:
>>> from enum import Enum
>>> class Fruit(Enum):
... Apple = 4
... Orange = 5
... Pear = 6
...
>>> Fruit(5)
<Fruit.Orange: 5>
From the Programmatic access to enumeration members and their attributes section of the documentation:
Sometimes it’s useful to access members in enumerations
programmatically (i.e. situations where Color.red won’t do because the
exact color is not known at program-writing time). Enum allows such
access:
>>> Color(1)
<Color.red: 1>
>>> Color(3)
<Color.blue: 3>
In a related note: to map a string value containing the name of an enum member, use subscription:
>>> s = 'Apple'
>>> Fruit[s]
<Fruit.Apple: 4>

I think it is in simple words is to convert the int value into Enum by calling EnumType(int_value), after that access the name of the Enum object:
my_fruit_from_int = Fruit(5) #convert to int
fruit_name = my_fruit_from_int.name #get the name
print(fruit_name) #Orange will be printed here
Or as a function:
def convert_int_to_fruit(int_value):
try:
my_fruit_from_int = Fruit(int_value)
return my_fruit_from_int.name
except:
return None

class Status(IntEnum):
UPLOADED = 1
DOWNLOADED = 5
SEGMENTED = 10
DIRECTED = 15
READYTODEEP = 20
to get the enum;
statusId = 5
Status(statusId)
to get the enum's string value;
statusId = 5
print(Status(statusId).name)

I wanted something similar so that I could access either part of the value pair from a single reference. The vanilla version:
#!/usr/bin/env python3
from enum import IntEnum
class EnumDemo(IntEnum):
ENUM_ZERO = 0
ENUM_ONE = 1
ENUM_TWO = 2
ENUM_THREE = 3
ENUM_INVALID = 4
#endclass.
print('Passes')
print('1) %d'%(EnumDemo['ENUM_TWO']))
print('2) %s'%(EnumDemo['ENUM_TWO']))
print('3) %s'%(EnumDemo.ENUM_TWO.name))
print('4) %d'%(EnumDemo.ENUM_TWO))
print()
print('Fails')
print('1) %d'%(EnumDemo.ENUM_TWOa))
The failure throws an exception as would be expected.
A more robust version:
#!/usr/bin/env python3
class EnumDemo():
enumeration = (
'ENUM_ZERO', # 0.
'ENUM_ONE', # 1.
'ENUM_TWO', # 2.
'ENUM_THREE', # 3.
'ENUM_INVALID' # 4.
)
def name(self, val):
try:
name = self.enumeration[val]
except IndexError:
# Always return last tuple.
name = self.enumeration[len(self.enumeration) - 1]
return name
def number(self, val):
try:
index = self.enumeration.index(val)
except (TypeError, ValueError):
# Always return last tuple.
index = (len(self.enumeration) - 1)
return index
#endclass.
print('Passes')
print('1) %d'%(EnumDemo().number('ENUM_TWO')))
print('2) %s'%(EnumDemo().number('ENUM_TWO')))
print('3) %s'%(EnumDemo().name(1)))
print('4) %s'%(EnumDemo().enumeration[1]))
print()
print('Fails')
print('1) %d'%(EnumDemo().number('ENUM_THREEa')))
print('2) %s'%(EnumDemo().number('ENUM_THREEa')))
print('3) %s'%(EnumDemo().name(11)))
print('4) %s'%(EnumDemo().enumeration[-1]))
When not used correctly this avoids creating an exception and, instead, passes back a fault indication. A more Pythonic way to do this would be to pass back "None" but my particular application uses the text directly.