In python, one can attribute some values to some of the keywords that are already predefined in python, unlike other languages. Why?
This is not all, some.
> range = 5
> range
> 5
But for
> def = 5
File "<stdin>", line 1
def = 5
^
SyntaxError: invalid syntax
One possible hypothesis is - Lazy coders with unique parsing rules.
For those new to python, yeah, this actually works, for keywords like True, False, range, len, so on.
I wrote a compiler for python in college and, if I remember correctly, the keywords list did not have them.
While range is nothing but a built-in function, def is a keyword. (Most IDEs should indicate the difference with appropriate colors.)
Functions - whether built-in or not - can be redefined. And they don't have to remain functions, but can become integers like range in your example. But you can never redefine keywords.
If you wish, you can print the list of all Python keywords with the following lines of code (borrowed from here):
import keyword
for keyword in keyword.kwlist:
print keyword
Output:
and
as
assert
break
class
continue
def
del
elif
else
except
exec
finally
for
from
global
if
import
in
is
lambda
not
or
pass
print
raise
return
try
while
with
yield
And for Python 3 (notice the absence of print):
False
None
True
and
as
assert
break
class
continue
def
del
elif
else
except
finally
for
from
global
if
import
in
is
lambda
nonlocal
not
or
pass
raise
return
try
while
with
yield
In contrast, the built-in functions can be found here: https://docs.python.org/2/library/functions.html
The keyword 'range' is a function, you can create some other vars as well as sum, max...
In the other hand, keyword 'def' expects a defined structure in order to create a function.
def <functionName>(args):
You are confused between keywords and built-in functions. def is a keyword, but range and len are simply built-in functions. Any function can always be overridden, but a keyword cannot.
The full list of keywords can be found in keywords.kwlist.
Related
I understand that you can use "pass" in a user defined function, say, when you do not want to put anything in it at the moment. Where else can I possibly use this keyword? When assigning it to a variable for example, I get an error: my_item = pass Why is this happening and where else can I use this keyword at?
pass is, by itself, an entire statement, and can be used anywhere a statement is expected, but almost always are found in the body of a compound statement.
def foo():
pass
if True:
pass
for x in y:
pass
class Bar:
pass
with foo as bar:
pass
etc.
It is typically used where some statement is needed to fill an otherwise empty code block. Note that since any expression can be used as a statement (a so-called expression statement), you can sometimes use a doc string in place of a pass statement:
class FooError(ValueError):
'''A very specific kind of ValueError'''
instead of
# Legal, but unnecessary
class FooError(ValueError):
'''A very specific kind of ValueError'''
pass
You can only place an expression, not a statement, on the right-hand side of an assignment statement, which is why my_item = pass is a syntax error.
How can I validate that a function includes a return keyword? I frequently forget the return line, so I am worried that the users of my package will too when they provide a function-based input.
def appler():
a = "apple"
# `return` is missing
def bananer():
b = "banana"
return b
I could parse the actual code string of the function for a final line that includes "return" but that isn't very robust (it could be triggered by comments).
def validate_funk(funk):
if condition_to_check_that_it_contains_rtrn:
pass
else:
raise ValueError(f"Yikes - The function you provided not contain a `return` statement:\n\n{funk}")
>>> validate_funk(appler)
#triggers ValueError
>>> validate_funk(bananer)
# passes
EDIT: ideally without running the function.
What you actually care about is probably not the return statement itself, but that the function returns something of a certain type. This you can most easily accomplish by using type hints (PEP 484):
def appler() -> str:
a = "apple"
# `return` is missing
def bananer() -> str:
b = "banana"
return b
Now, running a static analysis tool like mypy or Pyre (or many others): will emit a warning about a wrong return type in function appler (expected str, got NoneType).
Look for sabik's answer for a more general answer. Writing (unit) tests is another good practice that catches many more issues and - if done well - is an invest in code maintainability.
A function without return statement returns None by default.
>>> def abc():
pass
>>> print(abc())
None
>>>
You can add a check using this:
def validate_func(function):
if function() == None:
raise ValueError("Yikes - Does not contain a `return` statement")
There are few cons though.
You have to execute the function
It wont work if you are returning None in a function
Not much practical but yea, that is one way. You can also get a list of local functions or a list of methods in a class and loop through them without having to check each function individually.
For the question as asked, the ast module will let you check that.
However, it doesn't seem very useful just by itself - as others have pointed out, a function without a return is valid (it returns None), and just because a function does have a return doesn't mean that it returns the correct value, or even any value (the return could be in an if statement).
There are a couple of standard ways of dealing with this:
Unit tests - separate code that calls your function with various combinations of inputs (possibly just one, possibly hundreds) and checks that the answers match the ones you calculated manually, or otherwise satisfy requirements.
A more general implementation of the idea of checking for a return statement is "lint", in the case of Python pylint; that looks through your code and checks for various patterns that look like they could be mistakes. A side benefit is that it already exists and it checks dozens of common patterns.
Another, different more general implementation is the mypy type checker; that not only checks that there's a return statement, but also that it returns the correct type, as annotated in the header of the function.
Typically, these would be used together with a "gated trunk" development process; manual changes to the main version are forbidden, and only changes which pass the tests, lint and/or mypy are accepted into the main version.
As others have mentioned, simply calling the function is not enough: a return statement might only be present in a conditional, and thus, specific input would need to be passed in order to execute the return statement. That, too, is not a good indicator of the presence of a return, since it could return None, causing greater ambiguity. Instead, the inspect and ast module can be used:
Test functions:
def appler():
a = "apple"
# `return` is missing
def bananer():
b = "banana"
return b
def deeper_test(val, val1):
if val and val1:
if val+val1 == 10:
return
def gen_func(v):
for i in v:
if isinstance(i, list):
yield from gen_func(i)
else:
yield i
inspect.getsource returns the entire source of the function as a string, which can then be passed to ast.parse. From there, the syntax tree can be recursively traversed, searching for the presence of a return statement:
import inspect, ast
fs = [appler, bananer, deeper_test, gen_func]
def has_return(f_obj):
return isinstance(f_obj, ast.Return) or \
any(has_return(i) for i in getattr(f_obj, 'body', []))
result = {i.__name__:has_return(ast.parse(inspect.getsource(i))) for i in fs}
Output:
{'appler': False, 'bananer': True, 'deeper_test': True, 'gen_func': False}
With a defined validate_funk:
def validate_funk(f):
if not has_return(ast.parse(inspect.getsource(f))):
raise ValueError(f"function '{f.__name__}' does not contain a `return` statement")
return True
Notes:
This solution does not require the test functions to be called.
The solution must be run in a file. If it is run in the shell, an OSError will be raised. For the file, see this Github Gist.
You can simplify return checking with a decorator:
def ensure_return(func):
def wrapper(*args, **kwargs):
res = func(*args, **kwargs)
if res is None:
raise ValueError(f'{func} did not return a value')
return res
return wrapper
#ensure_return
def appler():
a = "apple"
# `return` is missing
#ensure_return
def bananer():
b = "banana"
return b
then:
>>> appler()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 5, in wrapper
ValueError: <function appler at 0x7f99d1a01160> did not return a value
>>> bananer()
'banana'
There's a notion of keywords in Clojure where you define them by adding a colon in front of your word you are trying to address as a keyword. Also, it evaluates to itself. For example:
:my-keyword
;=> :my-keyword
Is there any way to implement this in python by defining some custom class or any workarounds?
The reason for having this is to have more self-desctriptive parameters (strings are there, but one cannot keep track of having consistent strings while passing around).
A practical use case for this goes something like this:
def area(polygon_type):
return \
{
"square": lambda side: (side * side),
"triangle": lambda base, height: (0.5 * base * height)
}[polygon_type]
area("square")(2) # ==> 4
But, having strings in such manner leads to error at runtime, if mishandled. But having something like keywords even an auto-complete feature in any IDE suggests the mistake that has been made while passing in the polygon_type.
area("Sqaure")(2) # would lead to a KeyError
Is there some feature in python that solves this type of problem, that I am unaware of?
If not, how'd someone go about tackling this?
Edit:
I am not trying to solve the problem of having such a function in particular; but instead looking for a way of implementing keyword concept in python. As, with enums I have to bundle up and explicitly define them under some category (In this case polygon_type)
Keywords in Clojure are interned strings and Clojure provides special syntactic support for them. I suggest you take a look at how they are implemented. It seems like Python does some interning of its strings but I don't know much of its details.
The point of using keyword is fast comparisons and map lookup. Although I am not sure how you would benefit from it, you could try to implement your own keyword-like objects in Python using string interning, something like this:
str2kwd = {}
class Keyword:
def __init__(self, s):
self.s = s
def __repr__(self):
return str(self)
def __str__(self):
return ":" + self.s
def kwd(s):
"""Construct a keyword"""
k = str2kwd.get(s)
if k is None:
k = Keyword(s)
str2kwd[s] = k
return k
Whenever you want to construct a keyword, you call the kwd function. For the Keyword class, we rely on the default equality and hash methods. Then you could use it like this:
>>> kwd("a")
:a
>>> kwd("a") == kwd("a")
True
>>> kwd("b") == kwd("a")
False
>>> kwd_a = kwd("a")
>>> kwd_b = kwd("b")
>>> {kwd_a: 3, kwd_b: 4}
{:a: 3, :b: 4}
>>> {kwd_a: 3, kwd_b: 4}[kwd_a]
3
However, I have not measured if this results in faster comparisons and map-lookups than just using regular Python strings, which is probably the most idiomatic choice for Python anyway. I doubt you would see a significant improvement in performance from using this home-made keyword class. Also note that it is best to call the kwd function at the top-level of the module and assign it to a variable that you use, instead of calling kwd everytime you need a keyword. Obviously, you will not have the special keyword syntax as in Clojure.
UPDATE: How to avoid misspelling bugs
If you are worried about misspelling keys in your map, you can assign the keys to local variables and use those local variables instead of the key values directly. This way, if you misspell a local variable name you will likely get an error much sooner because you are referring to a local variable that does not exist.
>>> kwd_square = "square"
>>> kwd_triangle = "triangle"
>>> m = {kwd_square: 3, kwd_triangle: 4}
>>> m[kwd_square]
3
>>> m[Square]
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
NameError: name 'Square' is not defined
This question already has an answer here:
Why is it a syntax error to have an object attribute named "del", "return" etc?
(1 answer)
Closed 7 years ago.
I may have stumbled on an illegal variable name
pass = "Pass the monkey!"
print pass
Syntax error: invalid syntax
I'm aware that some keywords are verboten as variables.
Is there the Pythonic equivalent to JavaScript variable name validator?
You can test whether something is a keyword or not using the keyword module
>>> import keyword
>>> keyword.iskeyword("pass")
True
>>> keyword.iskeyword("not_pass")
False
https://docs.python.org/2/library/keyword.html
This module allows a Python program to determine if a string is a
keyword.
keyword.iskeyword(s)
Return true if s is a Python keyword.
Some variable names are illegal in Python because of it being a reserved word.
From the keywords section in the Python docs:
The following identifiers are used as reserved words, or keywords of
the language, and cannot be used as ordinary identifiers. They
must be spelled exactly as written here:
# Complete list of reserved words
and
del
from
not
while
as
elif
global
or
with
assert
else
if
pass
yield
break
except
import
print
class
exec
in
raise
continue
finally
is
return
def
for
lambda
try
True # Python 3 onwards
False # Python 3 onwards
None # Python 3 onwards
nonlocal # Python 3 onwards
async # in Python 3.7
await # in Python 3.7
So, you cannot use any of the above identifiers as a variable name.
This function will check if a name is a keyword in Python or one of Python built-in objects, which can be a function, a constant, a type or an exception class.
import keyword
def is_keyword_or_builtin(name):
return keyword.iskeyword(name) or name in dir(__builtins__)
While you can't use Python keywords as variable names, you are allowed to do it with Python built-ins though it's considered a bad practice so I will recommend to avoid it.
>>> list=[None]
>>> def list[0](x,y):
File "<stdin>", line 1
def list[0](x,y):
^
SyntaxError: invalid syntax
How can I define a function as an element of a list?
Python's def isn't flexible enough to handle generic lvalues such as list[0]. The language only allows you to use an identifier as function name. Here are the relevant parts of the grammar rule for the def-statement:
funcdef ::= "def" funcname "(" [parameter_list] ")" ":" suite
funcname ::= identifier
Instead, you can use a series of assignment and definition statements:
s = [None]
def f(x, y):
return x + y
s[0] = f
As an alternative, you could also store a lambda expression directly in a list:
s = [lambda x,y : x+y]
def f(whatever):
do_stuff()
l[0] = f
The function definition syntax doesn't allow you to define a function directly into a data structure, but you can just create the function and then assign it wherever it needs to go.
def someFunctionA(x, y):
return x+y
def someFunctionB(x, y):
return x*y
someList = [someFunctionA, someFunctionB]
print someList[0](2, 3)
print someList[1](5, 5)
Allowing such a freedom would make parsing harder... for example the parenthesized expression
...(x, y, z=3)
can be either a parameter declaration (where 3 is the default for keyword parameter z) or a call (that is passing z keyword parameter value as 3).
If you want to allow a generic assignable expression in def you also need to allow
def foo(x, y, z=3)[3](x, y, z=3):
...
where the first parenthesized part has a different semantic meaning and syntax rules from the second part.
Writing a parser for this is annoying (basically because you need to process an arbitrary unbounded amount of source code without understanding it) and is what for example lead to the worst parsing rule in the whole universe I know (it's the dreaded most vexing parse of C++) that basically just gave up on trying to get a decent language by resigning to ambiguity.
Note that in many cases when it's harder for a program to do the parsing it's because of ambiguity that would make also harder for a human to understand it.
Python correctly values readability as very important.
Functions in Python are however first class objects so you can solve your problem easily enough:
def foo(...):
...
mylist[index] = foo
or, only if the function is a single expression, with
mylist[index] = lambda ... : ...
(but lambda is very limited, both because it's sort of "hated" in the Python community and also because it would create some annoyance at the syntax level because of the need of handling indentation inside parenthesis).
Note also that something that a few Python novices don't know is that you can use def even inside a function; for example:
def register_http():
def handle_http(connection):
...
global_register['http'] = handle_http
that will assign a function as element of a global map without polluting the global (module) namespace with its name. A local def can also create a closure by capturing local state variables (read-only in 2.x or even read/write in 3.x).
Note also that if you need some processing of a function may be decorators can be useful. For example by defining
def register(name):
def do_registering(f):
global_register[name] = f
return f
return do_registering
you can just use
#register('http')
def handle_http(connection):
...