I see
df["col2"] = df["col1"].apply(len)
len(df["col1"])
My question is,
Why use "len" function without parenthesis in 1, but use it with parenthesis in 2?
What is the difference between the two?
I see this kind of occasion a lot, where using a function with and without parenthesis.
Can someone explain to me what exactly is going on?
Thanks.
The first example that you mentioned(the above code) maps the function len to the target variable df["col1"]
df["col2"] = df["col1"].apply(len)
Whenever we have to map a function to any iterable object, the syntax needs the function to be without parenthesis.
In your case, df["col1"] must be having elements whose length can be calculated. And it will return a Pandas Series will lengths of all the elements.
Take the following example.
a = ["1", "2","3","4"]
z = list( map( int, a ) ) >> [1, 2, 3, 4]
Here, we mapped the builtin int function(which does typecasting), to the entire list.
The second example that you mentioned would give out the length of the df["col1"] series.
len(df["col1"])
It won't do any operations on the elements within that Series.
Take the following example.
a = ["1", "2","3","4"]
z = len(a) >> 4
Since, on both the occasions, the function len was fed an iterable object, it didn't give any error. But, the outputs are completely different as I explained!
In 1, the function len is being passed to a method called apply. That method presumably will apply the function len along the first axis (probably returning something like a list of lengths). In 2, the function len is being called directly, with an argument df["col2"], presumably to get the length of the data frame.
The use in 1 is sometimes called a "higher order function", but in principle it's just passing a function to another function for it to use.
In the second case you are directly calling the len method and will get the result, i.e. how many rows are in col1 in the df.
In the first you are giving the reference to the len function to the apply function.
This is a shortcut for df["col2"] = df["col1"].apply(lambda x: len(x))
This version you use if you want to make the behavior of a method flexible by letting the user of the method hand in the function to influence some part of an algorithm. Like here in the case with the apply method. Depending of the conents in the column you want to fill the new column with something, and here it was decided to fill this with the lengths of the content of other column.
len(s) will return the lenght of the s variable
len will return the function itslelf. So if I do a=len, then I can do a(s). Of course, it is not recommended to do such thing as a=len.
Let's have a look at the documentation of DataFrame.apply:
its first parameter is func: function which is a function that we'll apply to each column or row of the DataFrame. In your case this function is len().
Now let's see what happens when you pass len as a parameter with parenthesis:
df.apply(len())
-----------------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/tmp/ipykernel_11920/3211016940.py in <module>
----> 1 df.apply(len())
TypeError: len() takes exactly one argument (0 given)
While this perfectly works when we use df.apply(len).
This is because your parameter must be a function and the way Python uses to distinguish between functions and the return value of the call to a function is the use of parenthesis in the second case.
Hope you're all well with the caotic world we're living...
This might be a very beginner level question, but I'd like to understand why It is like that.
Let's say I have a list of complex:
myList = [(1.231 +2.254j), (2.875 +23.543j), ...]
I've been trying to round the values with this function:
def round_complex(x, digits):
return complex(round(x.real, digits), round(x.imag, digits))
And for doing so, I've tried this:
for item in myList:
item = round_complex(item, 2)
Expecting that myList values get changed, for example:
myList = [(1.23 +2.25j), (2.88 +23.54j), ...]
But, It does not work.
I've also tried with a more simple example, like a list of floats and the base round function from python. It also does not work.
Is there a way for me to change a value of an iterable object with this kind of for loop (for-in)?
Or do I really have to do this:
for i in range(len(myList)):
myList[i] = round_complex(myList[i], 2)
The simple answer is: NO.
Python uses a mechanism, which is known as "Call-by-Object", sometimes also called "Call by Object Reference" or "Call by Sharing" when pass function parameters.
If you pass immutable arguments like integers, strings or tuples to a function, the passing acts like call-by-value. The object reference is passed to the function parameters. They can't be changed within the function, because they can't be changed at all, i.e. they are immutable. It's different, if we pass mutable arguments. They are also passed by object reference, but they can be changed in place within the function.
So, after your iterate the list, the value (1.231 +2.254j) would be a immutable argument which your change won't affect the outside variable. But if you pass the value like [1.231 +2.254j] to function, then it will make effect like next:
test.py:
myList2 = [[(1.231 +2.254j)], [(2.875 +23.543j)]]
print(myList2)
def round_complex(x, digits):
return complex(round(x.real, digits), round(x.imag, digits))
for item2 in myList2:
item2[0] = round_complex(item2[0], 2)
print(myList2)
Execution:
$ python3 test.py
[[(1.231+2.254j)], [(2.875+23.543j)]]
[[(1.23+2.25j)], [(2.88+23.54j)]]
In a word, for you scenario, if you insist organize your input data as that & iterate with that way, you can't change the outside value directly inside the function.
You may refers to this to learn more.
The thing I understand by reading for question that you want to Assign the values of i in myList. For doing so you can use append i.e
for i in mylist:
mylist.append(round_complex(i, 2))
Sometimes I find myself writing code like this:
def analyse(somedata):
result = bestapproach(somedata)
if result:
return result
else:
result = notasgood(somedata)
if result:
return result
else:
result = betterthannothing(somedata)
if result:
return result
else:
return None
That's pretty ugly. Of course, some people like to leave off some of that syntax:
def analyse(somedata):
result = bestapproach(somedata)
if result:
return result
result = notasgood(somedata)
if result:
return result
result = betterthannothing(somedata)
if result:
return result
But that's not much of an improvement; there's still a ton of duplicated code here, and it's still ugly.
I looked into using the built-in iter() with a sentinel value, but in this case the None value is being used to signal that the loop should keep going, as opposed to a sentinel which is used to signal that the loop should terminate.
Are there any other (sane) techniques in Python for implementing this sort of "keep trying until you find something that works" pattern?
I should clarify that "return value meets some condition" is not limited to cases where the condition is if bool(result) is True as in the example. It could be that the list of possible analysis functions each produce some coefficient measuring the degree of success (e.g. an R-squared value), and you want to set a minimum threshold for acceptance. Therefore, a general solution should not inherently rely on the truth value of the result.
Option #1: Using or
When the number of total functions is a) known, and b) small, and the test condition is based entirely on the truth value of the return, it's possible to simply use or as Grapsus suggested:
d = 'somedata'
result = f1(d) or f2(d) or f3(d) or f4(d)
Because Python's boolean operators short-circuit, the functions are executed from right to left until one of them produces a return value evaluated as True, at which point the assignment is made to result and the remaining functions are not evaluated; or until you run out of functions, and result is assigned False.
Option #2: Using generators
When the number of total functions is a) unknown, or b) very large, a one-liner generator comprehension method works, as Bitwise suggested:
result = (r for r in (f(somedata) for f in functions) if <test-condition>).next()
This has the additional advantage over option #1 that you can use any <test-condition> you wish, instead of relying only on truth value. Each time .next() is called:
We ask the outer generator for its next element
The outer generator asks the inner generator for its next element
The inner generator asks for an f from functions and tries to evaluate f(somedata)
If the expression can be evaluated (i.e., f is a function and somedata is a valid arugment), the inner generator yields the return value of f(somedata) to the outer generator
If <test-condition> is satisfied, the outer generator yields the return value of f(somedata) and we assign it as result
If <test-condition> was not satisfied in step 5, repeat steps 2-4
A weakness of this method is that nested comprehensions can be less intuitive than their multi-line equivalents. Also, if the inner generator is exhausted without ever satisfying the test condition, .next() raises a StopIteration which must be handled (in a try-except block) or prevented (by ensuring the last function will always "succeed").
Option #3: Using a custom function
Since we can place callable functions in a list, one option is to explicitly list the functions you want to "try" in the order they should be used, and then iterate through that list:
def analyse(somedata):
analysis_functions = [best, okay, poor]
for f in analysis_functions:
result = f(somedata)
if result:
return result
Advantages: Fixes the problem of repeated code, it's more clear that you're engaged in an iterative process, and it short-circuits (doesn't continue executing functions after it finds a "good" result).
This could also be written with Python's for ... else syntax:*
def analyse(somedata):
analysis_functions = [best, okay, poor]
for f in analysis_functions:
result = f(somedata)
if result:
break
else:
return None
return result
The advantage here is that the different ways to exit the function are identified, which could be useful if you want complete failure of the analyse() function to return something other than None, or to raise an exception. Otherwise, it's just longer and more esoteric.
*As described in "Transforming Code into Beautiful, Idiomatic Python", starting #15:50.
If the number of functions is not too high, why not use the or operator ?
d = 'somedata'
result = f1(d) or f2(d) or f3(d) or f4(d)
It will only apply the functions until one of them returns something not False.
This is pretty pythonic:
result = (i for i in (f(somedata) for f in funcs) if i is not None).next()
The idea is to use generators so you do lazy evaluation instead of evaluating all functions. Note that you can change the condition/funcs to be whatever you like, so this is more robust than the or solution proposed by Grapsus.
This is a good example why generators are powerful in Python.
A more detailed description of how this works:
We ask this generator for a single element. The outer generator then asks the inner generator (f(d) for f in funcs) for a single element, and evaluates it. If it passes the condition then we are done and it exits, otherwise it continues asking the inner generator for elements.
I have a situation in Python(cough, homework) where I need to multiply EACH ELEMENT in a given list of objects a specified number of times and return the output of the elements. The problem is that the sample inputs given are of different types. For example, one case may input a list of strings whose elements I need to multiply while the others may be ints. So my return type needs to vary. I would like to do this without having to test what every type of object is. Is there a way to do this? I know in C# i could just use "var" but I don't know if such a thing exists in Python?
I realize that variables don't have to be declared, but in this case I can't see any way around it. Here's the function I made:
def multiplyItemsByFour(argsList):
output = ????
for arg in argsList:
output += arg * 4
return output
See how I need to add to the output variable. If I just try to take away the output assignment on the first line, I get an error that the variable was not defined. But if I assign it a 0 or a "" for an empty string, an exception could be thrown since you can't add 3 to a string or "a" to an integer, etc...
Here are some sample inputs and outputs:
Input: ('a','b') Output: 'aaaabbbb'
Input: (2,3,4) Output: 36
Thanks!
def fivetimes(anylist):
return anylist * 5
As you see, if you're given a list argument, there's no need for any assignment whatsoever in order to "multiply it a given number of times and return the output". You talk about a given list; how is it given to you, if not (the most natural way) as an argument to your function? Not that it matters much -- if it's a global variable, a property of the object that's your argument, and so forth, this still doesn't necessitate any assignment.
If you were "homeworkically" forbidden from using the * operator of lists, and just required to implement it yourself, this would require assignment, but no declaration:
def multiply_the_hard_way(inputlist, multiplier):
outputlist = []
for i in range(multiplier):
outputlist.extend(inputlist)
return outputlist
You can simply make the empty list "magicaly appear": there's no need to "declare" it as being anything whatsoever, it's an empty list and the Python compiler knows it as well as you or any reader of your code does. Binding it to the name outputlist doesn't require you to perform any special ritual either, just the binding (aka assignment) itself: names don't have types, only objects have types... that's Python!-)
Edit: OP now says output must not be a list, but rather int, float, or maybe string, and he is given no indication of what. I've asked for clarification -- multiplying a list ALWAYS returns a list, so clearly he must mean something different from what he originally said, that he had to multiply a list. Meanwhile, here's another attempt at mind-reading. Perhaps he must return a list where EACH ITEM of the input list is multiplied by the same factor (whether that item is an int, float, string, list, ...). Well then:
define multiply_each_item(somelist, multiplier):
return [item * multiplier for item in somelist]
Look ma, no hands^H^H^H^H^H assignment. (This is known as a "list comprehension", btw).
Or maybe (unlikely, but my mind-reading hat may be suffering interference from my tinfoil hat, will need to go to the mad hatter's shop to have them tuned) he needs to (say) multiply each list item as if they were the same type as the first item, but return them as their original type, so that for example
>>> mystic(['zap', 1, 23, 'goo'], 2)
['zapzap', 11, 2323, 'googoo']
>>> mystic([23, '12', 15, 2.5], 2)
[46, '24', 30, 4.0]
Even this highly-mystical spec COULD be accomodated...:
>>> def mystic(alist, mul):
... multyp = type(alist[0])
... return [type(x)(mul*multyp(x)) for x in alist]
...
...though I very much doubt it's the spec actually encoded in the mysterious runes of that homework assignment. Just about ANY precise spec can be either implemented or proven to be likely impossible as stated (by requiring you to solve the Halting Problem or demanding that P==NP, say;-). That may take some work ("prove the 4-color theorem", for example;-)... but still less than it takes to magically divine what the actual spec IS, from a collection of mutually contradictory observations, no examples, etc. Though in our daily work as software developer (ah for the good old times when all we had to face was homework!-) we DO meet a lot of such cases of course (and have to solve them to earn our daily bread;-).
EditEdit: finally seeing a precise spec I point out I already implemented that one, anyway, here it goes again:
def multiplyItemsByFour(argsList):
return [item * 4 for item in argsList]
EditEditEdit: finally/finally seeing a MORE precise spec, with (luxury!-) examples:
Input: ('a','b') Output: 'aaaabbbb' Input: (2,3,4) Output: 36
So then what's wanted it the summation (and you can't use sum as it wouldn't work on strings) of the items in the input list, each multiplied by four. My preferred solution:
def theFinalAndTrulyRealProblemAsPosed(argsList):
items = iter(argsList)
output = next(items, []) * 4
for item in items:
output += item * 4
return output
If you're forbidden from using some of these constructs, such as built-ins items and iter, there are many other possibilities (slightly inferior ones) such as:
def theFinalAndTrulyRealProblemAsPosed(argsList):
if not argsList: return None
output = argsList[0] * 4
for item in argsList[1:]:
output += item * 4
return output
For an empty argsList, the first version returns [], the second one returns None -- not sure what you're supposed to do in that corner case anyway.
Very easy in Python. You need to get the type of the data in your list - use the type() function on the first item - type(argsList[0]). Then to initialize output (where you now have ????) you need the 'zero' or nul value for that type. So just as int() or float() or str() returns the zero or nul for their type so to will type(argsList[0])() return the zero or nul value for whatever type you have in your list.
So, here is your function with one minor modification:
def multiplyItemsByFour(argsList):
output = type(argsList[0])()
for arg in argsList:
output += arg * 4
return output
Works with::
argsList = [1, 2, 3, 4] or [1.0, 2.0, 3.0, 4.0] or "abcdef" ... etc,
Are you sure this is for Python beginners? To me, the cleanest way to do this is with reduce() and lambda, both of which are not typical beginner tools, and sometimes discouraged even for experienced Python programmers:
def multiplyItemsByFour(argsList):
if not argsList:
return None
newItems = [item * 4 for item in argsList]
return reduce(lambda x, y: x + y, newItems)
Like Alex Martelli, I've thrown in a quick test for an empty list at the beginning which returns None. Note that if you are using Python 3, you must import functools to use reduce().
Essentially, the reduce(lambda...) solution is very similar to the other suggestions to set up an accumulator using the first input item, and then processing the rest of the input items; but is simply more concise.
My guess is that the purpose of your homework is to expose you to "duck typing". The basic idea is that you don't worry about the types too much, you just worry about whether the behaviors work correctly. A classic example:
def add_two(a, b):
return a + b
print add_two(1, 2) # prints 3
print add_two("foo", "bar") # prints "foobar"
print add_two([0, 1, 2], [3, 4, 5]) # prints [0, 1, 2, 3, 4, 5]
Notice that when you def a function in Python, you don't declare a return type anywhere. It is perfectly okay for the same function to return different types based on its arguments. It's considered a virtue, even; consider that in Python we only need one definition of add_two() and we can add integers, add floats, concatenate strings, and join lists with it. Statically typed languages would require multiple implementations, unless they had an escape such as variant, but Python is dynamically typed. (Python is strongly typed, but dynamically typed. Some will tell you Python is weakly typed, but it isn't. In a weakly typed language such as JavaScript, the expression 1 + "1" will give you a result of 2; in Python this expression just raises a TypeError exception.)
It is considered very poor style to try to test the arguments to figure out their types, and then do things based on the types. If you need to make your code robust, you can always use a try block:
def safe_add_two(a, b):
try:
return a + b
except TypeError:
return None
See also the Wikipedia page on duck typing.
Python is dynamically typed, you don't need to declare the type of a variable, because a variable doesn't have a type, only values do. (Any variable can store any value, a value never changes its type during its lifetime.)
def do_something(x):
return x * 5
This will work for any x you pass to it, the actual result depending on what type the value in x has. If x contains a number it will just do regular multiplication, if it contains a string the string will be repeated five times in a row, for lists and such it will repeat the list five times, and so on. For custom types (classes) it depends on whether the class has an operation defined for the multiplication operator.
You don't need to declare variable types in python; a variable has the type of whatever's assigned to it.
EDIT:
To solve the re-stated problem, try this:
def multiplyItemsByFour(argsList):
output = argsList.pop(0) * 4
for arg in argsList:
output += arg * 4
return output
(This is probably not the most pythonic way of doing this, but it should at least start off your output variable as the right type, assuming the whole list is of the same type)
You gave these sample inputs and outputs:
Input: ('a','b') Output: 'aaaabbbb' Input: (2,3,4) Output: 36
I don't want to write the solution to your homework for you, but I do want to steer you in the correct direction. But I'm still not sure I understand what your problem is, because the problem as I understand it seems a bit difficult for an intro to Python class.
The most straightforward way to solve this requires that the arguments be passed in a list. Then, you can look at the first item in the list, and work from that. Here is a function that requires the caller to pass in a list of two items:
def handle_list_of_len_2(lst):
return lst[0] * 4 + lst[1] * 4
Now, how can we make this extend past two items? Well, in your sample code you weren't sure what to assign to your variable output. How about assigning lst[0]? Then it always has the correct type. Then you could loop over all the other elements in lst and accumulate to your output variable using += as you wrote. If you don't know how to loop over a list of items but skip the first thing in the list, Google search for "python list slice".
Now, how can we make this not require the user to pack up everything into a list, but just call the function? What we really want is some way to accept whatever arguments the user wants to pass to the function, and make a list out of them. Perhaps there is special syntax for declaring a function where you tell Python you just want the arguments bundled up into a list. You might check a good tutorial and see what it says about how to define a function.
Now that we have covered (very generally) how to accumulate an answer using +=, let's consider other ways to accumulate an answer. If you know how to use a list comprehension, you could use one of those to return a new list based on the argument list, with the multiply performed on each argument; you could then somehow reduce the list down to a single item and return it. Python 2.3 and newer have a built-in function called sum() and you might want to read up on that. [EDIT: Oh drat, sum() only works on numbers. See note added at end.]
I hope this helps. If you are still very confused, I suggest you contact your teacher and ask for clarification. Good luck.
P.S. Python 2.x have a built-in function called reduce() and it is possible to implement sum() using reduce(). However, the creator of Python thinks it is better to just use sum() and in fact he removed reduce() from Python 3.0 (well, he moved it into a module called functools).
P.P.S. If you get the list comprehension working, here's one more thing to think about. If you use a list comprehension and then pass the result to sum(), you build a list to be used once and then discarded. Wouldn't it be neat if we could get the result, but instead of building the whole list and then discarding it we could just have the sum() function consume the list items as fast as they are generated? You might want to read this: Generator Expressions vs. List Comprehension
EDIT: Oh drat, I assumed that Python's sum() builtin would use duck typing. Actually it is documented to work on numbers, only. I'm disappointed! I'll have to search and see if there were any discussions about that, and see why they did it the way they did; they probably had good reasons. Meanwhile, you might as well use your += solution. Sorry about that.
EDIT: Okay, reading through other answers, I now notice two ways suggested for peeling off the first element in the list.
For simplicity, because you seem like a Python beginner, I suggested simply using output = lst[0] and then using list slicing to skip past the first item in the list. However, Wooble in his answer suggested using output = lst.pop(0) which is a very clean solution: it gets the zeroth thing on the list, and then you can just loop over the list and you automatically skip the zeroth thing. However, this "mutates" the list! It's better if a function like this does not have "side effects" such as modifying the list passed to it. (Unless the list is a special list made just for that function call, such as a *args list.) Another way would be to use the "list slice" trick to make a copy of the list that has the first item removed. Alex Martelli provided an example of how to make an "iterator" using a Python feature called iter(), and then using iterator to get the "next" thing. Since the iterator hasn't been used yet, the next thing is the zeroth thing in the list. That's not really a beginner solution but it is the most elegant way to do this in Python; you could pass a really huge list to the function, and Alex Martelli's solution will neither mutate the list nor waste memory by making a copy of the list.
No need to test the objects, just multiply away!
'this is a string' * 6
14 * 6
[1,2,3] * 6
all just work
Try this:
def timesfourlist(list):
nextstep = map(times_four, list)
sum(nextstep)
map performs the function passed in on each element of the list(returning a new list) and then sum does the += on the list.
If you just want to fill in the blank in your code, you could try setting object=arglist[0].__class__() to give it the zero equivalent value of that class.
>>> def multiplyItemsByFour(argsList):
output = argsList[0].__class__()
for arg in argsList:
output += arg * 4
return output
>>> multiplyItemsByFour('ab')
'aaaabbbb'
>>> multiplyItemsByFour((2,3,4))
36
>>> multiplyItemsByFour((2.0,3.3))
21.199999999999999
This will crash if the list is empty, but you can check for that case at the beginning of the function and return whatever you feel appropriate.
Thanks to Alex Martelli, you have the best possible solution:
def theFinalAndTrulyRealProblemAsPosed(argsList):
items = iter(argsList)
output = next(items, []) * 4
for item in items:
output += item * 4
return output
This is beautiful and elegant. First we create an iterator with iter(), then we use next() to get the first object in the list. Then we accumulate as we iterate through the rest of the list, and we are done. We never need to know the type of the objects in argsList, and indeed they can be of different types as long as all the types can have operator + applied with them. This is duck typing.
For a moment there last night I was confused and thought that you wanted a function that, instead of taking an explicit list, just took one or more arguments.
def four_x_args(*args):
return theFinalAndTrulyRealProblemAsPosed(args)
The *args argument to the function tells Python to gather up all arguments to this function and make a tuple out of them; then the tuple is bound to the name args. You can easily make a list out of it, and then you could use the .pop(0) method to get the first item from the list. This costs the memory and time to build the list, which is why the iter() solution is so elegant.
def four_x_args(*args):
argsList = list(args) # convert from tuple to list
output = argsList.pop(0) * 4
for arg in argsList:
output += arg * 4
return output
This is just Wooble's solution, rewritten to use *args.
Examples of calling it:
print four_x_args(1) # prints 4
print four_x_args(1, 2) # prints 12
print four_x_args('a') # prints 'aaaa'
print four_x_args('ab', 'c') # prints 'ababababcccc'
Finally, I'm going to be malicious and complain about the solution you accepted. That solution depends on the object's base class having a sensible null or zero, but not all classes have this. int() returns 0, and str() returns '' (null string), so they work. But how about this:
class NaturalNumber(int):
"""
Exactly like an int, but only values >= 1 are possible.
"""
def __new__(cls, initial_value=1):
try:
n = int(initial_value)
if n < 1:
raise ValueError
except ValueError:
raise ValueError, "NaturalNumber() initial value must be an int() >= 1"
return super(NaturalNumber, cls).__new__ (cls, n)
argList = [NaturalNumber(n) for n in xrange(1, 4)]
print theFinalAndTrulyRealProblemAsPosed(argList) # prints correct answer: 24
print NaturalNumber() # prints 1
print type(argList[0])() # prints 1, same as previous line
print multiplyItemsByFour(argList) # prints 25!
Good luck in your studies, and I hope you enjoy Python as much as I do.