Related
I'm trying to make a custom tokenizer in python that works with inline tags. The goal is to take a string input like this:
'This is *tag1* a test *tag2*.'
and have it output the a list separated by tag and character:
['T', 'h', 'i', 's', ' ', 'i', 's', ' ', '*tag1*', ' ', 'a', ' ', 't', 'e', 's', 't', ' ', '*tag2*', '.']
without the tags, I would just use list(), and I think I found a solution for dealing with as single tag type, but there are multiple. There are also other multi character segments, such as ellipses, that are supposed to be encoded as a single feature.
One thing I tried is replacing the tag with a single unused character with regex and then using list() on the string:
text = 'This is *tag1* a test *tag2*.'
tidx = re.match(r'\*.*?\*', text)
text = re.sub(r'\*.*?\*', r'#', text)
text = list(text)
then I would iterate over it and replace the '#' with the extracted tags, but I have multiple different features I am trying to extract, and reiterating the process multiple times with different placeholder characters before splitting the string seems like poor practice. Is there any easier way to do something like this? I'm still quite new to this so there are still a lot of common methods I am unaware of. I guess I can also use a larger regex expression that encompasses all of the features i'm trying to extract but it still feels hacky, and I would prefer to use something more modular that can be used to find other features without writing a new expression every time.
You can use the following regex with re.findall:
\*[^*]*\*|.
See the regex demo. The re.S or re.DOTALL flag can be used with this pattern so that . could also match line break chars that it does not match by default.
Details
\*[^*]*\* - a * char, followed with zero or more chars other than * and then a *
| - or
. - any one char (with re.S).
See the Python demo:
import re
s = 'This is *tag1* a test *tag2*.'
print( re.findall(r'\*[^*]*\*|.', s, re.S) )
# => ['T', 'h', 'i', 's', ' ', 'i', 's', ' ', '*tag1*', ' ', 'a', ' ', 't', 'e', 's', 't', ' ', '*tag2*', '.']
I'm not sure exactly what would be best for you, but you should be able to use the split() method or the .format() method showcased below to get what you want.
# you can use this to get what you need
txt = 'This is *tag1* a test *tag2*.'
x = txt.split("*") #Splits up at *
x = txt.split() #Splits all the words up at the spaces
print(x)
# also, you may be looking for something like this to format a string
mystring = 'This is {} a test {}.'.format('*tag1*', '*tag2*')
print(mystring)
# using split to get ['T', 'h', 'i', 's', ' ', 'i', 's', ' ', '*tag1*', ' ', 'a', ' ', 't', 'e', 's', 't', ' ', '*tag2*', '.']
txt = 'This is *tag1* a test *tag2*.'
split = txt.split("*") #Splits up at *
finallist = [] # initialize the list
for string in split:
# print(string)
if string == '*tag1*':
finallist.append(string)
# pass
elif string == '*tag2*.':
finallist.append(string)
else:
for x in range(len(string)):
letter = string[x]
finallist.append(letter)
print(finallist)
So I wrote this code to remove the vowels from any string given.
it should work fine. And it actually does. Just not for all strings
which is weird why it would work for some strings and not for others
here's the code:
vowels = ["a", "e", "i", "o", "u"]
def anti_vowel(text):
text1 = list(text)
print text1
for i in text1:
if i.lower() in vowels:
text1.remove(i)
text2 = "".join(text1)
return text2
and here are the tests that I placed:
print anti_vowel("my name is Omar")
print anti_vowel("Hey look Words!")
print anti_vowel("Hey look more Words to look for!")
I tried placing print statements in the middle of the code to test it and I found something weird.
the for loop iterates about 3 or 4 times to remove one vowel.
I can't seem to know why
This weird case happens when 2 vowels are right next to each other. Basically, you are looping for each letter in the word, but when you remove the letter (if it is a vowel), then you shorten the length of the word, and therefore the next letter will have skipped over the real next letter. This is a problem when the letter being skipped is a vowel, but not when it is a consonant.
So how do we solve this? Well, instead of modifying the thing we're looping over, we will make a new string and modify it. So:
text2 = ""
for letter in text1:
if letter not in vowels:
text2 += letter
return text2
This can also be achieved with list comprehension:
return "".join ([letter for letter in text1 if letter not in vowels])
You don't have to convert the string to a list. Refer to this answer here:
Correct code to remove the vowels from a string in Python
Because the for statement checks through every letter within the list, even when it removes that vowel. Here's a quick example where it doesn't iterates 3-4 times to remove one vowel:
vowels = ["a", "e", "i", "o", "u"]
def anti_vowel(text):
text1 = list(text.lower())
text1 = [x for x in text1 if x not in vowels]
text2 = "".join(text1)
return (text2)
print anti_vowel("my name is Omar")
print anti_vowel("Hey look Words!")
print anti_vowel("Hey look more Words to look for!")
I am using Python 2.7. I modified your code slightly as follows,
vowels = 'aeiou'
def anti_vowel(text):
text1 = list(text)
print text1
for i in text:
if i.lower() in vowels:
text1.remove(i)
text2 = "".join(text1)
return text2
print anti_vowel("my name is Omar")
#['m', 'y', ' ', 'n', 'a', 'm', 'e', ' ', 'i', 's', ' ', 'O', 'm', 'a', 'r']
#my nm s mr
print anti_vowel("Hey look Words!")
#['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
#Hy lk Wrds!
print anti_vowel("Hey look more Words to look for!")
#['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'm', 'o', 'r', 'e', ' ', 'W', 'o', 'r', 'd', 's', ' ', 't', 'o', ' ', 'l', 'o', 'o', 'k', ' ', 'f', 'o', 'r', '!']
#Hy lk mr Wrds t lk fr!
I cannot duplicate your problem. The for loop sweeps through the input string once (one character by one character) and removes any vowel encountered. Perhaps you can post your output so we can debug.
This can be done without remove()
#!python2
def anti_vowel(text):
vowels = ["a", "e", "i", "o", "u"]
s1 = ''
for i in text:
if i.lower() in vowels:
pass
else:
s1 += i
return s1
print anti_vowel("my name is Omar")
print anti_vowel("Hey look Words!")
print anti_vowel("Hey look more Words to look for!")
This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 7 years ago.
in this code I am trying to create a function anti_vowel that will remove all vowels (aeiouAEIOU) from a string. I think it should work ok, but when I run it, the sample text "Hey look Words!" is returned as "Hy lk Words!". It "forgets" to remove the last 'o'. How can this be?
text = "Hey look Words!"
def anti_vowel(text):
textlist = list(text)
for char in textlist:
if char.lower() in 'aeiou':
textlist.remove(char)
return "".join(textlist)
print anti_vowel(text)
You're modifying the list you're iterating over, which is bound to result in some unintuitive behavior. Instead, make a copy of the list so you don't remove elements from what you're iterating through.
for char in textlist[:]: #shallow copy of the list
# etc
To clarify the behavior you're seeing, check this out. Put print char, textlist at the beginning of your (original) loop. You'd expect, perhaps, that this would print out your string vertically, alongside the list, but what you'll actually get is this:
H ['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
e ['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!'] # !
l ['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
o ['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
k ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!'] # Problem!!
['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
W ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
o ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
d ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
s ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
! ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
Hy lk Words!
So what's going on? The nice for x in y loop in Python is really just syntactic sugar: it still accesses list elements by index. So when you remove elements from the list while iterating over it, you start skipping values (as you can see above). As a result, you never see the second o in "look"; you skip over it because the index has advanced "past" it when you deleted the previous element. Then, when you get to the o in "Words", you go to remove the first occurrence of 'o', which is the one you skipped before.
As others have mentioned, list comprehensions are probably an even better (cleaner, clearer) way to do this. Make use of the fact that Python strings are iterable:
def remove_vowels(text): # function names should start with verbs! :)
return ''.join(ch for ch in text if ch.lower() not in 'aeiou')
Other answers tell you why for skips items as you alter the list. This answer tells you how you should remove characters in a string without an explicit loop, instead.
Use str.translate():
vowels = 'aeiou'
vowels += vowels.upper()
text.translate(None, vowels)
This deletes all characters listed in the second argument.
Demo:
>>> text = "Hey look Words!"
>>> vowels = 'aeiou'
>>> vowels += vowels.upper()
>>> text.translate(None, vowels)
'Hy lk Wrds!'
>>> text = 'The Quick Brown Fox Jumps Over The Lazy Fox'
>>> text.translate(None, vowels)
'Th Qck Brwn Fx Jmps vr Th Lzy Fx'
In Python 3, the str.translate() method (Python 2: unicode.translate()) differs in that it doesn't take a deletechars parameter; the first argument is a dictionary mapping Unicode ordinals (integer values) to new values instead. Use None for any character that needs to be deleted:
# Python 3 code
vowels = 'aeiou'
vowels += vowels.upper()
vowels_table = dict.fromkeys(map(ord, vowels))
text.translate(vowels_table)
You can also use the str.maketrans() static method to produce that mapping:
vowels = 'aeiou'
vowels += vowels.upper()
text.translate(text.maketrans('', '', vowels))
Quoting from the docs:
Note: There is a subtlety when the sequence is being modified by the
loop (this can only occur for mutable sequences, i.e. lists). An
internal counter is used to keep track of which item is used next, and
this is incremented on each iteration. When this counter has reached
the length of the sequence the loop terminates. This means that if the
suite deletes the current (or a previous) item from the sequence, the
next item will be skipped (since it gets the index of the current item
which has already been treated). Likewise, if the suite inserts an
item in the sequence before the current item, the current item will be
treated again the next time through the loop. This can lead to nasty
bugs that can be avoided by making a temporary copy using a slice of
the whole sequence, e.g.,
for x in a[:]:
if x < 0: a.remove(x)
Iterate over a shallow copy of the list using [:]. You're modifying a list while iterating over it, this will result in some letters being missed.
The for loop keeps track of index, so when you remove an item at index i, the next item at i+1th position shifts to the current index(i) and hence in the next iteration you'll actually pick the i+2th item.
Lets take an easy example:
>>> text = "whoops"
>>> textlist = list(text)
>>> textlist
['w', 'h', 'o', 'o', 'p', 's']
for char in textlist:
if char.lower() in 'aeiou':
textlist.remove(char)
Iteration 1 : Index = 0.
char = 'W' as it is at index 0. As it doesn't satisfies that condition you'll do noting.
Iteration 2 : Index = 1.
char = 'h' as it is at index 1. Nothing more to do here.
Iteration 3 : Index = 2.
char = 'o' as it is at index 2. As this item satisfies the condition so it'll be removed from the list and all the items to it's right will shift one place to the left to fill the gap.
now textlist becomes :
0 1 2 3 4
`['w', 'h', 'o', 'p', 's']`
As you can see the other 'o' moved to index 2, i.e the current index so it'll be skipped in the next iteration. So, this is the reason some items are bring skipped in your iteration. Whenever you remove an item the next item is skipped from the iteration.
Iteration 4 : Index = 3.
char = 'p' as it is at index 3.
....
Fix:
Iterate over a shallow copy of the list to fix this issue:
for char in textlist[:]: #note the [:]
if char.lower() in 'aeiou':
textlist.remove(char)
Other alternatives:
List comprehension:
A one-liner using str.join and a list comprehension:
vowels = 'aeiou'
text = "Hey look Words!"
return "".join([char for char in text if char.lower() not in vowels])
regex:
>>> import re
>>> text = "Hey look Words!"
>>> re.sub('[aeiou]', '', text, flags=re.I)
'Hy lk Wrds!'
You're modifying the data you're iterating over. Don't do that.
''.join(x for x in textlist in x not in VOWELS)
text = "Hey look Words!"
print filter(lambda x: x not in "AaEeIiOoUu", text)
Output
Hy lk Wrds!
You're iterating over a list and deleting elements from it at the same time.
First, I need to make sure you clearly understand the role of char in for char in textlist: .... Take the situation where we have reached the letter 'l'. The situation is not like this:
['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
char
There is no link between char and the position of the letter 'l' in the list. If you modify char, the list will not be modified. The situation is more like this:
['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
char = 'l'
Notice that I've kept the ^ symbol. This is the hidden pointer that the code managing the for char in textlist: ... loop uses to keep track of its position in the loop. Every time you enter the body of the loop, the pointer is advanced, and the letter referenced by the pointer is copied into char.
Your problem occurs when you have two vowels in succession. I'll show you what happens from the point where you reach 'l'. Notice that I've also changed the word "look" to "leap", to make it clearer what's going on:
advance pointer to next character ('l') and copy to char
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'l'
char ('l') is not a vowel, so do nothing
advance pointer to next character ('e') and copy to char
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'e'
char ('e') is a vowel, so delete the first occurrence of char ('e')
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', <- 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
advance pointer to next character ('p') and copy to char
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'p'
When you removed the 'e' all the characters after the 'e' moved one place to the left, so it was as if remove had advanced the pointer. The result is that you skipped past the 'a'.
In general, you should avoid modifying lists while iterating over them. It's better to construct a new list from scratch, and Python's list comprehensions are the perfect tool for doing this. E.g.
print ''.join([char for char in "Hey look Words" if char.lower() not in "aeiou"])
But if you haven't learnt about comprehensions yet, the best way is probably:
text = "Hey look Words!"
def anti_vowel(text):
textlist = list(text)
new_textlist = []
for char in textlist:
if char.lower() not in 'aeiou':
new_textlist.append(char)
return "".join(new_textlist)
print anti_vowel(text)
List Comprehensions:
vowels = 'aeiou'
text = 'Hey look Words!'
result = [char for char in text if char not in vowels]
print ''.join(result)
Others have already explained the issue with your code. For your task, a generator expression is easier and less error prone.
>>> text = "Hey look Words!"
>>> ''.join(c for c in text if c.lower() not in 'aeiou')
'Hy lk Wrds!'
or
>>> ''.join(c for c in text if c not in 'AaEeIiOoUu')
'Hy lk Wrds!'
however, str.translate is the best way to go.
You shouldn't delete items from list you iterating through:
But you can make new list from the old one with list comprehension syntax. List comprehension is very useful in this situation. You can read about list comprehension here
So you solution will look like this:
text = "Hey look Words!"
def anti_vowel(text):
return "".join([char for char in list(text) if char.lower() not in 'aeiou'])
print anti_vowel(text)
It's pretty, isn't it :P
Try to not use the list() function on a string. It will make things a lot more complicated.
Unlike Java, in Python, strings are considered as arrays. Then, try to use an index for loop and del keyword.
for x in range(len(string)):
if string[x].lower() in "aeiou":
del string[x]
This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 7 years ago.
in this code I am trying to create a function anti_vowel that will remove all vowels (aeiouAEIOU) from a string. I think it should work ok, but when I run it, the sample text "Hey look Words!" is returned as "Hy lk Words!". It "forgets" to remove the last 'o'. How can this be?
text = "Hey look Words!"
def anti_vowel(text):
textlist = list(text)
for char in textlist:
if char.lower() in 'aeiou':
textlist.remove(char)
return "".join(textlist)
print anti_vowel(text)
You're modifying the list you're iterating over, which is bound to result in some unintuitive behavior. Instead, make a copy of the list so you don't remove elements from what you're iterating through.
for char in textlist[:]: #shallow copy of the list
# etc
To clarify the behavior you're seeing, check this out. Put print char, textlist at the beginning of your (original) loop. You'd expect, perhaps, that this would print out your string vertically, alongside the list, but what you'll actually get is this:
H ['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
e ['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!'] # !
l ['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
o ['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
k ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!'] # Problem!!
['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
W ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
o ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
d ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
s ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
! ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
Hy lk Words!
So what's going on? The nice for x in y loop in Python is really just syntactic sugar: it still accesses list elements by index. So when you remove elements from the list while iterating over it, you start skipping values (as you can see above). As a result, you never see the second o in "look"; you skip over it because the index has advanced "past" it when you deleted the previous element. Then, when you get to the o in "Words", you go to remove the first occurrence of 'o', which is the one you skipped before.
As others have mentioned, list comprehensions are probably an even better (cleaner, clearer) way to do this. Make use of the fact that Python strings are iterable:
def remove_vowels(text): # function names should start with verbs! :)
return ''.join(ch for ch in text if ch.lower() not in 'aeiou')
Other answers tell you why for skips items as you alter the list. This answer tells you how you should remove characters in a string without an explicit loop, instead.
Use str.translate():
vowels = 'aeiou'
vowels += vowels.upper()
text.translate(None, vowels)
This deletes all characters listed in the second argument.
Demo:
>>> text = "Hey look Words!"
>>> vowels = 'aeiou'
>>> vowels += vowels.upper()
>>> text.translate(None, vowels)
'Hy lk Wrds!'
>>> text = 'The Quick Brown Fox Jumps Over The Lazy Fox'
>>> text.translate(None, vowels)
'Th Qck Brwn Fx Jmps vr Th Lzy Fx'
In Python 3, the str.translate() method (Python 2: unicode.translate()) differs in that it doesn't take a deletechars parameter; the first argument is a dictionary mapping Unicode ordinals (integer values) to new values instead. Use None for any character that needs to be deleted:
# Python 3 code
vowels = 'aeiou'
vowels += vowels.upper()
vowels_table = dict.fromkeys(map(ord, vowels))
text.translate(vowels_table)
You can also use the str.maketrans() static method to produce that mapping:
vowels = 'aeiou'
vowels += vowels.upper()
text.translate(text.maketrans('', '', vowels))
Quoting from the docs:
Note: There is a subtlety when the sequence is being modified by the
loop (this can only occur for mutable sequences, i.e. lists). An
internal counter is used to keep track of which item is used next, and
this is incremented on each iteration. When this counter has reached
the length of the sequence the loop terminates. This means that if the
suite deletes the current (or a previous) item from the sequence, the
next item will be skipped (since it gets the index of the current item
which has already been treated). Likewise, if the suite inserts an
item in the sequence before the current item, the current item will be
treated again the next time through the loop. This can lead to nasty
bugs that can be avoided by making a temporary copy using a slice of
the whole sequence, e.g.,
for x in a[:]:
if x < 0: a.remove(x)
Iterate over a shallow copy of the list using [:]. You're modifying a list while iterating over it, this will result in some letters being missed.
The for loop keeps track of index, so when you remove an item at index i, the next item at i+1th position shifts to the current index(i) and hence in the next iteration you'll actually pick the i+2th item.
Lets take an easy example:
>>> text = "whoops"
>>> textlist = list(text)
>>> textlist
['w', 'h', 'o', 'o', 'p', 's']
for char in textlist:
if char.lower() in 'aeiou':
textlist.remove(char)
Iteration 1 : Index = 0.
char = 'W' as it is at index 0. As it doesn't satisfies that condition you'll do noting.
Iteration 2 : Index = 1.
char = 'h' as it is at index 1. Nothing more to do here.
Iteration 3 : Index = 2.
char = 'o' as it is at index 2. As this item satisfies the condition so it'll be removed from the list and all the items to it's right will shift one place to the left to fill the gap.
now textlist becomes :
0 1 2 3 4
`['w', 'h', 'o', 'p', 's']`
As you can see the other 'o' moved to index 2, i.e the current index so it'll be skipped in the next iteration. So, this is the reason some items are bring skipped in your iteration. Whenever you remove an item the next item is skipped from the iteration.
Iteration 4 : Index = 3.
char = 'p' as it is at index 3.
....
Fix:
Iterate over a shallow copy of the list to fix this issue:
for char in textlist[:]: #note the [:]
if char.lower() in 'aeiou':
textlist.remove(char)
Other alternatives:
List comprehension:
A one-liner using str.join and a list comprehension:
vowels = 'aeiou'
text = "Hey look Words!"
return "".join([char for char in text if char.lower() not in vowels])
regex:
>>> import re
>>> text = "Hey look Words!"
>>> re.sub('[aeiou]', '', text, flags=re.I)
'Hy lk Wrds!'
You're modifying the data you're iterating over. Don't do that.
''.join(x for x in textlist in x not in VOWELS)
text = "Hey look Words!"
print filter(lambda x: x not in "AaEeIiOoUu", text)
Output
Hy lk Wrds!
You're iterating over a list and deleting elements from it at the same time.
First, I need to make sure you clearly understand the role of char in for char in textlist: .... Take the situation where we have reached the letter 'l'. The situation is not like this:
['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
char
There is no link between char and the position of the letter 'l' in the list. If you modify char, the list will not be modified. The situation is more like this:
['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
char = 'l'
Notice that I've kept the ^ symbol. This is the hidden pointer that the code managing the for char in textlist: ... loop uses to keep track of its position in the loop. Every time you enter the body of the loop, the pointer is advanced, and the letter referenced by the pointer is copied into char.
Your problem occurs when you have two vowels in succession. I'll show you what happens from the point where you reach 'l'. Notice that I've also changed the word "look" to "leap", to make it clearer what's going on:
advance pointer to next character ('l') and copy to char
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'l'
char ('l') is not a vowel, so do nothing
advance pointer to next character ('e') and copy to char
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'e'
char ('e') is a vowel, so delete the first occurrence of char ('e')
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', <- 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
advance pointer to next character ('p') and copy to char
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'p'
When you removed the 'e' all the characters after the 'e' moved one place to the left, so it was as if remove had advanced the pointer. The result is that you skipped past the 'a'.
In general, you should avoid modifying lists while iterating over them. It's better to construct a new list from scratch, and Python's list comprehensions are the perfect tool for doing this. E.g.
print ''.join([char for char in "Hey look Words" if char.lower() not in "aeiou"])
But if you haven't learnt about comprehensions yet, the best way is probably:
text = "Hey look Words!"
def anti_vowel(text):
textlist = list(text)
new_textlist = []
for char in textlist:
if char.lower() not in 'aeiou':
new_textlist.append(char)
return "".join(new_textlist)
print anti_vowel(text)
List Comprehensions:
vowels = 'aeiou'
text = 'Hey look Words!'
result = [char for char in text if char not in vowels]
print ''.join(result)
Others have already explained the issue with your code. For your task, a generator expression is easier and less error prone.
>>> text = "Hey look Words!"
>>> ''.join(c for c in text if c.lower() not in 'aeiou')
'Hy lk Wrds!'
or
>>> ''.join(c for c in text if c not in 'AaEeIiOoUu')
'Hy lk Wrds!'
however, str.translate is the best way to go.
You shouldn't delete items from list you iterating through:
But you can make new list from the old one with list comprehension syntax. List comprehension is very useful in this situation. You can read about list comprehension here
So you solution will look like this:
text = "Hey look Words!"
def anti_vowel(text):
return "".join([char for char in list(text) if char.lower() not in 'aeiou'])
print anti_vowel(text)
It's pretty, isn't it :P
Try to not use the list() function on a string. It will make things a lot more complicated.
Unlike Java, in Python, strings are considered as arrays. Then, try to use an index for loop and del keyword.
for x in range(len(string)):
if string[x].lower() in "aeiou":
del string[x]
This question already has answers here:
Strange result when removing item from a list while iterating over it
(8 answers)
Closed 7 years ago.
in this code I am trying to create a function anti_vowel that will remove all vowels (aeiouAEIOU) from a string. I think it should work ok, but when I run it, the sample text "Hey look Words!" is returned as "Hy lk Words!". It "forgets" to remove the last 'o'. How can this be?
text = "Hey look Words!"
def anti_vowel(text):
textlist = list(text)
for char in textlist:
if char.lower() in 'aeiou':
textlist.remove(char)
return "".join(textlist)
print anti_vowel(text)
You're modifying the list you're iterating over, which is bound to result in some unintuitive behavior. Instead, make a copy of the list so you don't remove elements from what you're iterating through.
for char in textlist[:]: #shallow copy of the list
# etc
To clarify the behavior you're seeing, check this out. Put print char, textlist at the beginning of your (original) loop. You'd expect, perhaps, that this would print out your string vertically, alongside the list, but what you'll actually get is this:
H ['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
e ['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!'] # !
l ['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
o ['H', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
k ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!'] # Problem!!
['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
W ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
o ['H', 'y', ' ', 'l', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
d ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
s ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
! ['H', 'y', ' ', 'l', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
Hy lk Words!
So what's going on? The nice for x in y loop in Python is really just syntactic sugar: it still accesses list elements by index. So when you remove elements from the list while iterating over it, you start skipping values (as you can see above). As a result, you never see the second o in "look"; you skip over it because the index has advanced "past" it when you deleted the previous element. Then, when you get to the o in "Words", you go to remove the first occurrence of 'o', which is the one you skipped before.
As others have mentioned, list comprehensions are probably an even better (cleaner, clearer) way to do this. Make use of the fact that Python strings are iterable:
def remove_vowels(text): # function names should start with verbs! :)
return ''.join(ch for ch in text if ch.lower() not in 'aeiou')
Other answers tell you why for skips items as you alter the list. This answer tells you how you should remove characters in a string without an explicit loop, instead.
Use str.translate():
vowels = 'aeiou'
vowels += vowels.upper()
text.translate(None, vowels)
This deletes all characters listed in the second argument.
Demo:
>>> text = "Hey look Words!"
>>> vowels = 'aeiou'
>>> vowels += vowels.upper()
>>> text.translate(None, vowels)
'Hy lk Wrds!'
>>> text = 'The Quick Brown Fox Jumps Over The Lazy Fox'
>>> text.translate(None, vowels)
'Th Qck Brwn Fx Jmps vr Th Lzy Fx'
In Python 3, the str.translate() method (Python 2: unicode.translate()) differs in that it doesn't take a deletechars parameter; the first argument is a dictionary mapping Unicode ordinals (integer values) to new values instead. Use None for any character that needs to be deleted:
# Python 3 code
vowels = 'aeiou'
vowels += vowels.upper()
vowels_table = dict.fromkeys(map(ord, vowels))
text.translate(vowels_table)
You can also use the str.maketrans() static method to produce that mapping:
vowels = 'aeiou'
vowels += vowels.upper()
text.translate(text.maketrans('', '', vowels))
Quoting from the docs:
Note: There is a subtlety when the sequence is being modified by the
loop (this can only occur for mutable sequences, i.e. lists). An
internal counter is used to keep track of which item is used next, and
this is incremented on each iteration. When this counter has reached
the length of the sequence the loop terminates. This means that if the
suite deletes the current (or a previous) item from the sequence, the
next item will be skipped (since it gets the index of the current item
which has already been treated). Likewise, if the suite inserts an
item in the sequence before the current item, the current item will be
treated again the next time through the loop. This can lead to nasty
bugs that can be avoided by making a temporary copy using a slice of
the whole sequence, e.g.,
for x in a[:]:
if x < 0: a.remove(x)
Iterate over a shallow copy of the list using [:]. You're modifying a list while iterating over it, this will result in some letters being missed.
The for loop keeps track of index, so when you remove an item at index i, the next item at i+1th position shifts to the current index(i) and hence in the next iteration you'll actually pick the i+2th item.
Lets take an easy example:
>>> text = "whoops"
>>> textlist = list(text)
>>> textlist
['w', 'h', 'o', 'o', 'p', 's']
for char in textlist:
if char.lower() in 'aeiou':
textlist.remove(char)
Iteration 1 : Index = 0.
char = 'W' as it is at index 0. As it doesn't satisfies that condition you'll do noting.
Iteration 2 : Index = 1.
char = 'h' as it is at index 1. Nothing more to do here.
Iteration 3 : Index = 2.
char = 'o' as it is at index 2. As this item satisfies the condition so it'll be removed from the list and all the items to it's right will shift one place to the left to fill the gap.
now textlist becomes :
0 1 2 3 4
`['w', 'h', 'o', 'p', 's']`
As you can see the other 'o' moved to index 2, i.e the current index so it'll be skipped in the next iteration. So, this is the reason some items are bring skipped in your iteration. Whenever you remove an item the next item is skipped from the iteration.
Iteration 4 : Index = 3.
char = 'p' as it is at index 3.
....
Fix:
Iterate over a shallow copy of the list to fix this issue:
for char in textlist[:]: #note the [:]
if char.lower() in 'aeiou':
textlist.remove(char)
Other alternatives:
List comprehension:
A one-liner using str.join and a list comprehension:
vowels = 'aeiou'
text = "Hey look Words!"
return "".join([char for char in text if char.lower() not in vowels])
regex:
>>> import re
>>> text = "Hey look Words!"
>>> re.sub('[aeiou]', '', text, flags=re.I)
'Hy lk Wrds!'
You're modifying the data you're iterating over. Don't do that.
''.join(x for x in textlist in x not in VOWELS)
text = "Hey look Words!"
print filter(lambda x: x not in "AaEeIiOoUu", text)
Output
Hy lk Wrds!
You're iterating over a list and deleting elements from it at the same time.
First, I need to make sure you clearly understand the role of char in for char in textlist: .... Take the situation where we have reached the letter 'l'. The situation is not like this:
['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
char
There is no link between char and the position of the letter 'l' in the list. If you modify char, the list will not be modified. The situation is more like this:
['H', 'e', 'y', ' ', 'l', 'o', 'o', 'k', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
char = 'l'
Notice that I've kept the ^ symbol. This is the hidden pointer that the code managing the for char in textlist: ... loop uses to keep track of its position in the loop. Every time you enter the body of the loop, the pointer is advanced, and the letter referenced by the pointer is copied into char.
Your problem occurs when you have two vowels in succession. I'll show you what happens from the point where you reach 'l'. Notice that I've also changed the word "look" to "leap", to make it clearer what's going on:
advance pointer to next character ('l') and copy to char
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'l'
char ('l') is not a vowel, so do nothing
advance pointer to next character ('e') and copy to char
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'e'
char ('e') is a vowel, so delete the first occurrence of char ('e')
['H', 'e', 'y', ' ', 'l', 'e', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', <- 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
^
advance pointer to next character ('p') and copy to char
['H', 'e', 'y', ' ', 'l', 'a', 'p', ' ', 'W', 'o', 'r', 'd', 's', '!']
-> ^
char = 'p'
When you removed the 'e' all the characters after the 'e' moved one place to the left, so it was as if remove had advanced the pointer. The result is that you skipped past the 'a'.
In general, you should avoid modifying lists while iterating over them. It's better to construct a new list from scratch, and Python's list comprehensions are the perfect tool for doing this. E.g.
print ''.join([char for char in "Hey look Words" if char.lower() not in "aeiou"])
But if you haven't learnt about comprehensions yet, the best way is probably:
text = "Hey look Words!"
def anti_vowel(text):
textlist = list(text)
new_textlist = []
for char in textlist:
if char.lower() not in 'aeiou':
new_textlist.append(char)
return "".join(new_textlist)
print anti_vowel(text)
List Comprehensions:
vowels = 'aeiou'
text = 'Hey look Words!'
result = [char for char in text if char not in vowels]
print ''.join(result)
Others have already explained the issue with your code. For your task, a generator expression is easier and less error prone.
>>> text = "Hey look Words!"
>>> ''.join(c for c in text if c.lower() not in 'aeiou')
'Hy lk Wrds!'
or
>>> ''.join(c for c in text if c not in 'AaEeIiOoUu')
'Hy lk Wrds!'
however, str.translate is the best way to go.
You shouldn't delete items from list you iterating through:
But you can make new list from the old one with list comprehension syntax. List comprehension is very useful in this situation. You can read about list comprehension here
So you solution will look like this:
text = "Hey look Words!"
def anti_vowel(text):
return "".join([char for char in list(text) if char.lower() not in 'aeiou'])
print anti_vowel(text)
It's pretty, isn't it :P
Try to not use the list() function on a string. It will make things a lot more complicated.
Unlike Java, in Python, strings are considered as arrays. Then, try to use an index for loop and del keyword.
for x in range(len(string)):
if string[x].lower() in "aeiou":
del string[x]