I have a list like
['hello', '...', 'h3.a', 'ds4,']
this should turn into
['hello', 'h3a', 'ds4']
and i want to remove only the punctuation leaving the letters and numbers intact.
Punctuation is anything in the string.punctuation constant.
I know that this is gunna be simple but im kinda noobie at python so...
Thanks,
giodamelio
Assuming that your initial list is stored in a variable x, you can use this:
>>> x = [''.join(c for c in s if c not in string.punctuation) for s in x]
>>> print(x)
['hello', '', 'h3a', 'ds4']
To remove the empty strings:
>>> x = [s for s in x if s]
>>> print(x)
['hello', 'h3a', 'ds4']
Use string.translate:
>>> import string
>>> test_case = ['hello', '...', 'h3.a', 'ds4,']
>>> [s.translate(None, string.punctuation) for s in test_case]
['hello', '', 'h3a', 'ds4']
For the documentation of translate, see http://docs.python.org/library/string.html
In python 3+ use this instead:
import string
s = s.translate(str.maketrans('','',string.punctuation))
import string
print ''.join((x for x in st if x not in string.punctuation))
ps st is the string. for the list is the same...
[''.join(x for x in par if x not in string.punctuation) for par in alist]
i think works well. look at string.punctuaction:
>>> print string.punctuation
!"#$%&\'()*+,-./:;<=>?#[\\]^_`{|}~
To make a new list:
[re.sub(r'[^A-Za-z0-9]+', '', x) for x in list_of_strings]
Just be aware that string.punctuation works in English, but may not work for other languages with other punctuation marks.
You could add them to a list LIST_OF_LANGUAGE_SPECIFIC_PUNCTUATION and then concatenate it to string.punctuation to get a fuller set of punctuation.
punctuation = string.punctuation + [LIST_OF_LANGUAGE_SPECIFIC_PUNCTUATION]
Related
I was trying to create a program that removes all sorts of punctuation from a given input sentence. The code looked somewhat like this
from string import punctuation
sent = str(input())
def rempunc(string):
for i in string:
word =''
list = [0]
if i in punctuation:
x = string.index(i)
word += string[list[-1]:x]+' '
list.append(x)
list_2 = word.split(' ')
return list_2
print(rempunc(sent))
However the output is coming out as follows:
This state ment has # 1 ! punc.
['This', 'state', 'ment', 'has', '#', '1', '!', 'punc', '']
Why isn't the punctuation being removed entirely? Am I missing something in the code?
I tried changing x with x-1 in line 7 but it did not help. Now I'm stuck and don't know what else to try.
Repeated string slicing isn't necessary here.
I would suggest using filter() to filter out the undesired characters for each word, and then reading that result into a list comprehension. From there, you can use a second filter() operation to remove the empty strings:
from string import punctuation
def remove_punctuation(s):
cleaned_words = [''.join(filter(lambda x: x not in punctuation, word))
for word in s.split()]
return list(filter(lambda x: x != "", cleaned_words))
print(remove_punctuation(input()))
This outputs:
['This', 'state', 'ment', 'has', '1', 'punc']
I want to switch text but I always fail.
Let's say I want to switch,
I with We inx='I are We'
I tried
x=x.replace('I','We').replace('We','I')
but it is obvious that it will print I are I
Can someone help?
You can use a regex to avoid going through your string several times (Each replace go through the list) and to make it more readable ! It also works on several occurences words.
string = 'I are We, I'
import re
replacements = {'I': 'We', 'We': 'I'}
print(re.sub("I|We", lambda x: replacements[x.group()], string)) # Matching words you want to replace, and replace them using a dict
Output
"We are I, We"
You can use re.sub with function as a substitution:
In [9]: import re
In [10]: x = 'I are We'
In [11]: re.sub('I|We', lambda match: 'We' if match.group(0) == 'I' else 'I', x)
Out[11]: 'We are I'
If you need to replace more than 2 substrings you may want to create a dict like d = {'I': 'We', 'We': 'I', 'You': 'Not You'} and pick correct replacement like lambda match: d[match.group(0)]. You may also want to construct regular expression dynamically based on the replacement strings, but make sure to escape them:
In [14]: d = {'We': 'I', 'I': 'We', 'ar|e': 'am'}
In [15]: re.sub('|'.join(map(re.escape, d.keys())), lambda match: d[match.group(0)], 'We ar|e I')
Out[15]: 'I am We'
x='I are We'
x=x.replace('I','You').replace('We','I').replace('You','We')
>>> x
'We are I'
It is a bit clunky, but i tend to do something along the lines of
x='I are We'
x=x.replace('I','we')
x=x.replace('We','I')
x=x.replace('we','We')
Which can be shortened to
`x=x.replace('I','we').replace('We','I').replace('we','We')
This doesn't make use of replace, but I hope it helps:
s = "I are We"
d = {"I": "We", "We": "I"}
" ".join([d.get(x, x) for x in s.split()])
>>> 'We are I'
x='I are We'
dic = {'I':'We','We':'I'}
sol = []
for i in x.split():
if i in dic:
sol.append(dic[i])
else:
sol.append(i)
result = ' '.join(sol)
print(result)
I have a list such as this
list = ['Hi', ',', 'my', 'name', 'is', 'Bob', '!']
I wanted to convert this to a string, and originally, I found on stackoverflow that .join() could be used. So i did:
x = ' '.join(list)
print(x)
which prints
"Hi , my name is Bob !"
when what I want printed is:
"Hi, my name is Bob!"
How do I not add spaces before periods and exclamation points? I want a more general case so that I can for example read in a text file as a list, and convert it to a string.
Thanks!
To solve it in a general case, use the nltk's "moses" detokenizer:
In [1]: l = ["Hi", ",", "my", "name", "is", "Bob", "!"]
In [2]: from nltk.tokenize.moses import MosesDetokenizer
In [3]: detokenizer = MosesDetokenizer()
In [4]: detokenizer.detokenize(l, return_str=True)
Out[4]: u'Hi, my name is Bob!'
The detokenizer is not yet a part of a stable nltk package. To be able to use it now, install nltk directly from github.
How about this, using simple regex?
import re
list = ['Hi', ',', 'my', 'name', 'is', 'Bob', '!']
x = re.sub(r' (\W)',r'\1',' '.join(list))
print(x)
>>> Hi, my name is Bob!
A little different solution:
>>> from string import punctuation
>>> lis = ["Hi", ",", "my", "name", "is", "Bob", "!"]
>>> string = ''
>>> for i, x in enumerate(lis):
if x not in punctuation and i != 0:
string += ' ' + x
elif x not in punctuation and i == 0:
string += x
else:
string += x
>>> print(string)
"Hi, my name is Bob!"
I'm trying to convert a string to a list of words using python. I want to take something like the following:
string = 'This is a string, with words!'
Then convert to something like this :
list = ['This', 'is', 'a', 'string', 'with', 'words']
Notice the omission of punctuation and spaces. What would be the fastest way of going about this?
I think this is the simplest way for anyone else stumbling on this post given the late response:
>>> string = 'This is a string, with words!'
>>> string.split()
['This', 'is', 'a', 'string,', 'with', 'words!']
Try this:
import re
mystr = 'This is a string, with words!'
wordList = re.sub("[^\w]", " ", mystr).split()
How it works:
From the docs :
re.sub(pattern, repl, string, count=0, flags=0)
Return the string obtained by replacing the leftmost non-overlapping occurrences of pattern in string by the replacement repl. If the pattern isn’t found, string is returned unchanged. repl can be a string or a function.
so in our case :
pattern is any non-alphanumeric character.
[\w] means any alphanumeric character and is equal to the character set
[a-zA-Z0-9_]
a to z, A to Z , 0 to 9 and underscore.
so we match any non-alphanumeric character and replace it with a space .
and then we split() it which splits string by space and converts it to a list
so 'hello-world'
becomes 'hello world'
with re.sub
and then ['hello' , 'world']
after split()
let me know if any doubts come up.
To do this properly is quite complex. For your research, it is known as word tokenization. You should look at NLTK if you want to see what others have done, rather than starting from scratch:
>>> import nltk
>>> paragraph = u"Hi, this is my first sentence. And this is my second."
>>> sentences = nltk.sent_tokenize(paragraph)
>>> for sentence in sentences:
... nltk.word_tokenize(sentence)
[u'Hi', u',', u'this', u'is', u'my', u'first', u'sentence', u'.']
[u'And', u'this', u'is', u'my', u'second', u'.']
The most simple way:
>>> import re
>>> string = 'This is a string, with words!'
>>> re.findall(r'\w+', string)
['This', 'is', 'a', 'string', 'with', 'words']
Using string.punctuation for completeness:
import re
import string
x = re.sub('['+string.punctuation+']', '', s).split()
This handles newlines as well.
Well, you could use
import re
list = re.sub(r'[.!,;?]', ' ', string).split()
Note that both string and list are names of builtin types, so you probably don't want to use those as your variable names.
Inspired by #mtrw's answer, but improved to strip out punctuation at word boundaries only:
import re
import string
def extract_words(s):
return [re.sub('^[{0}]+|[{0}]+$'.format(string.punctuation), '', w) for w in s.split()]
>>> str = 'This is a string, with words!'
>>> extract_words(str)
['This', 'is', 'a', 'string', 'with', 'words']
>>> str = '''I'm a custom-built sentence with "tricky" words like https://stackoverflow.com/.'''
>>> extract_words(str)
["I'm", 'a', 'custom-built', 'sentence', 'with', 'tricky', 'words', 'like', 'https://stackoverflow.com']
Personally, I think this is slightly cleaner than the answers provided
def split_to_words(sentence):
return list(filter(lambda w: len(w) > 0, re.split('\W+', sentence))) #Use sentence.lower(), if needed
A regular expression for words would give you the most control. You would want to carefully consider how to deal with words with dashes or apostrophes, like "I'm".
list=mystr.split(" ",mystr.count(" "))
This way you eliminate every special char outside of the alphabet:
def wordsToList(strn):
L = strn.split()
cleanL = []
abc = 'abcdefghijklmnopqrstuvwxyz'
ABC = abc.upper()
letters = abc + ABC
for e in L:
word = ''
for c in e:
if c in letters:
word += c
if word != '':
cleanL.append(word)
return cleanL
s = 'She loves you, yea yea yea! '
L = wordsToList(s)
print(L) # ['She', 'loves', 'you', 'yea', 'yea', 'yea']
I'm not sure if this is fast or optimal or even the right way to program.
def split_string(string):
return string.split()
This function will return the list of words of a given string.
In this case, if we call the function as follows,
string = 'This is a string, with words!'
split_string(string)
The return output of the function would be
['This', 'is', 'a', 'string,', 'with', 'words!']
This is from my attempt on a coding challenge that can't use regex,
outputList = "".join((c if c.isalnum() or c=="'" else ' ') for c in inputStr ).split(' ')
The role of apostrophe seems interesting.
Probably not very elegant, but at least you know what's going on.
my_str = "Simple sample, test! is, olny".lower()
my_lst =[]
temp=""
len_my_str = len(my_str)
number_letter_in_data=0
list_words_number=0
for number_letter_in_data in range(0, len_my_str, 1):
if my_str[number_letter_in_data] in [',', '.', '!', '(', ')', ':', ';', '-']:
pass
else:
if my_str[number_letter_in_data] in [' ']:
#if you want longer than 3 char words
if len(temp)>3:
list_words_number +=1
my_lst.append(temp)
temp=""
else:
pass
else:
temp = temp+my_str[number_letter_in_data]
my_lst.append(temp)
print(my_lst)
You can try and do this:
tryTrans = string.maketrans(",!", " ")
str = "This is a string, with words!"
str = str.translate(tryTrans)
listOfWords = str.split()
I have some python code that splits on comma, but doesn't strip the whitespace:
>>> string = "blah, lots , of , spaces, here "
>>> mylist = string.split(',')
>>> print mylist
['blah', ' lots ', ' of ', ' spaces', ' here ']
I would rather end up with whitespace removed like this:
['blah', 'lots', 'of', 'spaces', 'here']
I am aware that I could loop through the list and strip() each item but, as this is Python, I'm guessing there's a quicker, easier and more elegant way of doing it.
Use list comprehension -- simpler, and just as easy to read as a for loop.
my_string = "blah, lots , of , spaces, here "
result = [x.strip() for x in my_string.split(',')]
# result is ["blah", "lots", "of", "spaces", "here"]
See: Python docs on List Comprehension
A good 2 second explanation of list comprehension.
I came to add:
map(str.strip, string.split(','))
but saw it had already been mentioned by Jason Orendorff in a comment.
Reading Glenn Maynard's comment on the same answer suggesting list comprehensions over map I started to wonder why. I assumed he meant for performance reasons, but of course he might have meant for stylistic reasons, or something else (Glenn?).
So a quick (possibly flawed?) test on my box (Python 2.6.5 on Ubuntu 10.04) applying the three methods in a loop revealed:
$ time ./list_comprehension.py # [word.strip() for word in string.split(',')]
real 0m22.876s
$ time ./map_with_lambda.py # map(lambda s: s.strip(), string.split(','))
real 0m25.736s
$ time ./map_with_str.strip.py # map(str.strip, string.split(','))
real 0m19.428s
making map(str.strip, string.split(',')) the winner, although it seems they are all in the same ballpark.
Certainly though map (with or without a lambda) should not necessarily be ruled out for performance reasons, and for me it is at least as clear as a list comprehension.
Split using a regular expression. Note I made the case more general with leading spaces. The list comprehension is to remove the null strings at the front and back.
>>> import re
>>> string = " blah, lots , of , spaces, here "
>>> pattern = re.compile("^\s+|\s*,\s*|\s+$")
>>> print([x for x in pattern.split(string) if x])
['blah', 'lots', 'of', 'spaces', 'here']
This works even if ^\s+ doesn't match:
>>> string = "foo, bar "
>>> print([x for x in pattern.split(string) if x])
['foo', 'bar']
>>>
Here's why you need ^\s+:
>>> pattern = re.compile("\s*,\s*|\s+$")
>>> print([x for x in pattern.split(string) if x])
[' blah', 'lots', 'of', 'spaces', 'here']
See the leading spaces in blah?
Clarification: above uses the Python 3 interpreter, but results are the same in Python 2.
Just remove the white space from the string before you split it.
mylist = my_string.replace(' ','').split(',')
I know this has already been answered, but if you end doing this a lot, regular expressions may be a better way to go:
>>> import re
>>> re.sub(r'\s', '', string).split(',')
['blah', 'lots', 'of', 'spaces', 'here']
The \s matches any whitespace character, and we just replace it with an empty string ''. You can find more info here: http://docs.python.org/library/re.html#re.sub
map(lambda s: s.strip(), mylist) would be a little better than explicitly looping. Or for the whole thing at once: map(lambda s:s.strip(), string.split(','))
import re
result=[x for x in re.split(',| ',your_string) if x!='']
this works fine for me.
re (as in regular expressions) allows splitting on multiple characters at once:
$ string = "blah, lots , of , spaces, here "
$ re.split(', ',string)
['blah', 'lots ', ' of ', ' spaces', 'here ']
This doesn't work well for your example string, but works nicely for a comma-space separated list. For your example string, you can combine the re.split power to split on regex patterns to get a "split-on-this-or-that" effect.
$ re.split('[, ]',string)
['blah',
'',
'lots',
'',
'',
'',
'',
'of',
'',
'',
'',
'spaces',
'',
'here',
'']
Unfortunately, that's ugly, but a filter will do the trick:
$ filter(None, re.split('[, ]',string))
['blah', 'lots', 'of', 'spaces', 'here']
Voila!
s = 'bla, buu, jii'
sp = []
sp = s.split(',')
for st in sp:
print st
import re
mylist = [x for x in re.compile('\s*[,|\s+]\s*').split(string)]
Simply, comma or at least one white spaces with/without preceding/succeeding white spaces.
Please try!