I want to split a string into words on white-spaces or any special character. But, if the word before AND after the split contains a number, and it is not a white-space character, then I DON'T want it to split.
"abc abc-def a2b-def a2b-d3f"
Should become - (notice the last word)
"abc", " ", "abc", "-", "def", " ", "a2b", "-", "def", " ", "a2b-d3f"
I tried
b = "abc abc-def a2b-def a2b-d3f ab2-3cd"
print(re.split(r"((?<=\D)[\W]|[\W](?=\D)|\s)",b))
print(re.split(r"((?<!\b\w*\d\w*\b)[\W]|[\W](?!\b\w*\d\w*\b)|\s)",b))
The first one sort of works, but it only considers the last and first character of the previous or next word respectively. It maintained "ab2-3cd" as a single word, but it wouldn't work for "a2b-c3d".
The second one gives me an error "look-behind requires fixed-width pattern" because it doesn't allow me to use * in look-back or look-ahead.
Please help me out!
EDIT: the words can be of arbitrary length, "abcdef".
You can grab all patterns matching ptrn r'\w+|\W+' from the words that match the pattern r'\d\w*\W+\w*\d'
>>> import re
>>> txt = "abc abc-def a2b-def a2b-d3f"
>>> [w for s in txt.split() for w in ([s] if re.search(r'\d\w*\W+\w*\d', s) else re.findall(r'\w+|\W+', s)) + [' ']]
['abc', ' ', 'abc', '-', 'def', ' ', 'a2b', '-', 'def', ' ', 'a2b-d3f', ' ']
import re
s = "abc abc-def a2b-def a2b-d3f"
s = re.split(r'(?:(?<=[\da-z]{3})(\s|-)(?=[a-z]{3})|(?:(?<=[a-z]{3})(\s|-)(?=[a-z\d]{3})))', s)
s = [i for i in s if i is not None]
print(s)
Prints:
['abc', ' ', 'abc', '-', 'def', ' ', 'a2b', '-', 'def', ' ', 'a2b-d3f']
EDIT:
import re
s = "a2dc abc axx2b-dss3f abc-def a2b-abc a2b-d3f"
s = re.split(r'(\s|-)(?=[a-z]+(?:-|\s))', s)
out = []
for w in s:
out.extend(re.split(r'(?<=[a-z\d])(\s)(?=[a-z\d])', w))
print(out)
Prints:
['a2dc', ' ', 'abc', ' ', 'axx2b-dss3f', ' ', 'abc', '-', 'def', ' ', 'a2b', '-', 'abc', ' ', 'a2b-d3f']
Might it be possible to split a Python string (sentence) so it retains the whitespaces between words in the output, but within a split substring by appending it after each word?
For example:
given_string = 'This is my string!'
output = ['This ', 'is ', 'my ', 'string!']
I avoid regexes most of the time, but here it makes it really simple:
import re
given_string = 'This is my string!'
res = re.findall(r'\w+\W?', given_string)
# res ['This ', 'is ', 'my ', 'string!']
Maybe this will help?
>>> given_string = 'This is my string!'
>>> l = given_string.split(' ')
>>> l = [item + ' ' for item in l[:-1]] + l[-1:]
>>> l
['This ', 'is ', 'my ', 'string!']
just split and add the whitespace back:
a = " "
output = [e+a for e in given_string.split(a) if e]
output[len(output)-1] = output[len(output)-1][:-1]
the last line is for deleting space after thankyou!
say I have a text file with a format similarly to this
Q: hello what is your name?
A: Hi my name is John Smith
and I want to create a matrix such that it is a 2xn in this case
[['hello','what','is',your','name','?', ' '],['hi','my','name','is','John','Smith']]
note that the first row has an empty entry because it has 6 strings while second row has 7 strings
You can use re.split:
import re
file_data = open('filename.txt').read()
results = filter(None, re.split('A:\s|Q:\s', file_data))
new_results = [re.findall('\w+|\W', i) for i in results]
Output:
[['hello', ' ', 'what', ' ', 'is', ' ', 'your', ' ', 'name', '?', ' '], ['Hi', ' ', 'my', ' ', 'name', ' ', 'is', ' ', 'John', ' ', 'Smith']]
just split the strings, using the split function:
with open('txt.txt') as my_file:
lines = my_file.readlines()
#lines[0] = "Q: hello what is your name?"
#lines[1] = "A: Hi my name is John Smith"
then just use
output = [lines[0].split,lines[1].split]
I am creating some code that will replace spaces.
I want a double space to turn into a single space and a single space to become nothing.
Example:
string = "t e s t t e s t"
string = string.replace(' ', ' ').replace(' ', '')
print (string)
The output is "testest" because it replaces all the spaces.
How can I make the output "test test"?
Thanks
A regular expression approach is doubtless possible, but for a quick solution, first split on the double space, then rejoin on a single space after using a comprehension to remove the single spaces in each of the elements in the split:
>>> string = "t e s t t e s t"
>>> ' '.join(word.replace(' ', '') for word in string.split(' '))
'test test'
Just another idea:
>>> s = 't e s t t e s t'
>>> s.replace(' ', ' ').replace(' ', '').replace(' ', '')
'test test'
Seems to be faster:
>>> timeit(lambda: s.replace(' ', ' ').replace(' ', '').replace(' ', ''))
2.7822862677683133
>>> timeit(lambda: ' '.join(w.replace(' ','') for w in s.split(' ')))
7.702567737466012
And regex (at least this one) is shorter but a lot slower:
>>> timeit(lambda: re.sub(' ( ?)', r'\1', s))
37.2261058654488
I like this regex solution because you can easily read what's going on:
>>> import re
>>> string = "t e s t t e s t"
>>> re.sub(' {1,2}', lambda m: '' if m.group() == ' ' else ' ', string)
'test test'
We search for one or two spaces, and substitute one space with the empty string but two spaces with a single space.
I need to split strings of data using each character from string.punctuation and string.whitespace as a separator.
Furthermore, I need for the separators to remain in the output list, in between the items they separated in the string.
For example,
"Now is the winter of our discontent"
should output:
['Now', ' ', 'is', ' ', 'the', ' ', 'winter', ' ', 'of', ' ', 'our', ' ', 'discontent']
I'm not sure how to do this without resorting to an orgy of nested loops, which is unacceptably slow. How can I do it?
A different non-regex approach from the others:
>>> import string
>>> from itertools import groupby
>>>
>>> special = set(string.punctuation + string.whitespace)
>>> s = "One two three tab\ttabandspace\t end"
>>>
>>> split_combined = [''.join(g) for k, g in groupby(s, lambda c: c in special)]
>>> split_combined
['One', ' ', 'two', ' ', 'three', ' ', 'tab', '\t', 'tabandspace', '\t ', 'end']
>>> split_separated = [''.join(g) for k, g in groupby(s, lambda c: c if c in special else False)]
>>> split_separated
['One', ' ', 'two', ' ', 'three', ' ', 'tab', '\t', 'tabandspace', '\t', ' ', 'end']
Could use dict.fromkeys and .get instead of the lambda, I guess.
[edit]
Some explanation:
groupby accepts two arguments, an iterable and an (optional) keyfunction. It loops through the iterable and groups them with the value of the keyfunction:
>>> groupby("sentence", lambda c: c in 'nt')
<itertools.groupby object at 0x9805af4>
>>> [(k, list(g)) for k,g in groupby("sentence", lambda c: c in 'nt')]
[(False, ['s', 'e']), (True, ['n', 't']), (False, ['e']), (True, ['n']), (False, ['c', 'e'])]
where terms with contiguous values of the keyfunction are grouped together. (This is a common source of bugs, actually -- people forget that they have to sort by the keyfunc first if they want to group terms which might not be sequential.)
As #JonClements guessed, what I had in mind was
>>> special = dict.fromkeys(string.punctuation + string.whitespace, True)
>>> s = "One two three tab\ttabandspace\t end"
>>> [''.join(g) for k,g in groupby(s, special.get)]
['One', ' ', 'two', ' ', 'three', ' ', 'tab', '\t', 'tabandspace', '\t ', 'end']
for the case where we were combining the separators. .get returns None if the value isn't in the dict.
import re
import string
p = re.compile("[^{0}]+|[{0}]+".format(re.escape(
string.punctuation + string.whitespace)))
print p.findall("Now is the winter of our discontent")
I'm no big fan of using regexps for all problems, but I don't think you have much choice in this if you want it fast and short.
I'll explain the regexp since you're not familiar with it:
[...] means any of the characters inside the square brackets
[^...] means any of the characters not inside the square brackets
+ behind means one or more of the previous thing
x|y means to match either x or y
So the regexp matches 1 or more characters where either all must be punctuation and whitespace, or none must be. The findall method finds all non-overlapping matches of the pattern.
Try this:
import re
re.split('(['+re.escape(string.punctuation + string.whitespace)+']+)',"Now is the winter of our discontent")
Explanation from the Python documentation:
If capturing parentheses are used in pattern, then the text of all groups in the pattern are also returned as part of the resulting list.
Solution in linear (O(n)) time:
Let's say you have a string:
original = "a, b...c d"
First convert all separators to space:
splitters = string.punctuation + string.whitespace
trans = string.maketrans(splitters, ' ' * len(splitters))
s = original.translate(trans)
Now s == 'a b c d'. Now you can use itertools.groupby to alternate between spaces and non-spaces:
result = []
position = 0
for _, letters in itertools.groupby(s, lambda c: c == ' '):
letter_count = len(list(letters))
result.append(original[position:position + letter_count])
position += letter_count
Now result == ['a', ', ', 'b', '...', 'c', ' ', 'd'], which is what you need.
My take:
from string import whitespace, punctuation
import re
pattern = re.escape(whitespace + punctuation)
print re.split('([' + pattern + '])', 'now is the winter of')
Depending on the text you are dealing with, you may be able to simplify your concept of delimiters to "anything other than letters and numbers". If this will work, you can use the following regex solution:
re.findall(r'[a-zA-Z\d]+|[^a-zA-Z\d]', text)
This assumes that you want to split on each individual delimiter character even if they occur consecutively, so 'foo..bar' would become ['foo', '.', '.', 'bar']. If instead you expect ['foo', '..', 'bar'], use [a-zA-Z\d]+|[^a-zA-Z\d]+ (only difference is adding + at the very end).
from string import punctuation, whitespace
s = "..test. and stuff"
f = lambda s, c: s + ' ' + c + ' ' if c in punctuation else s + c
l = sum([reduce(f, word).split() for word in s.split()], [])
print l
For any arbitrary collection of separators:
def separate(myStr, seps):
answer = []
temp = []
for char in myStr:
if char in seps:
answer.append(''.join(temp))
answer.append(char)
temp = []
else:
temp.append(char)
answer.append(''.join(temp))
return answer
In [4]: print separate("Now is the winter of our discontent", set(' '))
['Now', ' ', 'is', ' ', 'the', ' ', 'winter', ' ', 'of', ' ', 'our', ' ', 'discontent']
In [5]: print separate("Now, really - it is the winter of our discontent", set(' ,-'))
['Now', ',', '', ' ', 'really', ' ', '', '-', '', ' ', 'it', ' ', 'is', ' ', 'the', ' ', 'winter', ' ', 'of', ' ', 'our', ' ', 'discontent']
Hope this helps
from itertools import chain, cycle, izip
s = "Now is the winter of our discontent"
words = s.split()
wordsWithWhitespace = list( chain.from_iterable( izip( words, cycle([" "]) ) ) )
# result : ['Now', ' ', 'is', ' ', 'the', ' ', 'winter', ' ', 'of', ' ', 'our', ' ', 'discontent', ' ']