Develop Reference

Python - How to split a string by non alpha characters

I'm trying to use python to parse lines of c++ source code. The only thing I am interested in is include directives.
#include "header.hpp"
I want it to be flexible and still work with poor coding styles like:
# include"header.hpp"
I have gotten to the point where I can read lines and trim whitespace before and after the #. However I still need to find out what directive it is by reading the string until a non-alpha character is encountered regardless of weather it is a space, quote, tab or angled bracket.
So basically my question is: How can I split a string starting with alphas until a non alpha is encountered?
I think I might be able to do this with regex, but I have not found anything in the documentation that looks like what I want.
Also if anyone has advice on how I would get the file name inside the quotes or angled brackets that would be a plus.

Your instinct on using regex is correct.
import re
re.split('[^a-zA-Z]', string_to_split)
The [^a-zA-Z] part means "not alphabetic characters".

You can do that with a regex. However, you can also use a simple while loop.
def splitnonalpha(s):
pos = 1
while pos < len(s) and s[pos].isalpha():
pos+=1
return (s[:pos], s[pos:])
Test:
>>> splitnonalpha('#include"blah.hpp"')
('#include', '"blah.hpp"')

The two options mentioned by others that are best in my opinion are re.split and re.findall:
>>> import re
>>> re.split(r'\W+', '#include "header.hpp"')
['', 'include', 'header', 'hpp', '']
>>> re.findall(r'\w+', '#include "header.hpp"')
['include', 'header', 'hpp']
A quick benchmark:
>>> setup = "import re; word_pattern = re.compile(r'\w+'); sep_pattern = re.compile(r'\W+')"
>>> iterations = 10**6
>>> timeit.timeit("re.findall(r'\w+', '#header foo bar!')", setup=setup, number=iterations)
3.000092029571533
>>> timeit.timeit("word_pattern.findall('#header foo bar!')", setup=setup, number=iterations)
1.5247418880462646
>>> timeit.timeit("re.split(r'\W+', '#header foo bar!')", setup=setup, number=iterations)
3.786440134048462
>>> timeit.timeit("sep_pattern.split('#header foo bar!')", setup=setup, number=iterations)
2.256173849105835
The functional difference is that re.split keeps empty tokens. That’s usually not useful for tokenization purposes, but the following should be identical to the re.findall solution:
>>> filter(bool, re.split(r'\W+', '#include "header.hpp"'))
['include', 'header', 'hpp']

You can use regex. The \W token will match all non-word characters (which is about the same as non-alphanumeric). Word characters are A-Z, a-z, 0-9, and _. If you want to match underscores as well you could just do [\W_].
>>> import re
>>> line = '# include"header.hpp" '
>>> m = re.match(r'^\s*#\s*include\W+([\w\.]+)\W*$', line)
>>> m.group(1)
'header.hpp'

import re
s = 'foo bar- blah/hm.lala'
print(re.findall(r"\w+",s))
output : ['foo', 'bar', 'blah', 'hm', 'lala']

import re
re.split('[^a-zA-Z0-9]', string_to_split)
for all !(alphanumaric) characters

While not exact, most parse header directives like this
(?m)^\h*#\h*include\h*["<](\w[\w.]*)\h*[">]
Where, (?m) is multi-line mode, \h is horizontal whitespace (aka [^\S\r\n] ).

This works:
import re
test_str = ' # include "header.hpp"'
match = re.match(r'\s*#\s*include\s*("[\w.]*")', test_str)
if match:
print match.group(1)

Python re can't split zero-width anchors? [duplicate]

This question already has answers here:
Python regex: splitting on pattern match that is an empty string
(2 answers)
Closed 5 years ago.
import re
s = 'PythonCookbookListOfContents'
# the first line does not work
print re.split('(?<=[a-z])(?=[A-Z])', s )
# second line works well
print re.sub('(?<=[a-z])(?=[A-Z])', ' ', s)
# it should be ['Python', 'Cookbook', 'List', 'Of', 'Contents']
How to split a string from the border of a lower case character and an upper case character using Python re?
Why does the first line fail to work while the second line works well?

According to re.split:
Note that split will never split a string on an empty pattern match.
For example:
>>> re.split('x*', 'foo')
['foo']
>>> re.split("(?m)^$", "foo\n\nbar\n")
['foo\n\nbar\n']
How about using re.findall instead? (Instead of focusing on separators, focus on the item you want to get.)
>>> import re
>>> s = 'PythonCookbookListOfContents'
>>> re.findall('[A-Z][a-z]+', s)
['Python', 'Cookbook', 'List', 'Of', 'Contents']
UPDATE
Using regex module (Alternative regular expression module, to replace re), you can split on zero-width match:
>>> import regex
>>> s = 'PythonCookbookListOfContents'
>>> regex.split('(?<=[a-z])(?=[A-Z])', s, flags=regex.VERSION1)
['Python', 'Cookbook', 'List', 'Of', 'Contents']
NOTE: Specify regex.VERSION1 flag to enable split-on-zero-length-match behavior.

Python: strip a wildcard word

I have strings with words separated by points.
Example:
string1 = 'one.two.three.four.five.six.eight'
string2 = 'one.two.hello.four.five.six.seven'
How do I use this string in a python method, assigning one word as wildcard (because in this case for example the third word varies). I am thinking of regular expressions, but do not know if the approach like I have it in mind is possible in python.
For example:
string1.lstrip("one.two.[wildcard].four.")
or
string2.lstrip("one.two.'/.*/'.four.")
(I know that I can extract this by split('.')[-3:], but I am looking for a general way, lstrip is just an example)

Use re.sub(pattern, '', original_string) to remove matching part from original_string:
>>> import re
>>> string1 = 'one.two.three.four.five.six.eight'
>>> string2 = 'one.two.hello.four.five.six.seven'
>>> re.sub(r'^one\.two\.\w+\.four', '', string1)
'.five.six.eight'
>>> re.sub(r'^one\.two\.\w+\.four', '', string2)
'.five.six.seven'
BTW, you are misunderstanding str.lstrip:
>>> 'abcddcbaabcd'.lstrip('abcd')
''
str.replace is more appropriate (of course, re.sub, too):
>>> 'abcddcbaabcd'.replace('abcd', '')
'dcba'
>>> 'abcddcbaabcd'.replace('abcd', '', 1)
'dcbaabcd'

Trying to split a string [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Python: Split string with multiple delimiters
I have a small syntax problem. I have a string and another string that has a list of seperators. I need to split it via the .split method.
I can't seem to figure out how, this certainly gives a Type error.
String.split([' ', '{', '='])
How can i split it with multiple seperators?

str.split() only accepts one separator.
Use re.split() to split using a regular expression.
import re
re.split(r"[ {=]", "foo bar=baz{qux")
Output:
['foo', 'bar', 'baz', 'qux']

That's not how the built-in split() method works. It simply uses a single string as the separator, not a list of single-character separators.
You can use regular-expression based splitting, instead. This would probably mean building a regular expression that is the "or" of all your desired delimiters:
splitters = "|".join([" ", "{", "="])
re.split(splitters, my_string)

You can do this with the re (regex) library like so:
import re
result=re.split("[abc]", "my string with characters i want to split")
Where the characters in the square brackets are the characters you want to split with.

Use split from regular expressions instead:
>>> import re
>>> s = 'toto + titi = tata'
>>> re.split('[+=]', s)
['toto ', ' titi ', ' tata']
>>>

import re
string_test = "abc cde{fgh=ijk"
re.split('[\s{=]',string_test)

Split string with multiple delimiters in Python [duplicate]

This question already has answers here:
Split Strings into words with multiple word boundary delimiters
(31 answers)
Closed 8 years ago.
I found some answers online, but I have no experience with regular expressions, which I believe is what is needed here.
I have a string that needs to be split by either a ';' or ', '
That is, it has to be either a semicolon or a comma followed by a space. Individual commas without trailing spaces should be left untouched
Example string:
"b-staged divinylsiloxane-bis-benzocyclobutene [124221-30-3], mesitylene [000108-67-8]; polymerized 1,2-dihydro-2,2,4- trimethyl quinoline [026780-96-1]"
should be split into a list containing the following:
('b-staged divinylsiloxane-bis-benzocyclobutene [124221-30-3]' , 'mesitylene [000108-67-8]', 'polymerized 1,2-dihydro-2,2,4- trimethyl quinoline [026780-96-1]')

Luckily, Python has this built-in :)
import re
re.split('; |, ', string_to_split)
Update:Following your comment:
>>> a='Beautiful, is; better*than\nugly'
>>> import re
>>> re.split('; |, |\*|\n',a)
['Beautiful', 'is', 'better', 'than', 'ugly']

Do a str.replace('; ', ', ') and then a str.split(', ')

Here's a safe way for any iterable of delimiters, using regular expressions:
>>> import re
>>> delimiters = "a", "...", "(c)"
>>> example = "stackoverflow (c) is awesome... isn't it?"
>>> regex_pattern = '|'.join(map(re.escape, delimiters))
>>> regex_pattern
'a|\\.\\.\\.|\\(c\\)'
>>> re.split(regex_pattern, example)
['st', 'ckoverflow ', ' is ', 'wesome', " isn't it?"]
re.escape allows to build the pattern automatically and have the delimiters escaped nicely.
Here's this solution as a function for your copy-pasting pleasure:
def split(delimiters, string, maxsplit=0):
import re
regex_pattern = '|'.join(map(re.escape, delimiters))
return re.split(regex_pattern, string, maxsplit)
If you're going to split often using the same delimiters, compile your regular expression beforehand like described and use RegexObject.split.
If you'd like to leave the original delimiters in the string, you can change the regex to use a lookbehind assertion instead:
>>> import re
>>> delimiters = "a", "...", "(c)"
>>> example = "stackoverflow (c) is awesome... isn't it?"
>>> regex_pattern = '|'.join('(?<={})'.format(re.escape(delim)) for delim in delimiters)
>>> regex_pattern
'(?<=a)|(?<=\\.\\.\\.)|(?<=\\(c\\))'
>>> re.split(regex_pattern, example)
['sta', 'ckoverflow (c)', ' is a', 'wesome...', " isn't it?"]
(replace ?<= with ?= to attach the delimiters to the righthand side, instead of left)

In response to Jonathan's answer above, this only seems to work for certain delimiters. For example:
>>> a='Beautiful, is; better*than\nugly'
>>> import re
>>> re.split('; |, |\*|\n',a)
['Beautiful', 'is', 'better', 'than', 'ugly']
>>> b='1999-05-03 10:37:00'
>>> re.split('- :', b)
['1999-05-03 10:37:00']
By putting the delimiters in square brackets it seems to work more effectively.
>>> re.split('[- :]', b)
['1999', '05', '03', '10', '37', '00']

This is how the regex look like:
import re
# "semicolon or (a comma followed by a space)"
pattern = re.compile(r";|, ")
# "(semicolon or a comma) followed by a space"
pattern = re.compile(r"[;,] ")
print pattern.split(text)