I have a string s with nested brackets: s = "AX(p>q)&E((-p)Ur)"
I want to remove all characters between all pairs of brackets and store in a new string like this: new_string = AX&E
i tried doing this:
p = re.compile("\(.*?\)", re.DOTALL)
new_string = p.sub("", s)
It gives output: AX&EUr)
Is there any way to correct this, rather than iterating each element in the string?
Another simple option is removing the innermost parentheses at every stage, until there are no more parentheses:
p = re.compile("\([^()]*\)")
count = 1
while count:
s, count = p.subn("", s)
Working example: http://ideone.com/WicDK
You can just use string manipulation without regular expression
>>> s = "AX(p>q)&E(qUr)"
>>> [ i.split("(")[0] for i in s.split(")") ]
['AX', '&E', '']
I leave it to you to join the strings up.
>>> import re
>>> s = "AX(p>q)&E(qUr)"
>>> re.compile("""\([^\)]*\)""").sub('', s)
'AX&E'
Yeah, it should be:
>>> import re
>>> s = "AX(p>q)&E(qUr)"
>>> p = re.compile("\(.*?\)", re.DOTALL)
>>> new_string = p.sub("", s)
>>> new_string
'AX&E'
Nested brackets (or tags, ...) are something that are not possible to handle in a general way using regex. See http://www.amazon.de/Mastering-Regular-Expressions-Jeffrey-Friedl/dp/0596528124/ref=sr_1_1?ie=UTF8&s=gateway&qid=1304230523&sr=8-1-spell for details why. You would need a real parser.
It's possible to construct a regex which can handle two levels of nesting, but they are already ugly, three levels will already be quite long. And you don't want to think about four levels. ;-)
You can use PyParsing to parse the string:
from pyparsing import nestedExpr
import sys
s = "AX(p>q)&E((-p)Ur)"
expr = nestedExpr('(', ')')
result = expr.parseString('(' + s + ')').asList()[0]
s = ''.join(filter(lambda x: isinstance(x, str), result))
print(s)
Most code is from: How can a recursive regexp be implemented in python?
You could use re.subn():
import re
s = 'AX(p>q)&E((-p)Ur)'
while True:
s, n = re.subn(r'\([^)(]*\)', '', s)
if n == 0:
break
print(s)
Output
AX&E
this is just how you do it:
# strings
# double and single quotes use in Python
"hey there! welcome to CIP"
'hey there! welcome to CIP'
"you'll understand python"
'i said, "python is awesome!"'
'i can\'t live without python'
# use of 'r' before string
print(r"\new code", "\n")
first = "code in"
last = "python"
first + last #concatenation
# slicing of strings
user = "code in python!"
print(user)
print(user[5]) # print an element
print(user[-3]) # print an element from rear end
print(user[2:6]) # slicing the string
print(user[:6])
print(user[2:])
print(len(user)) # length of the string
print(user.upper()) # convert to uppercase
print(user.lstrip())
print(user.rstrip())
print(max(user)) # max alphabet from user string
print(min(user)) # min alphabet from user string
print(user.join([1,2,3,4]))
input()
Related
What is the best way to use Regex to extract and transform one statement to another?
Specifically, I have implemented the below to find and extract a sudent number from a block of text and transform it as follows: AB123CD to AB-123-CD
Right now, this is implemented as 3 statements as follows:
gg['student_num'] = gg['student_test'].str.extract('(\d{2})\w{3}\d{2}') + \
'-' + gg['student_num'].str.extract('\d{2}(\w{3})\d{2}') + \
'-' + gg['student_test'].str.extract('\d{2}\w{3}(\d{2})')
It doesn't feel right to me that I would need to have three statements -
one for each group - concatenated together below (or even more if this was more complicated) and wondered if there was a better way to find and transform some text?
You could get list of segments using regexp and then join them this way:
'-'.join(re.search(r'(\d{2})(\w{3})(\d{2})', string).groups())
You could get AttributeError if string doesn't contain needed pattern (re.search() returns None), so you might want to wrap this expression in try...except block.
This is not regex, but it is quick and concise:
s = "AB123CD"
first = [i for i, a in enumerate(s) if a.isdigit()][0]
second = [i for i, a in enumerate(s) if a.isdigit()][-1]
new_form = s[:first]+"-"+s[first:second+1]+"-"+s[second+1:]
Output:
AB-123-CD
Alternative regex solution:
letters = re.findall("[a-zA-Z]+", s)
numbers = re.findall("[0-9]+", s)
letters.insert(1, numbers[0])
final = '-'.join(letters)
print(final)
Output:
AB-123-CD
Try this. Hope that helps
>>> import re
>>> s = r'ABC123DEF'
>>> n = re.search(r'\d+',s).group()
>>> f = re.findall(r'[A-Za-z]+',s)
>>> new_s = f[0]+"-"+n+"-"+f[1]
>>> new_s
Output:
'ABC-123-DEF'
I call a function that returns code with all kinds of characters ranging from ( to ", and , and numbers.
Is there an elegant way to remove all of these so I end up with nothing but letters?
Given
s = '##24A-09=wes()&8973o**_##me' # contains letters 'Awesome'
You can filter out non-alpha characters with a generator expression:
result = ''.join(c for c in s if c.isalpha())
Or filter with filter:
result = ''.join(filter(str.isalpha, s))
Or you can substitute non-alpha with blanks using re.sub:
import re
result = re.sub(r'[^A-Za-z]', '', s)
A solution using RegExes is quite easy here:
import re
newstring = re.sub(r"[^a-zA-Z]+", "", string)
Where string is your string and newstring is the string without characters that are not alphabetic. What this does is replace every character that is not a letter by an empty string, thereby removing it. Note however that a RegEx may be slightly overkill here.
A more functional approach would be:
newstring = "".join(filter(str.isalpha, string))
Unfortunately you can't just call stron a filterobject to turn it into a string, that would look much nicer...
Going the pythonic way it would be
newstring = "".join(c for c in string if c.isalpha())
You didn't mention you want only english letters, here's an international solution:
import unicodedata
str = u"hello, ѱϘяԼϷ!"
print ''.join(c for c in str if unicodedata.category(c).startswith('L'))
Here's another one, using string.ascii_letters
>>> import string
>>> "".join(x for x in s if x in string.ascii_letters)
`
>>> import re
>>> string = "';''';;';1123123!##!##!#!$!sd sds2312313~~\"~s__"
>>> re.sub("[\W\d_]", "", string)
'sdsdss'
Well, I use this for myself in this kind of situations
Sorry, if it's outdated :)
string = "The quick brown fox jumps over the lazy dog!"
alphabet = "abcdefghijklmnopqrstuvwxyz"
def letters_only(source):
result = ""
for i in source.lower():
if i in alphabet:
result += i
return result
print(letters_only(string))
s = '##24A-09=wes()&8973o**_##me'
print(filter(str.isalpha, s))
# Awesome
About return value of filter:
filter(function or None, sequence) -> list, tuple, or string
Assume I have a string as follows: expression = '123 + 321'.
I am walking over the string character-by-character as follows: for p in expression. I am I am checking if p is a digit using p.isdigit(). If p is a digit, I'd like to grab the whole number (so grab 123 and 321, not just p which initially would be 1).
How can I do that in Python?
In C (coming from a C background), the equivalent would be:
int x = 0;
sscanf(p, "%d", &x);
// the full number is now in x
EDIT:
Basically, I am accepting a mathematical expression from a user that accepts positive integers, +,-,*,/ as well as brackets: '(' and ')'. I am walking the string character by character and I need to be able to determine whether the character is a digit or not. Using isdigit(), I can that. If it is a digit however, I need to grab the whole number. How can that be done?
>>> from itertools import groupby
>>> expression = '123 + 321'
>>> expression = ''.join(expression.split()) # strip whitespace
>>> for k, g in groupby(expression, str.isdigit):
if k: # it's a digit
print 'digit'
print list(g)
else:
print 'non-digit'
print list(g)
digit
['1', '2', '3']
non-digit
['+']
digit
['3', '2', '1']
This is one of those problems that can be approached from many different directions. Here's what I think is an elegant solution based on itertools.takewhile:
>>> from itertools import chain, takewhile
>>> def get_numbers(s):
... s = iter(s)
... for c in s:
... if c.isdigit():
... yield ''.join(chain(c, takewhile(str.isdigit, s)))
...
>>> list(get_numbers('123 + 456'))
['123', '456']
This even works inside a list comprehension:
>>> def get_numbers(s):
... s = iter(s)
... return [''.join(chain(c, takewhile(str.isdigit, s)))
... for c in s if c.isdigit()]
...
>>> get_numbers('123 + 456')
['123', '456']
Looking over other answers, I see that this is not dissimilar to jamylak's groupby solution. I would recommend that if you don't want to discard the extra symbols. But if you do want to discard them, I think this is a bit simpler.
The Python documentation includes a section on simulating scanf, which gives you some idea of how you can use regular expressions to simulate the behavior of scanf (or sscanf, it's all the same in Python). In particular, r'\-?\d+' is the Python string that corresponds to the regular expression for an integer. (r'\d+' for a nonnegative integer.) So you could embed this in your loop as
integer = re.compile(r'\-?\d+')
for p in expression:
if p.isdigit():
# somehow find the current position in the string
integer.match(expression, curpos)
But that still reflects a very C-like way of thinking. In Python, your iterator variable p is really just an individual character that has actually been pulled out of the original string and is standing on its own. So in the loop, you don't naturally have access to the current position within the string, and trying to calculate it is going to be less than optimal.
What I'd suggest instead is using Python's built in regexp matching iteration method:
integer = re.compile(r'\-?\d+') # only do this once in your program
all_the_numbers = integer.findall(expression)
and now all_the_numbers is a list of string representations of all the integers in the expression. If you wanted to actually convert them to integers, then you could do this instead of the last line:
all_the_numbers = [int(s) for s in integer.finditer(expression)]
Here I've used finditer instead of findall because you don't have to make a list of all the strings before iterating over them again to convert them to integers.
Though I'm not familiar with sscanf, I'm no C developer, it looks like it's using format strings in a way not dissimilar to what I'd use python's re module for. Something like this:
import re
nums = re.compile('\d+')
found = nums.findall('123 + 321')
# if you know you're only looking for two values.
left, right = found
You can use shlex http://docs.python.org/library/shlex.html
>>> from shlex import shlex
>>> expression = '123 + 321'
>>> for e in shlex(expression):
... print e
...
123
+
321
>>> expression = '(92831 * 948) / 32'
>>> for e in shlex(expression):
... print e
...
(
92831
*
948
)
/
32
I'd split the string up on the ' + ' string, giving you what's outside of them:
>>> expression = '123 + 321'
>>> ex = expression.split(' + ')
>>> ex
['123', '321']
>>> int_ex = map(int, ex)
>>> int_ex
[123, 321]
>>> sum(int_ex)
444
It's dangerous, but you could use eval:
>>> eval('123 + 321')
444
I'm just taking a stab at you parsing the string, and doing raw calculations on it.
e_array = expression.split('+')
i_array = map(int, e_array)
And i_array holds all integers in the expression.
UPDATE
If you already know all the special characters in your expression and you want to eliminate them all
import re
e_array = re.split('[*/+\-() ]', expression) # all characters here is mult, div, plus, minus, left- right- parathesis and space
i_array = map(int, filter(lambda x: len(x), e_array))
I need help in trying to write a certain part of a program.
The idea is that a person would input a bunch of gibberish and the program will read it till it reaches an "!" (exclamation mark) so for example:
input("Type something: ")
Person types: wolfdo65gtornado!salmontiger223
If I ask the program to print the input it should only print wolfdo65gtornado and cut anything once it reaches the "!" The rest of the program is analyzing and counting the letters, but those part I already know how to do. I just need help with the first part. I been trying to look through the book but it seems I'm missing something.
I'm thinking, maybe utilizing a for loop and then placing restriction on it but I can't figure out how to make the random imputed string input be analyzed for a certain character and then get rid of the rest.
If you could help, I'll truly appreciate it. Thanks!
The built-in str.partition() method will do this for you. Unlike str.split() it won't bother to cut the rest of the str into different strs.
text = raw_input("Type something:")
left_text = text.partition("!")[0]
Explanation
str.partition() returns a 3-tuple containing the beginning, separator, and end of the string. The [0] gets the first item which is all you want in this case. Eg.:
"wolfdo65gtornado!salmontiger223".partition("!")
returns
('wolfdo65gtornado', '!', 'salmontiger223')
>>> s = "wolfdo65gtornado!salmontiger223"
>>> s.split('!')[0]
'wolfdo65gtornado'
>>> s = "wolfdo65gtornadosalmontiger223"
>>> s.split('!')[0]
'wolfdo65gtornadosalmontiger223'
if it doesnt encounter a "!" character, it will just grab the entire text though. if you would like to output an error if it doesn't match any "!" you can just do like this:
s = "something!something"
if "!" in s:
print "there is a '!' character in the context"
else:
print "blah, you aren't using it right :("
You want itertools.takewhile().
>>> s = "wolfdo65gtornado!salmontiger223"
>>> '-'.join(itertools.takewhile(lambda x: x != '!', s))
'w-o-l-f-d-o-6-5-g-t-o-r-n-a-d-o'
>>> s = "wolfdo65gtornado!salmontiger223!cvhegjkh54bgve8r7tg"
>>> i = iter(s)
>>> '-'.join(itertools.takewhile(lambda x: x != '!', i))
'w-o-l-f-d-o-6-5-g-t-o-r-n-a-d-o'
>>> '-'.join(itertools.takewhile(lambda x: x != '!', i))
's-a-l-m-o-n-t-i-g-e-r-2-2-3'
>>> '-'.join(itertools.takewhile(lambda x: x != '!', i))
'c-v-h-e-g-j-k-h-5-4-b-g-v-e-8-r-7-t-g'
Try this:
s = "wolfdo65gtornado!salmontiger223"
m = s.index('!')
l = s[:m]
To explain accepted answer.
Splitting
partition() function splits string in list with 3 elements:
mystring = "123splitABC"
x = mystring.partition("split")
print(x)
will give:
('123', 'split', 'ABC')
Access them like list elements:
print (x[0]) ==> 123
print (x[1]) ==> split
print (x[2]) ==> ABC
Suppose we have:
s = "wolfdo65gtornado!salmontiger223" + some_other_string
s.partition("!")[0] and s.split("!")[0] are both a problem if some_other_string contains a million strings, each a million characters long, separated by exclamation marks. I recommend the following instead. It's much more efficient.
import itertools as itts
get_start_of_string = lambda stryng, last, *, itts=itts:\
str(itts.takewhile(lambda ch: ch != last, stryng))
###########################################################
s = "wolfdo65gtornado!salmontiger223"
start_of_string = get_start_of_string(s, "!")
Why the itts=itts
Inside of the body of a function, such as get_start_of_string, itts is global.
itts is evaluated when the function is called, not when the function is defined.
Consider the following example:
color = "white"
get_fleece_color = lambda shoop: shoop + ", whose fleece was as " + color + " as snow."
print(get_fleece_color("Igor"))
# [... many lines of code later...]
color = "pink polka-dotted"
print(get_fleece_color("Igor's cousin, 3 times removed"))
The output is:
Igor, whose fleece was white as snow.
Igor's cousin, 3 times removed Igor, whose fleece was as pink polka-dotted as snow.
You can extract the beginning of a string, up until the first delimiter is encountered, by using regular expressions.
import re
slash_if_special = lambda ch:\
"\\" if ch in "\\^$.|?*+()[{" else ""
prefix_slash_if_special = lambda ch, *, _slash=slash_if_special: \
_slash(ch) + ch
make_pattern_from_char = lambda ch, *, c=prefix_slash_if_special:\
"^([^" + c(ch) + "]*)"
def get_string_up_untill(x_stryng, x_ch):
i_stryng = str(x_stryng)
i_ch = str(x_ch)
assert(len(i_ch) == 1)
pattern = make_pattern_from_char(ch)
m = re.match(pattern, x_stryng)
return m.groups()[0]
An example of the code above being used:
s = "wolfdo65gtornado!salmontiger223"
result = get_string_up_untill(s, "!")
print(result)
# wolfdo65gtornado
We can use itertools
s = "wolfdo65gtornado!salmontiger223"
result = "".join(itertools.takewhile(lambda x : x!='!' , s))
>>"wolfdo65gtornado"
I want to remove any brackets from a string. Why doesn't this work properly?
>>> name = "Barack (of Washington)"
>>> name = name.strip("(){}<>")
>>> print name
Barack (of Washington
Because that's not what strip() does. It removes leading and trailing characters that are present in the argument, but not those characters in the middle of the string.
You could do:
name= name.replace('(', '').replace(')', '').replace ...
or:
name= ''.join(c for c in name if c not in '(){}<>')
or maybe use a regex:
import re
name= re.sub('[(){}<>]', '', name)
I did a time test here, using each method 100000 times in a loop. The results surprised me. (The results still surprise me after editing them in response to valid criticism in the comments.)
Here's the script:
import timeit
bad_chars = '(){}<>'
setup = """import re
import string
s = 'Barack (of Washington)'
bad_chars = '(){}<>'
rgx = re.compile('[%s]' % bad_chars)"""
timer = timeit.Timer('o = "".join(c for c in s if c not in bad_chars)', setup=setup)
print "List comprehension: ", timer.timeit(100000)
timer = timeit.Timer("o= rgx.sub('', s)", setup=setup)
print "Regular expression: ", timer.timeit(100000)
timer = timeit.Timer('for c in bad_chars: s = s.replace(c, "")', setup=setup)
print "Replace in loop: ", timer.timeit(100000)
timer = timeit.Timer('s.translate(string.maketrans("", "", ), bad_chars)', setup=setup)
print "string.translate: ", timer.timeit(100000)
Here are the results:
List comprehension: 0.631745100021
Regular expression: 0.155561923981
Replace in loop: 0.235936164856
string.translate: 0.0965719223022
Results on other runs follow a similar pattern. If speed is not the primary concern, however, I still think string.translate is not the most readable; the other three are more obvious, though slower to varying degrees.
string.translate with table=None works fine.
>>> name = "Barack (of Washington)"
>>> name = name.translate(None, "(){}<>")
>>> print name
Barack of Washington
Because strip() only strips trailing and leading characters, based on what you provided. I suggest:
>>> import re
>>> name = "Barack (of Washington)"
>>> name = re.sub('[\(\)\{\}<>]', '', name)
>>> print(name)
Barack of Washington
strip only strips characters from the very front and back of the string.
To delete a list of characters, you could use the string's translate method:
import string
name = "Barack (of Washington)"
table = string.maketrans( '', '', )
print name.translate(table,"(){}<>")
# Barack of Washington
Since strip only removes characters from start and end, one idea could be to break the string into list of words, then remove chars, and then join:
s = 'Barack (of Washington)'
x = [j.strip('(){}<>') for j in s.split()]
ans = ' '.join(j for j in x)
print(ans)
For example string s="(U+007c)"
To remove only the parentheses from s, try the below one:
import re
a=re.sub("\\(","",s)
b=re.sub("\\)","",a)
print(b)