>>> import re
>>> p = re.compile('.*&l=(.*)(&|$)')
>>> p.search('foo&l=something here&bleh').group(1)
'something here&bleh' # want to remove strings after &
>>> p.search('foo&l=something here').group(1)
'something here' # this is OK
The python documents (2.7) says that the or operator '|' is never greedy. But my codes has not been working fine. I want the regex to stop searching when it reached the next & instead going through the entire string.
You need change .* inside the first capturing group to [^&]*
p = re.compile('.*&l=([^&]*)')
Your regex p = re.compile('.*&l=(.*)(&|$)') matches also the extra chars because .* inside the first capturing group is greedy which matches all the chars upto the last. You all know $ matches the boundary which exists at the last. Hence finds a match.
So .* and then $ finds a match, so it won't get backtarck.
Your regex tries to match everything (.*), then when it reaches the end of the string, it begins to backtrack until it matches &. That's why you are getting that result.
Change your regex to
.*&l=(.*?)(&|$)
Adding the ? will make your regex lazy.
Simple example that demonstrate the issue:
Let's say you want to match everything until the first % character appears, and let's say you write the following regex:
.*%
Let's see how the engine works given the string "abc%def%g".
It first see .*, will try to consume everything, so it'll match the whole string. But then, it tries to match % and fails, so it backtracks to the previous character, it's g, still no match. Will backtrack again, and then it reaches %, it does match! So you'll get abc%def% as a result.
Related
I'm trying to find any text between a '>' character and a new line, so I came up with this regex:
result = re.search(">(.*)\n", text).group(1)
It works perfectly with only one result, such as:
>test1
(something else here)
Where the result, as intended, is
test1
But whenever there's more than one result, it only shows the first one, like in:
>test1
(something else here)
>test2
(something else here)
Which should give something like
test1\ntest2
But instead just shows
test1
What am I missing? Thank you very much in advance.
re.search only returns the first match, as documented:
Scan through string looking for the first location where the regular
expression pattern produces a match, and return a corresponding
MatchObject instance.
To find all the matches, use findall.
Return all non-overlapping matches of pattern in string, as a list of
strings. The string is scanned left-to-right, and matches are returned
in the order found.
Here's an example from the shell:
>>> import re
>>> re.findall(">(.*)\n", ">test1\nxxx>test2\nxxx")
['test1', 'test2']
Edit: I just read your question again and realised that you want "test1\ntest2" as output. Well, just join the list with \n:
>>> "\n".join(re.findall(">(.*)\n", ">test1\nxxx>test2\nxxx"))
'test1\ntest2'
You could try:
y = re.findall(r'((?:(?:.+?)(?:(?=[\n\r][^\n\r])\n|))+)', text)
Which returns ['t1\nt2\nt3'] for 't1\nt2\nt3\n'. If you simply want the string, you can get it by:
s = y[0]
Although it seems much larger than your initial code, it will give you your desired string.
Explanation -
((?:(?:.+?)(?:(?=[\n\r][^\n\r])\n|))+) is the regex as well as the match.
(?:(?:.+?)(?:(?=[\n\r][^\n\r])\n|)) is the non-capturing group that matches any text followed by a newline, and is repeatedly found one-or-more times by the + after it.
(?:.+?) matches the actual words which are then followed by a newline.
(?:(?=[\n\r][^\n\r])\n|) is a non-capturing conditional group which tells the regex that if the matched text is followed by a newline, then it should match it, provided that the newline is not followed by another newline or carriage return
(?=[\n\r][^\n\r]) is a positive look-ahead which ascertains that the text found is followed by a newline or carriage return, and then some non-newline characters, which combined with the \n| after it, tells the regex to match a newline.
Granted, after typing this big mess out, the regex is pretty long and complicated, so you would be better off implementing the answers you understand, rather than this answer, which you may not. However, this seems to be the only one-line answer to get the exact output you desire.
I have some Python code that involves a lot of re.sub() commands. In some cases, I want to replace a character but only if it comes after certain other characters. The following is an example of how I currently am doing this in python:
secStress = "[aeiou],"[-1]
So my input for this would be a string like "a,s I walk, I hum." And I want to replace that first comma but not the "a" that comes before it.
The problem is that Python doesn't like when I give it a variable as input for re.sub(). Is there a way I can write a regex that specifies that only its final character is supposed to be matched?
You are looking for either a capturing group/backreference or a positive lookbehind solution:
s = "a,s I walk, I hum."
# Capturing group / backreference
print(re.sub(r"([aeiou]),", r"\1", s))
# Positive lookbehind
print(re.sub(r"(?<=[aeiou]),", "", s))
See the Python demo.
First approach details
The ([aeiou]) is a capturing group that matches a vowel and stores it in a special memory buffer that you can refer to from the replacement pattern using backreferences. Here, the Group ID is 1, so you can access that value using r"\1".
Second approach details
The (?<=[aeiou]) is a positive lookbehind that only checks (but does not add the text to the match value) if there is a vowel immediately before the current position. So, only those commas are matched that are preceded with a vowel and it is enough to replace with an empty string to get rid of the comma since it is the only symbol kept in the match.
If I understand you correctly,
>>> import re
>>> def doit(matchobj):
... return matchobj.group()[0]
...
>>> re.sub(r'[aeiou],', doit, "a,s I walk, I hum.")
'as I walk, I hum.'
If the regex matches then doit is called with the object that matched. Whatever string doit returns (and it must be a string) is put in place of the match.
I am using python 2.6 and trying to find a bunch of repeating characters in a string, let's say a bunch of n's, e.g. nnnnnnnABCnnnnnnnnnDEF. In any place of the string the number of n's can be variable.
If I construct a regex like this:
re.findall(r'^(((?i)n)\2{2,})', s),
I can find occurences of case-insensitive n's only in the beginning of the string, which is fine. If I do it like this:
re.findall(r'(((?i)n)\2{2,}$)', s),
I can detect the ones only in the end of the sequence. But what about just in the middle?
At first, I thought of using re.findall(r'(((?i)n)\2{2,})', s) and the two previous regex(-ices?) to check the length of the returned list and the presence of n's either in the beginning or end of the string and make logical tests, but it became an ugly if-else mess very quickly.
Then, I tried re.findall(r'(?!^)(((?i)n)\2{2,})', s), which seems to exlude the beginning just fine but (?!$) or (?!\z) at the end of the regex only excludes the last n in ABCnnnn. Finally, I tried re.findall(r'(?!^)(((?i)n)\2{2,})\w+', s) which seems to work sometimes, but I get weird results at others. It feels like I need a lookahead or lookbehind, but I can't wrap my head around them.
Instead of using a complicated regex in order to refuse of matching the leading and trailing n characters. As a more pythonic approach you can strip() your string then find all the sequence of ns using re.findall() and a simple regex:
>>> s = "nnnABCnnnnDEFnnnnnGHInnnnnn"
>>> import re
>>>
>>> re.findall(r'n{2,}', s.strip('n'), re.I)
['nnnn', 'nnnnn']
Note : re.I is Ignore-case flag which makes the regex engine matches upper case and lower case characters.
Since "n" is a character (and not a subpattern), you can simply use:
re.findall(r'(?<=[^n])nn+(?=[^n])(?i)', s)
or better:
re.findall(r'n(?<=[^n]n)n+(?=[^n])(?i)', s)
NOTE: This solution assumes n may be a sequence of some characters. For more efficient alternatives when n is just 1 character, see other answers here.
You can use
(?<!^)(?<!n)((n)\2{2,})(?!$)(?!n)
See the regex demo
The regex will match repeated consecutive ns (ignoring case can be achieved with re.I flag) that are not at the beginning ((?<!^)) or end ((?!$)) of the string and not before ((?!n)) or after ((?<!n)) another n.
The (?<!^)(?<!n) is a sequence of 2 lookbehinds: (?<!^) means do not consume the next pattern if preceded with the start of the string. The (?<!n) negative lookbehind means do not consume the next pattern if preceded with n. The negative lookaheads (?!$) and (?!n)have similar meanings: (?!$) fails a match if after the current position the end of string occurs and (?!n) will fail a match if n occurs after the current position in string (that is, right after matching all consecutive ns. The lookaround conditions must all be met, that is why we only get the innermost matches.
See IDEONE demo:
import re
p = re.compile(r'(?<!^)(?<!n)((n)\2{2,})(?!$)(?!n)', re.IGNORECASE)
s = "nnnnnnnABCnnnnnNnnnnDEFnNn"
print([x.group() for x in p.finditer(s)])
I'm trying to use python re to find a set of the same letter or number repeated a specific number of times. (.) works just fine for identifying what will be repeated, but I cannot find how to keep it from just repeating different characters. here is what I have:
re.search(r'(.){n}', str)
so for example it would match 9999 from 99997 if n = 4, but not if n = 3.
thanks
How about
(?:^|(?<=(.)))(?!\1)(.)\2{n-1}(?!\2)
This will:
(?:^|(?<=(.))): Make sure that:
^: Either we are at the beginning of the string
(?<=(.)): Either we are not at the beginning of the string; then, capture the character before the match and save it into \1
(?!\1)(.): Match any character that is not \1 and save it into \2
\2{n-1}: Match \2 n-1 times
(?!\2): Make sure \2 cannot be matched looking forward
(The n-1 is only symbolic; obviously you want to replace this with the actual value of n-1, not with 8-1 or something).
Important edit: The previous version of the regex ((.)\1{n-1}(?!\1)) does not work because it fails to account for character matching \1 behind the match. The regex above fixes this problem.
Given is the following python script:
text = '<?xml version="1.24" encoding="utf-8">'
mu = (".??[?]?[?]", "....")
for item in mu:
print item,":",re.search(item, text).group()
Can someone please explain why the first hit with the regex .??[?]?[?] returns <? instead of just ?.
My explaination:
.?? should match nothing as .? can match or not any char and the second ? makes it not greedy.
[?]? can match ? or not, so nothing is good, too
[?] just matches ?
That should result in ? and not in <?
For the same reason o*?bar matches oobar in foobar. Even if the quantifier is non-greedy the regex will try to match from the first char in all possible ways, before moving on to the next.
First the .?? matches an empty string, but when the regex engine backtracks to it, it matches <, thus making the rest of the regex match, without moving the start position of the match to the next character.
Regex "greediness" only affects backtracking; it doesn't mean that the regex engine will skip earlier potential match points — a regex always takes the first possible match. In this case, that means <? because it starts farther to the left than ?.