Is it possible to perform simple math on the output from Python regular expressions?
I have a large file where I need to divide numbers following a ")" by 100. For instance, I would convert the following line containing )75 and )2:
((words:0.23)75:0.55(morewords:0.1)2:0.55);
to )0.75 and )0.02:
((words:0.23)0.75:0.55(morewords:0.1)0.02:0.55);
My first thought was to use re.sub using the search expression "\)\d+", but I don't know how to divide the integer following the parenthesis by 100, or if this is even possible using re.
Any thoughts on how to solve this? Thanks for your help!
You can do it by providing a function as the replacement:
s = "((words:0.23)75:0.55(morewords:0.1)2:0.55);"
s = re.sub("\)(\d+)", lambda m: ")" + str(float(m.groups()[0]) / 100), s)
print s
# ((words:0.23)0.75:0.55(morewords:0.1)0.02:0.55);
Incidentally, if you wanted to do it using BioPython's Newick tree parser instead, it would look like this:
from Bio import Phylo
# assuming you want to read from a string rather than a file
from StringIO import StringIO
tree = Phylo.read(StringIO(s), "newick")
for c in tree.get_nonterminals():
if c.confidence != None:
c.confidence = c.confidence / 100
print tree.format("newick")
(while this particular operation takes more lines than the regex version, other operations involving trees might be made much easier with it).
The replacement expression for re.sub can be a function. Write a function that takes the matched text, converts it to a number, divides it by 100, and then returns the string form of the result.
Related
I have to create a unique textual marker in my document using Python 2.7, with the following function:
def build_textual_marker(number, id):
return "[xxxcixxx[[_'" + str(number) + "'] [_'" + id + "']]xxxcixxx]"
the output looks like this : [xxxcixxx[[_'1'] [_'24']]xxxcixxx]
And then I have to catch any occurrence of this expression in my document. I ended up to the following regular expression but it seems not working fine:
marker_regex = "\[xxxcixxx\[(\[_*?\])\s(\[_*?\])\]xxxcixxx\]"
I was wondering how should I write the correct regex in this case?
Try using
\[xxxcixxx\[\[_'.*?'\] \[_'.*?'\]\]xxxcixxx\]
Demo: http://regexr.com/3d887
Rather than the lazy star, you might as well get along with a digit class directly (the function build_textual_marker takes a number parameter, doesn't it?):
\[xxxcixxx\[(\[_'\d+'\])\s(\[_'\d+'\])\]xxxcixxx\]
See a demo on regex101.com.
Is there a way to search for a substring in a larger string, and then return a different substring x places left or right of the original substring?
I want to look through a string like "blahblahlink:"www.example.com"blahblah for the string "link:" and return the subsequent url.
Thanks!
Python 3, if that matters.
I think you should use regular expressions. There is module called re in python for that.
Pure Python solution would be to use index which tells you where in the string the first match occurs, then use the [start:end] slicing notation to select the string from that point:
"blahblahlink:www.example.comblahblah".index("link")
# returns 8
i = "blahblahlink:www.example.comblahblah".index("link")
"blahblahlink:www.example.comblahblah"[i+5:i+20]
# returns 'www.example.com'
Ive got a string in this format
a = "[a,b,c],[e,d,f],[g,h,i]"
Each part I want to be split is separated by ],[. I tried a.split("],[") and I get the end brackets removed.
In my example that would be:
["[a,b,c","e,d,f","g,h,i]"]
I was wondering if there was a way to keep the brackets after the split?
Desired outcome:
["[a,b,c]","[e,d,f]","[g,h,i]"]
The problem is that str.split removes whatever substring you split on from the resulting list. I think it would be better in this case to use the slightly more powerful split function from the re module:
>>> from re import split
>>> a = "[a,b,c],[e,d,f],[g,h,i]"
>>> split(r'(?<=\]),(?=\[)', a)
['[a,b,c]', '[e,d,f]', '[g,h,i]']
>>>
(?<=\]) is a lookbehind assertion which looks for ]. Similarly, (?=\[) is a lookahead assertion which looks for [. Both constructs are explained in Regular Expression Syntax.
Python is very flexible, so you just have to manage it a bit and be adaptive to your case.
In [8]:a = "[a,b,c],[e,d,f],[g,h,i]"
a.replace('],[','] [').split(" ")
Out[8]:['[a,b,c]', '[e,d,f]', '[g,h,i]']
The other answers are correct, but here is another way to go.
Important note: this is just to present another option that may prove useful in certain cases. Don't do it in the general case, and do so only in you're absolutely certain that you have the control over the expression you're passing into exec statement.
# provided you declared a, b, c, d, e, f, g, h, i beforehand
>>> exp = "[a,b,c],[e,d,f],[g,h,i]"
>>> exec("my_object = " + exp)
>>> my_object
([a,b,c],[e,d,f],[g,h,i])
Then, you can do whatever you like with my_object.
Provided that you have full control over exp, this way of doing sounds more appropriate and Pythonic to me because you are treating a piece of Python code written in a string as a... piece of Python code written in a string (hence the exec statement). Without manipulating it through regexp or artificial hacks.
Just keep in mind that it can be dangerous.
I'm writing my first script and trying to learn python.
But I'm stuck and can't get out of this one.
I'm writing a script to change file names.
Lets say I have a string = "this.is.tEst3.E00.erfeh.ervwer.vwtrt.rvwrv"
I want the result to be string = "This Is Test3 E00"
this is what I have so far:
l = list(string)
//Transform the string into list
for i in l:
if "E" in l:
p = l.index("E")
if isinstance((p+1), int () is True:
if isinstance((p+2), int () is True:
delp = p+3
a = p-3
del l[delp:]
new = "".join(l)
new = new.replace("."," ")
print (new)
get in index where "E" and check if after "E" there are 2 integers.
Then delete everything after the second integer.
However this will not work if there is an "E" anyplace else.
at the moment the result I get is:
this is tEst
because it is finding index for the first "E" on the list and deleting everything after index+3
I guess my question is how do I get the index in the list if a combination of strings exists.
but I can't seem to find how.
thanks for everyone answers.
I was going in other direction but it is also not working.
if someone could see why it would be awesome. It is much better to learn by doing then just coping what others write :)
this is what I came up with:
for i in l:
if i=="E" and isinstance((i+1), int ) is True:
p = l.index(i)
print (p)
anyone can tell me why this isn't working. I get an error.
Thank you so much
Have you ever heard of a Regular Expression?
Check out python's re module. Link to the Docs.
Basically, you can define a "regex" that would match "E and then two integers" and give you the index of it.
After that, I'd just use python's "Slice Notation" to choose the piece of the string that you want to keep.
Then, check out the string methods for str.replace to swap the periods for spaces, and str.title to put them in Title Case
An easy way is to use a regex to find up until the E followed by 2 digits criteria, with s as your string:
import re
up_until = re.match('(.*?E\d{2})', s).group(1)
# this.is.tEst3.E00
Then, we replace the . with a space and then title case it:
output = up_until.replace('.', ' ').title()
# This Is Test3 E00
The technique to consider using is Regular Expressions. They allow you to search for a pattern of text in a string, rather than a specific character or substring. Regular Expressions have a bit of a tough learning curve, but are invaluable to learn and you can use them in many languages, not just in Python. Here is the Python resource for how Regular Expressions are implemented:
http://docs.python.org/2/library/re.html
The pattern you are looking to match in your case is an "E" followed by two digits. In Regular Expressions (usually shortened to "regex" or "regexp"), that pattern looks like this:
E\d\d # ('\d' is the specifier for any digit 0-9)
In Python, you create a string of the regex pattern you want to match, and pass that and your file name string into the search() method of the the re module. Regex patterns tend to use a lot of special characters, so it's common in Python to prepend the regex pattern string with 'r', which tells the Python interpreter not to interpret the special characters as escape characters. All of this together looks like this:
import re
filename = 'this.is.tEst3.E00.erfeh.ervwer.vwtrt.rvwrv'
match_object = re.search(r'E\d\d', filename)
if match_object:
# The '0' means we want the first match found
index_of_Exx = match_object.end(0)
truncated_filename = filename[:index_of_Exx]
# Now take care of any more processing
Regular expressions can get very detailed (and complex). In fact, you can probably accomplish your entire task of fully changing the file name using a single regex that's correctly put together. But since I don't know the full details about what sorts of weird file names might come into your program, I can't go any further than this. I will add one more piece of information: if the 'E' could possibly be lower-case, then you want to add a flag as a third argument to your pattern search which indicates case-insensitive matching. That flag is 're.I' and your search() method would look like this:
match_object = re.search(r'E\d\d', filename, re.I)
Read the documentation on Python's 're' module for more information, and you can find many great tutorials online, such as this one:
http://www.zytrax.com/tech/web/regex.htm
And before you know it you'll be a superhero. :-)
The reason why this isn't working:
for i in l:
if i=="E" and isinstance((i+1), int ) is True:
p = l.index(i)
print (p)
...is because 'i' contains a character from the string 'l', not an integer. You compare it with 'E' (which works), but then try to add 1 to it, which errors out.
I have been googling this one fervently, but I can't really narrow it down. I am attempting to interpret a csv file of values, common enough sort of behaviour. But I am being punished by values over a thousand, i.e. in quotations and involving a comma. I have kinda gotten around it by using the csv reader, which creates a list of numbers from the row, but I then have to pick the commas out afterwards.
For purely academic reasons, is there a better way to edit a string with regular expressions? Going from 08/09/2010,"25,132","2,909",650 to 08/09/2010,25132,2909,650.
(If you are into Vim, basically I want to put Python on this:
:1,$s/"\([0-9]*\),\([0-9]*\)"/\1\2/g :D )
Use the csv module for first-stage parsing, and a regex only for seeing if the result can be transformed to a number.
import csv, re
num_re = re.compile('^[0-9]+[0-9,]+$')
for row in csv.reader(open('input_file.csv')):
for el_num in len(row):
if num_re.match(row[el_num]):
row[el_num] = row[el_num].replace(',', '')
...although it would probably be faster not to use the regular expression at all:
for row in ([item.replace(',', '') for item in row]
for row in csv.reader(open('input_file.csv'))):
do_something_with_your(row)
I think what you're looking for is, assuming that commas will only appear in numbers, and that those entries will always be quoted:
import re
def remove_commas(mystring):
return re.sub(r'"(\d+?),(\d+?)"', r'\1\2', mystring)
UPDATE:
Adding cdarke's comments below, the following should work for arbitrary-length numbers:
import re
def remove_commas_and_quotes(mystring):
return re.sub(r'","|",|"', ',', re.sub(r'(?:(\d+?),)',r'\1',mystring))
Python has a regular expressions module, "re":
http://docs.python.org/library/re.html
However, in this case, you might want to consider using the "partition" function:
>>> s = 'some_long_string,"12,345",more_string,"56,6789",and_some_more'
>>> left_part,quote_mark,right_part = s.partition(")
>>> right_part
'12,345",more_string,"56,6789",and_some_more'
>>> number,quote_mark,remainder = right_part.partition(")
'12,345'
string.partition("character") splits a string into 3 parts, stuff to the left of the first occurrence of "character", "character" itself and stuff to the right.
Here's a simple regex for removing commas from numbers of any length:
re.sub(r'(\d+),?([\d+]?)',r'\1\2',mystring)