I would like to extract and construct some strings after identifying a substring that matches a pattern contain within square braquets:
e.g: if my text is '2 cups [9 oz] [10 g] flour '
i want to generate 4 strings out of this input:
"2 cups" -> us
"9 oz" -> uk imperial
"10 g" -> metric
"flour" -> ingredient name
As a beginning I have started to identify any square braquet that contains the oz keyword and wrote the following code but the match is not occurring. Any ideas and best practices to accomplish this?
p_oz = re.compile(r'\[(.+) oz\]', re.IGNORECASE) # to match uk metric
text = '2 cups [9 oz] flour'
m = p_oz.match(text)
if m:
found = m.group(1)
print found
You need to use search instead of match.
m = p_oz.search(text)
re.match tries to match the entire input string against the regex. That's not what you want. You want to find a substring that matches your regex, and that's what re.search is for.
I'm just expanding upon BrenBarn's accepted answer. I like a good problem to solve during lunch. Below is my full implementation of your question:
Given the string 2 cups [9 oz] [10 g] flour
import re
text = '2 cups [9 oz] [10 g] flour'
units = {'oz': 'uk imperical',
'cups': 'us',
'g': 'metric'}
# strip out brackets & trim white space
text = text.replace('[', '').replace(']', '').strip()
# replace numbers like 9 to "9
text = re.sub(r'(\d+)', r'"\1', text)
# expand units like `cups` to `cups" -> us`
for unit in units:
text = text.replace(unit, unit + '" -> ' + units[unit] + "~")
# matches the last word in the string
text = re.sub(r'(\w+$)', r'"\1" -> ingredient name', text)
print "raw text: \n" + text + "\n"
print "Array:"
print text.split('~ ')
Will return an array of strings:
raw text:
"2 cups" -> us~ "9 oz" -> uk imperical~ "10 g" -> metric~ "flour" -> ingredient name
Array: [
'"2 cups" -> us',
'"9 oz" -> uk imperical',
'"10 g" -> metric',
'"flour" -> ingredientname'
]
Related
I want to replace all characters in a text by spaces, but I want to leave a list of words.
For instante:
text = "John Thomas bought 300 shares of Acme Corp. in 2006."
list_of_words = ['Acme Corp.', 'John Thomas']
My wanted output would be:
output_text = "*********** ********** "
I would like to change unwanted characters to spaces before I do the * replacement:
"John Thomas Acme Corp. "
Right know I know how to replace only the list of words, but cannot come out with the spaces part.
rep = {key: len(key)*'_**_' for key in list_of_words}
rep = dict((re.escape(k), v) for k, v in rep.items())
pattern = re.compile("|".join(rep.keys()))
pattern.sub(lambda m: rep[re.escape(m.group(0))], text)
You may build a pattern like
(?s)word1|word2|wordN|(.)
When Group 1 matches, replace with a space, else, replace with the same amount of asterisks as the match text length:
import re
text = "John Thomas bought 300 shares of Acme Corp. in 2006."
list_of_words = ['Acme Corp.', 'John Thomas']
pat = "|".join(sorted(map(re.escape, list_of_words), key=len, reverse=True))
pattern = re.compile(f'{pat}|(.)', re.S)
print(pattern.sub(lambda m: " " if m.group(1) else len(m.group(0))*"*", text))
=> '*********** ********** '
See the Python demo
Details
sorted(map(re.escape, list_of_words), key=len, reverse=True) - escapes words in list_of_words and sorts the list by length in descending order (it will be necessary if there are multiword items)
"|".join(...) - build the alternatives out of list_of_words items
lambda m: " " if m.group(1) else len(m.group(0))*"*" - if Group 1 matches, replace with a space, else with the asterisks of the same length as the match length.
In Python, I need to create a regex that inserts a space between any concatenated AlphaNum combinations. For example, this is what I want:
8min15sec ==> 8 min 15 sec
7m12s ==> 7 m 12 s
15mi25s ==> 15 mi 25 s
RegEx101 demo
I am blundering around with solutions found online, but they are a bit too complex for me to parse/modify. For example, I have this:
[a-zA-Z][a-zA-Z\d]*
but it only identifies the first insertion point: 8Xmin15sec (the X)
And this
(?<=[a-z])(?=[A-Z0-9])|(?<=[0-9])(?=[A-Z])
but it only finds this point: 8minX15sec (the X)
I could sure use a hand with the full syntax for finding each insertion point and inserting the spaces.
RegEx101 demo (same link as above)
How about the following approach:
import re
for test in ['8min15sec', '7m12s', '15mi25s']:
print(re.sub(r'(\d+|\D+)', r'\1 ', test).strip())
Which would give you:
8 min 15 sec
7 m 12 s
15 mi 25 s
You can use this regex, which marks the point which are boundaries of numbers and alphabets with either order i.e. number first then alphabets or vice versa.
(?<=\d)(?=[a-zA-Z])|(?<=[a-zA-Z])(?=\d)
This regex (?<=\d)(?=[a-zA-Z]) marks a point with positive lookahead to look for an alphabet and positive look behind to look for a digit.
Similarly, (?<=[a-zA-Z])(?=\d) does same but in opposite order.
And then just replace that mark by a space.
Demo
Here is sample python code for same.
import re
arr = ['8min15sec', '7m12s', '15mi25s']
for s in arr:
print (s + ' --> ' + re.sub('(?<=\d)(?=[a-zA-Z])|(?<=[a-zA-Z])(?=\d)', ' ',s))
Which prints following output,
8min15sec --> 8 min 15 sec
7m12s --> 7 m 12 s
15mi25s --> 15 mi 25 s
How about:
"(\d+)([a-zA-Z]+)"
to
"\1 \2 "
https://regex101.com/r/yvqCtQ/2
And in python:
In [59]: re.sub(r'(\d+)([a-zA-Z]+)', r'\1 \2 ', '8min15sec')
Out[59]: '8 min 15 sec '
I wanna use regular expression to find phrases that contains
1 - One of the N words (any)
2 - All the N words (all )
>>> import re
>>> reg = re.compile(r'.country.|.place')
>>> phrases = ["This is an place", "France is a European country, and a wonderful place to visit", "Paris is a place, it s the capital of the country.side"]
>>> for phrase in phrases:
... found = re.findall(reg,phrase)
... print found
...
Result:
[' place']
[' country,', ' place']
[' place', ' country.']
It seems that I am messing around, I need to specify that I need to find a word, not just a part of word in both cases.
Can anyone help by pointing to the issue ?
Because you are trying to match entire words, use \b to match word boundaries:
reg = re.compile(r'\bcountry\b|\bplace\b')
I have a set of two names with spaces in them. I want to do a regex search for "George Bush" or "Barack Obama". Following this example I tried this, which gets the desired output
p = "(George\sBush|Barack\sObama)"
s = "recent Presidents George Bush and Barack Obama"
print re.findall(p,s) #Prints George Bush and Barack Obama
However, now I want to go from a list ["George Bush", "Barack Obama"] to the pattern shown above.
I tried this:
for l in list:
p = p + "|" + l
p = p.strip("|")
p = ('.{75}(' + p + ').{75}').replace(" ", "\s")
But it gives : '.{75}(George\\sBush|Barack\\sObama).{75}'
How can I replace space characters with just "\s" instead of "\\s"?
You already have. The backslash is special and must be escaped in the representation (and should be escaped in the string), but you really do have "\s". Try printing the string instead.
:) I tried using w = Word(printables), but it isn't working. How should I give the spec for this. 'w' is meant to process Hindi characters (UTF-8)
The code specifies the grammar and parses accordingly.
671.assess :: अहसास ::2
x=number + "." + src + "::" + w + "::" + number + "." + number
If there is only english characters it is working so the code is correct for the ascii format but the code is not working for the unicode format.
I mean that the code works when we have something of the form
671.assess :: ahsaas ::2
i.e. it parses words in the english format, but I am not sure how to parse and then print characters in the unicode format. I need this for English Hindi word alignment for purpose.
The python code looks like this:
# -*- coding: utf-8 -*-
from pyparsing import Literal, Word, Optional, nums, alphas, ZeroOrMore, printables , Group , alphas8bit ,
# grammar
src = Word(printables)
trans = Word(printables)
number = Word(nums)
x=number + "." + src + "::" + trans + "::" + number + "." + number
#parsing for eng-dict
efiledata = open('b1aop_or_not_word.txt').read()
eresults = x.parseString(efiledata)
edict1 = {}
edict2 = {}
counter=0
xx=list()
for result in eresults:
trans=""#translation string
ew=""#english word
xx=result[0]
ew=xx[2]
trans=xx[4]
edict1 = { ew:trans }
edict2.update(edict1)
print len(edict2) #no of entries in the english dictionary
print "edict2 has been created"
print "english dictionary" , edict2
#parsing for hin-dict
hfiledata = open('b1aop_or_not_word.txt').read()
hresults = x.scanString(hfiledata)
hdict1 = {}
hdict2 = {}
counter=0
for result in hresults:
trans=""#translation string
hw=""#hin word
xx=result[0]
hw=xx[2]
trans=xx[4]
#print trans
hdict1 = { trans:hw }
hdict2.update(hdict1)
print len(hdict2) #no of entries in the hindi dictionary
print"hdict2 has been created"
print "hindi dictionary" , hdict2
'''
#######################################################################################################################
def translate(d, ow, hinlist):
if ow in d.keys():#ow=old word d=dict
print ow , "exists in the dictionary keys"
transes = d[ow]
transes = transes.split()
print "possible transes for" , ow , " = ", transes
for word in transes:
if word in hinlist:
print "trans for" , ow , " = ", word
return word
return None
else:
print ow , "absent"
return None
f = open('bidir','w')
#lines = ["'\
#5# 10 # and better performance in business in turn benefits consumers . # 0 0 0 0 0 0 0 0 0 0 \
#5# 11 # vHyaapaar mEmn bEhtr kaam upbhOkHtaaomn kE lIe laabhpHrdd hOtaa hAI . # 0 0 0 0 0 0 0 0 0 0 0 \
#'"]
data=open('bi_full_2','rb').read()
lines = data.split('!##$%')
loc=0
for line in lines:
eng, hin = [subline.split(' # ')
for subline in line.strip('\n').split('\n')]
for transdict, source, dest in [(edict2, eng, hin),
(hdict2, hin, eng)]:
sourcethings = source[2].split()
for word in source[1].split():
tl = dest[1].split()
otherword = translate(transdict, word, tl)
loc = source[1].split().index(word)
if otherword is not None:
otherword = otherword.strip()
print word, ' <-> ', otherword, 'meaning=good'
if otherword in dest[1].split():
print word, ' <-> ', otherword, 'trans=good'
sourcethings[loc] = str(
dest[1].split().index(otherword) + 1)
source[2] = ' '.join(sourcethings)
eng = ' # '.join(eng)
hin = ' # '.join(hin)
f.write(eng+'\n'+hin+'\n\n\n')
f.close()
'''
if an example input sentence for the source file is:
1# 5 # modern markets : confident consumers # 0 0 0 0 0
1# 6 # AddhUnIk baajaar : AshHvsHt upbhOkHtaa . # 0 0 0 0 0 0
!##$%
the ouptut would look like this :-
1# 5 # modern markets : confident consumers # 1 2 3 4 5
1# 6 # AddhUnIk baajaar : AshHvsHt upbhOkHtaa . # 1 2 3 4 5 0
!##$%
Output Explanation:-
This achieves bidirectional alignment.
It means the first word of english 'modern' maps to the first word of hindi 'AddhUnIk' and vice versa. Here even characters are take as words as they also are an integral part of bidirectional mapping. Thus if you observe the hindi WORD '.' has a null alignment and it maps to nothing with respect to the English sentence as it doesn't have a full stop.
The 3rd line int the output basically represents a delimiter when we are working for a number of sentences for which your trying to achieve bidirectional mapping.
What modification should i make for it to work if the I have the hindi sentences in Unicode(UTF-8) format.
Pyparsing's printables only deals with strings in the ASCII range of characters. You want printables in the full Unicode range, like this:
unicodePrintables = u''.join(unichr(c) for c in xrange(sys.maxunicode)
if not unichr(c).isspace())
Now you can define trans using this more complete set of non-space characters:
trans = Word(unicodePrintables)
I was unable to test against your Hindi test string, but I think this will do the trick.
(If you are using Python 3, then there is no separate unichr function, and no xrange generator, just use:
unicodePrintables = ''.join(chr(c) for c in range(sys.maxunicode)
if not chr(c).isspace())
EDIT:
With the recent release of pyparsing 2.3.0, new namespace classes have been defined to give printables, alphas, nums, and alphanums for various Unicode language ranges.
import pyparsing as pp
pp.Word(pp.pyparsing_unicode.printables)
pp.Word(pp.pyparsing_unicode.Devanagari.printables)
pp.Word(pp.pyparsing_unicode.देवनागरी.printables)
As a general rule, do not process encoded bytestrings: make them into proper unicode strings (by calling their .decode method) as soon as possible, do all of your processing always on unicode strings, then, if you have to for I/O purposes, .encode them back into whatever bytestring encoding you require.
If you're talking about literals, as it seems you are in your code, the "as soon as possible" is at once: use u'...' to express your literals. In a more general case, where you're forced to do I/O in encoded form, it's immediately after input (just as it's immediately before output if you need to perform output in a specific encoded form).
I Was searching about french unicode chars and fall on this question. If you search french or other latin accents, with pyparsing 2.3.0 you can use:
>>> pp.pyparsing_unicode.Latin1.alphas
'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyzªµºÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõöøùúûüýþÿ'