Given a list of string L, and a sublist M from L. I want to find an efficient way to find the index span (index_a, index_b) from L where L[index_a: index_b] = M.
For example,
L = ['Clement', 'Joseph', 'Man', '##ey', 'was', 'an', 'American', 'businessman', 'from', 'Boston', 'who', 'was', 'president', 'of', 'a', 'contracting', 'company', 'and', 'a', 'minority', 'owner', 'and', 'treasurer', 'of', 'the', 'Boston', 'Braves', 'baseball', 'team', '.']
M = ['Clement', 'Joseph', 'Man', '##ey']
Expected_result = (0,4) # since L[0:4] = M
L = ['Clement', 'Joseph', 'Man', '##ey', 'was', 'an', 'American', 'businessman', 'from', 'Boston', 'who', 'was', 'president', 'of', 'a', 'contracting', 'company', 'and', 'a', 'minority', 'owner', 'and', 'treasurer', 'of', 'the', 'Boston', 'Braves', 'baseball', 'team', '.']
M = ['Boston']
Expected_result = (9,9) # both (9,9) and (25,25) are correct since L[9]=L[25]=M
Use a comprehension,and check the length when return the value:
def getIndex(lst1, lst2):
index1 = next((i for i in range(len(lst1) - len(lst2)) if lst1[i:i + len(lst2)] == lst2), None)
if index1 is not None: # when sub list doesn't exist in another list
return (index1, index1) if len(lst2) == 1 else (index1, index1+len(lst2))
else:
return None
L1 = ['Clement', 'Joseph', 'Man', '##ey', 'was', 'an', 'American', 'businessman', 'from', 'Boston', 'who', 'was',
'president', 'of', 'a', 'contracting', 'company', 'and', 'a', 'minority', 'owner', 'and', 'treasurer', 'of', 'the',
'Boston', 'Braves', 'baseball', 'team', '.']
M1 = ['Clement', 'Joseph', 'Man', '##ey']
L2 = ['Clement', 'Joseph', 'Man', '##ey', 'was', 'an', 'American', 'businessman', 'from', 'Boston', 'who', 'was', 'president', 'of', 'a', 'contracting', 'company', 'and', 'a', 'minority', 'owner', 'and', 'treasurer', 'of', 'the', 'Boston', 'Braves', 'baseball', 'team', '.']
M2 = ['Boston']
print(getIndex(L1, M1))
print(getIndex(L2, M2))
Result:
(0, 4)
(9, 9)
Or one line:
def getIndex(lst1, lst2):
return next(((i, i + (len(lst2) if len(lst2) > 1 else 0)) for i in range(len(lst1) - len(lst2)) if lst1[i:i + len(lst2)] == lst2), None)
def getIndices(L,M):
results = []
for i in range(len(L) - len(M)):
if L[i:i+len(M)] == M:
results.append((i, i+len(M)))
return results
Traversing the list L in blocks of len(M)and checking if it's equal to the list M
Related
Let us say I have the following tokens in list :
['I', 'want', 'to', 'learn', 'coding', 'in', 'r', 'and', 'c++', 'today', ',', 'and', 'then', 'I', "'ll", 'be', 'learning', 'c#', 'and', 'c']
I want to remove all tokens of length < 2 but want to keep elements of this sub-list:
['r','c','js','c#']
How can I do this in a single python list-comprehension ?
You can use list comprehension with two conditions.
lst1 = ['I', 'want', 'to', 'learn', 'coding', 'in', 'r', 'and', 'c++', 'today', ',', 'and', 'then', 'I', "'ll", 'be', 'learning', 'c#', 'and', 'c']
lst2 = ['r','c','js','c#']
res = [l for l in lst1 if (len(l)>=2) or (l in lst2)]
print(res)
['want',
'to',
'learn',
'coding',
'in',
'r',
'and',
'c++',
'today',
'and',
'then',
"'ll",
'be',
'learning',
'c#',
'and',
'c']
I have a number of sentences which I would like to split on specific words (e.g. and). However, when splitting the sentences sometimes there are two or more combinations of a word I'd like to split on in a sentence.
Example sentences:
['i', 'am', 'just', 'hoping', 'for', 'strength', 'and', 'guidance', 'because', 'i', 'have', 'no', 'idea', 'why']
['maybe', 'that', 'is', 'why', 'he', 'does', 'not', 'come', 'home', 'and', 'tell', 'you', 'how', 'good', 'his', 'day', 'at', 'work', 'was', 'because', 'he', 'is', 'been', 'told', 'not', 'to', 'talk']
so I have written some code to split a sentence:
split_on_word = []
no_splitting = []
indexPosList = [ i for i in range(len(kth)) if kth[i] == 'and'] # check if word is in sentence
for e in example:
kth = e.split() # split strings into list so it looks like example sentence
for n in indexPosList:
if n > 4: # only split when the word's position is 4 or more
h = e.split("and")
for i in h:
split_on_word.append(i)# append split sentences
else:
no_splitting.append(kth) #append sentences that don't need to be split
However, you can see that when using this code more than once (e.g.: replace the word to split on with another) I will create duplicates or part duplicates of the sentences that I append to a new list.
Is there any way to check for multiple conditions, so that if a sentence contains both or other combinations of it that I split the sentence in one go?
The output from the examples should then look like this:
['i', 'am', 'just', 'hoping', 'for', 'strength']
['guidance', 'because']
['i', 'have', 'no', 'idea', 'why']
['maybe', 'that', 'is', 'why', 'he', 'does', 'not', 'come', 'home']
[ 'tell', 'you', 'how', 'good', 'his', 'day', 'at', 'work', 'was']
['he', 'is', 'been', 'told', 'not', 'to', 'talk']
You can use itertools.groupby with a function that checks whether a word is a split-word:
In [11]: split_words = {'and', 'because'}
In [12]: [list(g) for k, g in it.groupby(example, key=lambda x: x not in split_words) if k]
Out[12]:
[['maybe', 'that', 'is', 'why', 'he', 'does', 'not', 'come', 'home'],
['tell', 'you', 'how', 'good', 'his', 'day', 'at', 'work', 'was'],
['he', 'is', 'been', 'told', 'not', 'to', 'talk']]
I lemmatised several sentences, and it turns out the results like this,this is for the first two sentences.
['She', 'be', 'start', 'on', 'Levofloxacin', 'but', 'the', 'patient', 'become', 'hypotensive', 'at', 'that', 'point', 'with', 'blood', 'pressure', 'of', '70/45', 'and', 'receive', 'a', 'normal', 'saline', 'bolus', 'to', 'boost', 'her', 'blood', 'pressure', 'to', '99/60', ';', 'however', 'the', 'patient', 'be', 'admit', 'to', 'the', 'Medical', 'Intensive', 'Care', 'Unit', 'for', 'overnight', 'observation', 'because', 'of', 'her', 'somnolence', 'and', 'hypotension', '.', '11', '.', 'History', 'of', 'hemoptysis', ',', 'on', 'Coumadin', '.', 'There', 'be', 'ST', 'scoop', 'in', 'the', 'lateral', 'lead', 'consistent', 'with', 'Dig', 'vs.', 'a', 'question', 'of', 'chronic', 'ischemia', 'change', '.']
which all the words are generated together like a list. but i need them to be like sentence by sentence, the output format would be better like this:
['She be start on Levofloxacin but the patient become hypotensive at that point with blood pressure of 70/45 and receive a normal saline bolus to boost her blood pressure to 99/60 ; however the patient be admit to the Medical Intensive Care Unit for overnight observation because of her somnolence and hypotension .','11 . History of hemoptysis , on Coumadin .','There be ST scoop in the lateral lead consistent with Dig vs. a question of chronic ischemia change .']
can anyone help me please? thanks a lot
Try this code:
final = []
sentence = []
for word in words:
if word in ['.']: # and whatever other punctuation marks you want to use.
sentence.append(word)
final.append(' '.join(sentence))
sentence = []
else:
sentence.append(word)
print (final)
Hope this helps! :)
A good starting point might be str.join():
>>> wordsList = ['She', 'be', 'start', 'on', 'Levofloxacin']
>>> ' '.join(wordsList)
'She be start on Levofloxacin'
words=['She', 'be', 'start', 'on', 'Levofloxacin', 'but', 'the', 'patient', 'become', 'hypotensive', 'at', 'that', 'point', 'with', 'blood', 'pressure', 'of', '70/45', 'and', 'receive', 'a', 'normal', 'saline', 'bolus', 'to', 'boost', 'her', 'blood', 'pressure', 'to', '99/60', ';', 'however', 'the', 'patient', 'be', 'admit', 'to', 'the', 'Medical', 'Intensive', 'Care', 'Unit', 'for', 'overnight', 'observation', 'because', 'of', 'her', 'somnolence', 'and', 'hypotension', '.', '11', '.', 'History', 'of', 'hemoptysis', ',', 'on', 'Coumadin', '.', 'There', 'be', 'ST', 'scoop', 'in', 'the', 'lateral', 'lead', 'consistent', 'with', 'Dig', 'vs.', 'a', 'question', 'of', 'chronic', 'ischemia', 'change', '.']
def Wordify(words,sen_lim):
Array=[]
word=""
sen_len=0
for w in words:
word+=w+" "
if(w.isalnum()):
sen_len+=1
if(w=="." and sen_len>sen_lim):
Array.append(word)
word=""
sen_len=0
return(Array)
print(Wordify(words,5))
Basically you append the characters to a new string and separate out sentence if there is a period , but also ensure that the current sentence has a minimum number of words. This ensures sentences that like "11." are avoided.
sen_lim
is a parameter you could tune according to your convenience.
You can try string concatenation by looping through the list
list1 = ['She', 'be', 'start', 'on', 'Levofloxacin', 'but', 'the',
'patient', 'become', 'hypotensive', 'at', 'that', 'point', 'with', 'blood',
'pressure', 'of', '70/45', 'and', 'receive', 'a', 'normal', 'saline',
'bolus', 'to', 'boost', 'her', 'blood', 'pressure', 'to', '99/60', ';',
'however', 'the', 'patient', 'be', 'admit', 'to', 'the', 'Medical',
'Intensive', 'Care', 'Unit', 'for', 'overnight', 'observation', 'because',
'of', 'her', 'somnolence', 'and', 'hypotension', '.', '11', '.', 'History',
'of', 'hemoptysis', ',', 'on', 'Coumadin', '.', 'There', 'be', 'ST',
'scoop', 'in', 'the', 'lateral', 'lead', 'consistent', 'with', 'Dig', 'vs.',
'a', 'question', 'of', 'chronic', 'ischemia', 'change', '.']
list2 = []
string = ""
for element in list1:
if(string == "" or element == "."):
string = string + element
else:
string = string + " " + element
list2.append(string)
print(list2)
You could try this:
# list of words.
words = ['This', 'is', 'a', 'sentence', '.']
def sentence_from_list(words):
sentence = ""
# iterate the list and append to the string.
for word in words:
sentence += word + " "
result = [sentence]
# print the result.
print result
sentence_from_list(words)
You may need to delete the last space, just before the '.'
I have some "big" lists, that fit into some other lists of the same list of the "big" lists, and I would like to map the sublists into one another, to get the one containing all the others. Its sorted by lenghth of the items first.
lists = [['Linden'], ['Linden', 'N.J.'], ['chemical', 'plants'], ['some', 'federal', 'officials'], ['the', 'other', 'side', 'of', 'the', 'country'], ['the', 'most', 'dangerous', 'two', 'miles', 'in', 'America'], ['that', 'some', 'federal', 'officials', 'refer', 'to', 'as', 'the', 'most', 'dangerous', 'two', 'miles', 'in', 'America'], ['an', 'industrial', 'corridor', 'of', 'chemical', 'plants', 'that', 'some', 'federal', 'officials', 'refer', 'to', 'as', 'the', 'most', 'dangerous', 'two', 'miles', 'in', 'America'], ['At', 'the', 'other', 'side', 'of', 'the', 'country', 'Linden', 'N.J.', 'is', 'part', 'of', 'an', 'industrial', 'corridor', 'of', 'chemical', 'plants', 'and', 'oil', 'refineries', 'that', 'some', 'federal', 'officials', 'refer', 'to', 'as', 'the', 'most', 'dangerous', 'two', 'miles', 'in', 'America']]
and the result should be:
['At', ['the', 'other', 'side', 'of', 'the', 'country'], [['Linden'], 'N.J.'], 'is', 'part', 'of', ['an', 'industrial', 'corridor', 'of', ['chemical', 'plants'], 'and', 'oil', 'refineries', 'that', ['some', 'federal', 'officials'], 'refer', 'to', 'as', ['the', 'most', 'dangerous', 'two', 'miles', 'in', 'America']]]
How to solve that?
I found my own solution, after some inhibited attempts to ask that on stackoverflow. So here is it:
If you have optimization ideas, I would be happy about this.
I would like to use some iterator instead of the recursive function, like here (iterate python nested lists efficiently), but in that trial I had, it flattened the nesting of the list.
from more_itertools import replace
import collections
nestlist = [['Linden'],['Linden', 'N.J.'], ['chemical', 'plants'], ['some', 'federal', 'officials'], ['the', 'other', 'side', 'of', 'the', 'country'], ['the', 'most', 'dangerous', 'two', 'miles', 'in', 'America'], ['that', 'some', 'federal', 'officials', 'refer', 'to', 'as', 'the', 'most', 'dangerous', 'two', 'miles', 'in', 'America'], ['an', 'industrial', 'corridor', 'of', 'chemical', 'plants', 'that', 'some', 'federal', 'officials', 'refer', 'to', 'as', 'the', 'most', 'dangerous', 'two', 'miles', 'in', 'America'], ['At', 'the', 'other', 'side', 'of', 'the', 'country', 'Linden', 'N.J.', 'is', 'part', 'of', 'an', 'industrial', 'corridor', 'of', 'chemical', 'plants', 'and', 'oil', 'refineries', 'that', 'some', 'federal', 'officials', 'refer', 'to', 'as', 'the', 'most', 'dangerous', 'two', 'miles', 'in', 'America']]
#nestlist = [['Linden', 'N.J.'], ['chemical', 'plants'], ['country', 'Linden', 'N.J.', 'of', 'chemical', 'plants']]
#nestlist = [['Linden', 'N.J.'], ['Linden', 'N.J.', 'of'], ['country', 'Linden', 'N.J.', 'of', 'chemical', 'plants']]
def flatten(iterable):
for el in iterable:
if isinstance(el, collections.Iterable) and not isinstance(el, str):
yield from flatten(el)
else:
yield el
def on_each_iterable (lol, fun):
out = []
if isinstance(lol, collections.Iterable) and not isinstance(lol, str):
for x in lol:
out.append(on_each_iterable(x, fun))
out = fun(out)
else:
out = lol
return out
def stack_matrjoshka(nesting_list):
nesting_list = sorted(nesting_list, key=lambda x: len(x))
n = 0
while n < (len(nesting_list) - 2):
to_fit_there = nesting_list[n]
flatted_to_fit_there = list(flatten(to_fit_there[:]))
def is_fitting(*xs):
flatted_compared = list(flatten(xs[:]))
decision = flatted_compared == list(flatted_to_fit_there)
return decision
for m in range(n + 1, len(nesting_list)):
through = list(nesting_list[m])
def replacing_fun(x):
return list(replace(list(x), is_fitting, [to_fit_there], window_size=len(to_fit_there)))
nesting_list[m] = on_each_iterable(through, replacing_fun)
n = n + 1
return (nesting_list[-1])
nested_list = stack_matrjoshka(nestlist)
print (nested_list)
makes
['At', ['the', 'other', 'side', 'of', 'the', 'country'], [['Linden'], 'N.J.'], 'is', 'part', 'of', 'an', 'industrial', 'corridor', 'of', ['chemical', 'plants'], 'and', 'oil', 'refineries', ['that', ['some', 'federal', 'officials'], 'refer', 'to', 'as', ['the', 'most', 'dangerous', 'two', 'miles', 'in', 'America']]]
Just a quick beginners question here with NLTK.
I am trying to tokenize and generate bigrams, trigrams and quadgrams from a corpus.
I need to add <s> to the beginning of my lists and </s> to the end in place of a period if there is one.
The list is taken from the brown corpus in nltk. (and a specific section at that)
so.. I have
from nltk.corpus import brown
news = brown.sents(categories = 'editorial')
Am I making this too difficult?
Thanks a lot.
import nltk.corpus as corpus
def mark_sentence(row):
if row[-1] == '.':
row[-1] = '</s>'
else:
row.append('</s>')
return ['<s>'] + row
news = corpus.brown.sents(categories = 'editorial')
for row in news[:5]:
print(mark_sentence(row))
yields
['<s>', 'Assembly', 'session', 'brought', 'much', 'good', '</s>']
['<s>', 'The', 'General', 'Assembly', ',', 'which', 'adjourns', 'today', ',', 'has', 'performed', 'in', 'an', 'atmosphere', 'of', 'crisis', 'and', 'struggle', 'from', 'the', 'day', 'it', 'convened', '</s>']
['<s>', 'It', 'was', 'faced', 'immediately', 'with', 'a', 'showdown', 'on', 'the', 'schools', ',', 'an', 'issue', 'which', 'was', 'met', 'squarely', 'in', 'conjunction', 'with', 'the', 'governor', 'with', 'a', 'decision', 'not', 'to', 'risk', 'abandoning', 'public', 'education', '</s>']
['<s>', 'There', 'followed', 'the', 'historic', 'appropriations', 'and', 'budget', 'fight', ',', 'in', 'which', 'the', 'General', 'Assembly', 'decided', 'to', 'tackle', 'executive', 'powers', '</s>']
['<s>', 'The', 'final', 'decision', 'went', 'to', 'the', 'executive', 'but', 'a', 'way', 'has', 'been', 'opened', 'for', 'strengthening', 'budgeting', 'procedures', 'and', 'to', 'provide', 'legislators', 'information', 'they', 'need', '</s>']