Develop Reference

Regex for multiple lines separated with "return" and multiple unnecessary spaces

I was trying to parse together a script for a movie into a dataset containing two columns 'speaker_name' and 'line_spoken'. I don't have any issue with the Python part of the problem but parsing the script is the problem.
The schema of the script goes like this:
Now, this, if copied and pasted into a .txt file is something like this:
ARTHUR
Yeah. I mean, that's just--
SOCIAL WORKER
Does my reading it upset you?
He leans in.
ARTHUR
No. I just,-- some of it's
personal. You know?
SOCIAL WORKER
I understand. I just want to make
sure you're keeping up with it.
She slides his journal back to him. He holds it in his lap.
In the above case, the regex filtering should return the speaker name and the dialogue and not what is happening in actions like the last line: "slides his journal back". The dialogues often exceed more than two lines so please do not provide hard-coded solutions for 2 lines only. I think I am thinking about this problem in just one direction, some other method to filter can also work.
I have worked with scripts that are colon-separated and I don't have any problem parsing those. But in this case, I am getting no specific endpoints to end the search at. It would be a great help if the answer you give has 2 groups, one with name, the other with the dialogue. Like in the case of colon-separated, my regex was:
pattern = r'(^[a-zA-z]+):(.+)'
Also, if possible, please try and explain why you used that certain regex. It will be a learning experience for me.

Use https://www.onlineocr.net/ co convert pdf to text,
It shows immediately the outcome, where names and on the same line with dialogs,
which could allow for a simple processing
ARTHUR Yeah. I mean, that's just--
SOCIAL WORKER Does my reading it upset you?
He leans in.
ARTHUR No. I just,-- some of its personal. You know me ?
SOCIAL WORKER I understand. I just want to make sure you're keeping up with it.
She slides his journal back to him. He holds it in his lap.
Not sure will it work for longer dialogs.
Another solution is to extract data from the text file that you can download by clicking the "download output file" link . That file is formatted differently. In that file
10 leading spaces will indicate the dialog, and 5 leading spaces the name - a the least for you sample screenshot
The regex is
r" (.+)(\n( [^ ].+\n)+)"
https://regex101.com/r/FQk8uH/1
it puts in group 1 whatever starts with ten spaces and whatever starts with at the exactly five space into the second :
the subexpression " [^ ].+\n" denotes a line where the first five symbols are spaces, the sixth symbol is anything but space, and the rest of symbols until the end of line are arbitrary. Since dialogs tend to be multiline that expression is followed with another plus.
You will have to delete extra white space from dialogue with additional code and/or regex.
If the amount of spaces varies a bit (say 4-6 and 7 - 14 respectively) but has distinct section the regex needs to be adjusted by using variable repetition operator (curly braces {4, 6}) or optional spaces ?.
r" {7, 14}(.+)(\n( {4-6}[^ ].+\n)+)"
The last idea is to use preexisting list of names in play to match them e.g. (SOCIAL WORKER|JOHN|MARY|ARTUR). The https://www.onlineocr.net/ website still could be used to help spot and delete actions

In Python, you can use DOTALL:
re_pattern = re.compile(r'(\b[A-Z ]{3,}(?=\n))\n*(.*?)\n*(?=\b[A-Z ]{3,}\n|$)', re.DOTALL)
print(re.findall(re_pattern, mystr))
\b[A-Z ]{3,}(?=\n) matches speaker name.
\b matches a word boundary
[A-Z ]{3,} matches three or more upper case letters or spaces. (this means this regex won't recognize speaker names with less than three characters. I did this to avoid false positives in special cases but you might wanna change it. Also check what kind of characters might occur in speaker name (dots, minus, lower case...))
(?=\n) is a lookahead insuring the speaker name is directly followed by a new line (avoids false positive if a similar expression appears in a spoken line)
\n* matches newlines
(.*?) matches everything (including new lines thanks to DOTALL) until the next part of the expression (? makes it lazy instead of greedy)
\n* matches newlines
(?=\b[A-Z ]{3,}\n|$) is a lookahead i.e. a non capturing expression insuring the following part is either a speaker name or the end of your string
Output:
[('ARTHUR', "Yeah. I mean, that's just--"), ('SOCIAL WORKER', 'Does my reading it upset you?\n\nHe leans in.'), ('ARTHUR', "No. I just,-- some of it's\n\npersonal. You know?"), ('SOCIAL WORKER', "I understand. I just want to make\n\nsure you're keeping up with it.\n\nShe slides his journal back to him. He holds it in his lap.")]
You'll have to adjust formatting if you want to remove actions from the result though.

Regex fetch text align-left

I'm new to the Regex world and I've browse many site without finding what I'm looking for.
I have a file where I need to fetch the address. The address is align-left of the paper (there's text in the same line at the right).
Some information on multiple line (6)
that I don't need and can't paste because
it contains some personal information.
So imagine a lot of text here...
So imagine a lot of text here...
So imagine a lot of text here...
Sold To Bill To
Some Cie Some Other Cie
1111 chemin some-road 2222 chemin some-other-road
City-Here QC J0Q 1W0 Other City-Here QC J0Q 1W0
Canada Canada
I need to fetch the text in the 'Sold To' side.
I tried to use the \r but it returns nothing!
I don't know how to fetch the text from the start of the line until there's a bunch of spaces.
Ex: Some Cie (if more than 1 spaces, go to next line)
then I have: Sold\sTo(?=\s{2,100}) but it won't work while (?=\s{2, 100}) returns everything!!!
I saw this: ^((?:\S+\s+){2}\S+).*, which is very close to what I want, but I don't understand the whole thing. I would like to match from 2 to 5 words.
Then I have this: ^([A-Za-z0-9-]*)(?=\s{2,100}) which I thought would match At the beginning of the line until there's more than 2 spaces.
What am I getting wrong?
I need to do this in pure Regex (no flags allowed).
I'm completely lost. Some guidance would be much appreciated.

You're pretty close on your last attempt. Here's what I came up with:
^.+?(?=[^\S\n]{2,})
Explanation:
.+ - One or more characters
? - Non-greedy, to give the next part priority, i.e. avoid matching a bunch of spaces
[^\S\n] - Any whitespace character except newline (this is like \s minus \n)
{2,} - Two or more
Matches from the example:
Sold To
Some Cie
1111 chemin some-road
City-Here QC J0Q 1W0
Canada
Try it on Regex101
Simple example in Python:
import re
pattern = re.compile(r'^.+?(?=[^\S\n]{2,})')
with open(filename) as f:
for line in f:
m = pattern.match(line)
if m:
print(m.group())

python regex matching paragraph(s) starting with labels

I'm trying to match a paragraph or paragraphs which are lead by letters. I'm testing on and have tried dotALL, lookaheads, multiline, etc and I can't seem to get one to work. The string I'm trying to match looks like this:
A-B: Object, procedure:
- Somethings.
- More things, might run over several lines like this where the sentence just keeps on going and going and going and sometimes isn't even a sentence.
- Another line, sometimes not ending with period
- Variable amount of white space at the beginning of new lines
Comment (A-B): sometimes, there are comments which are separated by two \n\n characters like this.*
C. Second object, other procedure:
- More lines.
- Can have various leads (including no ' - ' leading.
- Variable number of lines.
The closest I've come to a match was using '(.+?\n\n|.+?$)' and dotALL (which I realize is sloppy), but even this didn't work because it misses comments or paragraphs separated by more lines but still under the header ([A-Z]?-?[A-Z]).
Ideally I'd like to capture the header or title (A-B:) or (C.) in match.group(1) and the rest of the paragraphs(s) before the next title in match.group(2), but I'd just be happy to capture everything. I tried lookaheads to catch everything between titles, but that misses the last instance which won't have a title at the end.
I'm a newb and I apologize if this has already been answered or if I'm not clear (but I have been looking for the past 2 days without success). Thanks!

so here is my proposed solution for you :)
import re
with open('./samplestring.txt') as f:
header =[]
nonheader = []
yourString = f.read()
for line in content.splitlines():
if(re.match('(^[A-Z]?-?[A-Z]:)|(^[A-Z]\.)',line.lstrip())):
header.append(line)
else:
nonheader.append(line)

I ended up giving up on capturing comments and everything after them. I used the following code to capture the letter for each header (group(1)), the text for the header (group(2)), and the text in the paragraph excluding comments (group(3)).
([A-Z]{1,2}|[A-Z]-[A-Z])(?::|.) +(\w.+)\n+((\s*(- *.+))+)
([A-Z]{1,2}|[A-Z]-[A-Z])(?::|.) +
captures the letter (group 1), the colon or period, and the space(s) after that
(\w.+)\n+
captures the text of the header, and the next line(s)
((\s*(- *.+))+)
captures multiple lines starting variably with a space, dash, space, and text
I appreciate all your help with this! :)

You can use
(^[^\n]+)(?:\n *-.+(?:\n.+)*|\n\n.+\n)+
(^[^\n]+) - Match the header line, then repeatedly alternate between
\n *-.+(?:\n.+)* - A non-comment line: starts with whitespace, followed by -, optionally running across multiple lines
\n\n.+\n - Or, match a comment line
(no dotall flag)
https://regex101.com/r/6kle0u/2
This depends on the comment lines always having \n\n before them.

Extract from text with python and regex

Let's say we have text within which some quotes are stored in the form:
user:quote
we can have multiple quotes within a text.
Agatha Drake: She records her videos from the future? What is she, a
f**ing time lord? Is she Michael J. Fox?
Harvey Spencer: This is just like that one movie where that one guy
changed one tiny, little thing in his childhood to stop the girl of
his dreams from being a crackhead in the future!
How can i extract the quotes (She records her videos from ..., This is just like that one movie....) from the text in python?
I tried
re.findall('\S\:\s?(.*)', text)
But it's not doing the job.
https://regex101.com/r/vH63Go/1
How can I do it in Python?

If your string is following the consistent format of user at the start of a line and double newlines ending a quote, you could use this:
(?m)^[^:\n]+:\s?((?:.+\n?)*)
It uses multiline mode and matches the start of a line, followed by characters that are neither : nor newline, folllowed by :. Then captures all following lines with content.
Here's a demo on regex101.

Removing "\n"s when printing sentences from text file in python?

I am trying to print a list of sentences from a text file (one of the Project Gutenberg eBooks). When I print the file as a single string string it looks fine:
file = open('11.txt','r+')
alice = file.read()
print(alice[:500])
Output is:
ALICE'S ADVENTURES IN WONDERLAND
Lewis Carroll
THE MILLENNIUM FULCRUM EDITION 3.0
CHAPTER I. Down the Rabbit-Hole
Alice was beginning to get very tired of sitting by her sister on the
bank, and of having nothing to do: once or twice she had peeped into the
book her sister was reading, but it had no pictures or conversations in
it, 'and what is the use of a book,' thought Alice 'without pictures or
conversations?'
So she was considering in her own mind (as well as she could, for the
hot d
Now, when I split it into sentences (The assignment was specifically to do this by "splitting at the periods," so it's a very simplified split), I get this:
>>> print(sentences[:5])
["ALICE'S ADVENTURES IN WONDERLAND\n\nLewis Carroll\n\nTHE MILLENNIUM FULCRUM EDITION 3", '0\n\n\n\n\nCHAPTER I', " Down the Rabbit-Hole\n\nAlice was beginning to get very tired of sitting by her sister on the\nbank, and of having nothing to do: once or twice she had peeped into the\nbook her sister was reading, but it had no pictures or conversations in\nit, 'and what is the use of a book,' thought Alice 'without pictures or\nconversations?'\n\nSo she was considering in her own mind (as well as she could, for the\nhot day made her feel very sleepy and stupid), whether the pleasure\nof making a daisy-chain would be worth the trouble of getting up and\npicking the daisies, when suddenly a White Rabbit with pink eyes ran\nclose by her", "\n\nThere was nothing so VERY remarkable in that; nor did Alice think it so\nVERY much out of the way to hear the Rabbit say to itself, 'Oh dear!\nOh dear! I shall be late!' (when she thought it over afterwards, it\noccurred to her that she ought to have wondered at this, but at the time\nit all seemed quite natural); but when the Rabbit actually TOOK A WATCH\nOUT OF ITS WAISTCOAT-POCKET, and looked at it, and then hurried on,\nAlice started to her feet, for it flashed across her mind that she had\nnever before seen a rabbit with either a waistcoat-pocket, or a watch\nto take out of it, and burning with curiosity, she ran across the field\nafter it, and fortunately was just in time to see it pop down a large\nrabbit-hole under the hedge", '\n\nIn another moment down went Alice after it, never once considering how\nin the world she was to get out again']
Where do the extra "\n" characters come from and how can I remove them?

If you want to replace all the newlines with one space, do this:
import re
new_sentences = [re.sub(r'\n+', ' ', s) for s in sentences]

You may not want to use regex, but I would do:
import re
new_sentences = []
for s in sentences:
new_sentences.append(re.sub(r'\n{2,}', '\n', s))
This should replace all instances of two or more '\n' with a single newline, so you still have newlines, but don't have "extra" newlines.
If you want to avoid creating a new list, and instead modify the existing one (credit to #gavriel and Andrew L.: I hadn't thought of using enumerate when I first posted my answer):
import re
for i, s in enumerate(sentences):
sentences[i] = re.sub(r'\n{2,}', '\n', s)
The extra newlines aren't really extra, by which I mean they are meant to be there and are visible in the text in your question: the more '\n' there are, the more space there is visible between the lines of text (i.e., one between the chapter heading and the first paragraph, and many between the edition and the chapter heading.

You'll understand where the \n characters come from with this little example:
alice = """ALICE'S ADVENTURES IN WONDERLAND
Lewis Carroll
THE MILLENNIUM FULCRUM EDITION 3.0
CHAPTER I. Down the Rabbit-Hole
Alice was beginning to get very tired of sitting by her sister on the
bank, and of having nothing to do: once or twice she had peeped into the
book her sister was reading, but it had no pictures or conversations in
it, 'and what is the use of a book,' thought Alice 'without pictures or
conversations?'
So she was considering in her own mind (as well as she could, for the
hot d"""
print len(alice.split("."))
print len(alice.split("\n"))
It all depends the way you're splitting your text, the above example will give this output:
3
19
Which means there are 3 substrings if you were to split the text using . or 19 substrings if you splitted using \n as separator. You can read more about str.split
In your case you've splitted your text using ., so the 3 substrings will contain multiple newlines characters \n, to get rid of them you can either split these substrings again or just get rid of them using str.replace

The text uses newlines to delimit sentences as well as fullstops. You have an issue where just replacing the new line characters with an empty string will result in having words without spaces between them. Before you split alice by '.', I would use something along the lines of #elethan's solution to replace all of the multiple new lines in alice with a '.' Then you could do alice.split('.') and all of the sentences separated with multiple new lines would be split appropriately along with the sentences separated with . initially.
Then your only issue is the decimal point in the version number.

file = open('11.txt','r+')
file.read().split('\n')