Develop Reference

turning one column into multiple pro-rated column

I have a data regarding an insurance customer's premium during a certain year.
User ID
Period From
Period to
Period from-period to
Total premium
A8856
Jan 2022
Apr 2022
4
$600
A8857
Jan 2022
Feb 2022
2
$400
And I'm trying to turn it into a pro-rated one
Assuming that the input I'm expecting is like this:
User ID
Period From
Total premium
A8856
Jan 2022
$150
A8856
Feb 2022
$150
A8856
Mar 2022
$150
A8856
Apr 2022
$150
A8857
Jan 2022
$200
A8857
Feb 2022
$200
What kind of code do you think I should use? I use python and help is really appreciated.

web-scraping an unordered table

I'm trying to scrape with BeautifulSoup then print it out in Pandas, but the table I need to work with is having spans randomly every month.
"https://to.sze.hu/kezdolap" the top table in the middle div
The path is soup.select("#content > div:nth-child(2) > div > div > div > table")

You can do:
df_list = pd.read_html('https://to.sze.hu/kezdolap', header=0)
Which will add your tables to a list, then simply:
df_list[0]
Gives you:
Ügyfélfogadás NAPPALI tagozatos hallgatók számára (full time students) Ügyfélfogadás NAPPALI tagozatos hallgatók számára (full time students).1 Ügyfélfogadás NAPPALI tagozatos hallgatók számára (full time students).2
0 2021. augusztus 25-augusztus 27./ 25. August -... szerda/Wednesday 9.30-11.00
1 2021. augusztus 25-augusztus 27./ 25. August -... csütörtök/Thursday 13.00-14.30
2 2021. augusztus 25-augusztus 27./ 25. August -... péntek/Friday 13.00-14.30
3 2021. augusztus 30-szeptember 3./ 30. August -... kedd/Tuesday 9.30-11.00
4 2021. augusztus 30-szeptember 3./ 30. August -... csütörtök/Thursday 13.00-14.30
5 2021. augusztus 30-szeptember 3./ 30. August -... péntek/Friday 13.00-14.30
6 2021. szeptember 6-17. / 06. September-17. Sep... kedd/Tuesday 9.30 – 11.00
7 2021. szeptember 6-17. / 06. September-17. Sep... csütörtök/Thursday 13.00 – 14.30
8 2021. szeptember 20-30. / 20-30 September 2021 kedd/Tuesday 10.00 – 11.00
9 2021. szeptember 20-30. / 20-30 September 2021 szerda/Wednesday 10.00 – 11.00
10 2021. szeptember 20-30. / 20-30 September 2021 szerda/Wednesday 13.00 – 14.00
11 2021. szeptember 20-30. / 20-30 September 2021 csütörtök/Thursday 10.00 - 11.00

How to calcuate the overlap date in pyspark

I have data with users who has worked with multiple companies.Some users who worked in more than one companies at the same time. How to aggregate the overall experience without considering overlap experience.
I have gone through some of the links could get right solutions.Any help will appreciated.
EMP CSV DATA
fullName,Experience_datesEmployeed,Experience_expcompany,Experience_expduraation, Experience_position
David,Feb 1999 - Sep 2001, Foothill,2 yrs 8 mos, Marketing Assoicate
David,1994 - 1997, abc,3 yrs,Senior Auditor
David,Jun 2020 - Present, Fellows INC,3 mos,Director Board
David,2017 - Jun 2019, Fellows INC ,2 yrs,Fellow - Class 22
David,Sep 2001 - Present, The John D.,19 yrs, Manager
Expected output:
FullName,Total_Experience
David,24.8 yrs

Using Python and BeautifulSoup to scrape list from an URL

I am new to BeautifulSoup so please excuse any beginner mistakes here. I am attempting to scrape an url and want to store list of movies under one date.
Below is the code I have so far:
import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.imdb.com/calendar?region=IN&ref_=rlm")
soup = BeautifulSoup(page.content, 'html.parser')
date = soup.find_all("h4")
ul = soup.find_all("ul")
for h4,h1 in zip(date,ul):
dd_=h4.get_text()
mv=ul.find_all('a')
for movie in mv:
text=movie.get_text()
print (dd_,text)
movielist.append((dd_,text))
I am getting "AttributeError: ResultSet object has no attribute 'find_all'. You're probably treating a list of items like a single item. Did you call find_all() when you meant to call find()?"
Expected result in list or dataframe
29th May 2020 Romantic
29th May 2020 Sohreyan Da Pind Aa Gaya
5th June 2020 Lakshmi Bomb
and so on
Thanks in advance for help.

This script will get all movies and corresponding dates to a dataframe:
import requests
import pandas as pd
from bs4 import BeautifulSoup
url = 'https://www.imdb.com/calendar?region=IN&ref_=rlm'
soup = BeautifulSoup(requests.get(url).content, 'html.parser')
out, last = [], ''
for tag in soup.select('#main h4, #main li'):
if tag.name == 'h4':
last = tag.get_text(strip=True)
else:
out.append({'Date':last, 'Movie':tag.get_text(strip=True).rsplit('(', maxsplit=1)[0]})
df = pd.DataFrame(out)
print(df)
Prints:
Date Movie
0 29 May 2020 Romantic
1 29 May 2020 Sohreyan Da Pind Aa Gaya
2 05 June 2020 Laxmmi Bomb
3 05 June 2020 Roohi Afzana
4 05 June 2020 Nikamma
.. ... ...
95 26 March 2021 Untitled Luv Ranjan Film
96 02 April 2021 F9
97 02 April 2021 Bell Bottom
98 02 April 2021 NTR Trivikiram Untitled Movie
99 09 April 2021 Manje Bistre 3
[100 rows x 2 columns]

I think you should replace "ul" with "h1" on the 10th line. And add definition of variable "movielist" ahead.
import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.imdb.com/calendar?region=IN&ref_=rlm")
soup = BeautifulSoup(page.content, 'html.parser')
date = soup.find_all("h4")
ul = soup.find_all("ul")
# add code here
movielist = []
for h4,h1 in zip(date,ul):
dd_=h4.get_text()
# replace ul with h1 here
mv=h1.find_all('a')
for movie in mv:
text=movie.get_text()
print (dd_,text)
movielist.append((dd_,text))
print(movielist)

I didn't specify a list to receive, and I changed it from 'h1' to 'text capture' instead of 'h4'.
import requests
from bs4 import BeautifulSoup
page = requests.get("https://www.imdb.com/calendar?region=IN&ref_=rlm")
soup = BeautifulSoup(page.content, 'html.parser')
movielist = []
date = soup.find_all("h4")
ul = soup.find_all("ui")
for h4,h1 in zip(date,ul):
dd_=h4.get_text()
mv=h1.find_all('a')
for movie in mv:
text=movie.get_text()
print (dd_,text)
movielist.append((dd_,text))
The reason the date doesn't match in the output result is that the 'date' retrieved looks like the following, so you need to fix the logic.
There are multiple titles on the same release date, so the release date and number of titles don't match up. I can't help you that much because I don't have the time. Have a good night.
29 May 2020
05 June 2020
07 June 2020
07 June 2020 Romantic
12 June 2020
12 June 2020 Sohreyan Da Pind Aa Gaya
18 June 2020
18 June 2020 Laxmmi Bomb
19 June 2020
19 June 2020 Roohi Afzana
25 June 2020
25 June 2020 Nikamma
26 June 2020
26 June 2020 Naandhi
02 July 2020
02 July 2020 Mandela
03 July 2020
03 July 2020 Medium Spicy
10 July 2020
10 July 2020 Out of the Blue

How do I find a block containing 2 tags in a loop?

I am looking to scrape the contents of the following html and want to capture the h2 then each until the next h2 using beautiful soup. Is this possible?
<hr /><h2>California</h2>
<p><strong>Term 1:</strong> (Eastern division): Tuesday 29 January —
Friday
12 April</p>
<p><strong>Term 1:</strong> (Western division): Tuesday 5 February —
Friday
12 April</p>
<p><strong>Term 2</strong><strong>:</strong> Monday 29 April — Friday 5
July</p>
<p><strong>Term 3:</strong> Monday 22 July — Friday 27 September</p>
<p><strong>Term 4:</strong> Monday 14 October — Friday 20 December</p>
<hr /><h2>New York</h2>
<p><strong>Term 1</strong>: Tuesday 29 January — Friday 12 April</p>
<p><strong>Term 2:</strong> Monday 29 April — Friday 5 July</p>
<p><strong>Term 3</strong>: Monday 22 July — Friday 27 September</p>
<p><strong>Term 4</strong>: Monday 14 October — Friday 13 December</p>
</pre>
soup = BeautifulSoup(page.text, 'html.parser')
for each_div in soup.findAll(['h2', 'p']):
myval = str(each_div.prettify("ascii"))
I want to get the following results for each state

Here's something I think you should be able to work with. The list capture
keeps track of the elements you want for each header. The code is using the
find_next_siblings method to get all of the siblings in the tree and iterate
over them. When it reaches another h2 tag, it breaks.
soup = BeautifulSoup(content, 'html.parser')
for head in soup.find_all('h2'):
capture = [head]
for sibling in head.find_next_siblings():
if sibling.name == 'h2':
break
capture += [sibling]
I would just change how you store off the captured tags.
Edit: Forgot to mention that content is the html string provided in
your question.