Develop Reference

How do I get the second div element from request

HTML code
<div class="f_fs13">
<div class="bg_blue">first</div>
<div>range 5</div>
<div>
My code
race = page_soup.find_all("div",{"class":"f_fs13"})
I want to get the text of second div element(range 5)
Any quick way to get second div element
race = page_soup.find_all("div",{"class":"f_fs13 margin_top15"},[1]) #but not working

I'm getting desired output using css selector as follows:
from bs4 import BeautifulSoup
html = """
<div class="f_fs13"><div class="bg_blue">first</div><div>range 5</div>
"""
soup = BeautifulSoup(html, 'html.parser')
#print(soup.prettify())
p= soup.select_one('div.f_fs13 div:nth-child(2)')
print(p.text)
Output:
range 5

How to get the text of nested tag with Beautifulsoup in Python?

After running this code
section = soup.find_all('section', class_='b-branches')
I get
<div class="b-branches__item"><i class="icon fa"><b>Firm</b> </i>RJT Roadlines</div>
Now i want to just extract RJIT Roadlines only not...Firm
So i tried
for i in section: firm = i.find('div', class_='b-branches__item') print(firm)
It will return both Firm and RJIT Roadlines
So, how to extract only div tag's text??

You can use tag.contents[1] to get your expected output.
Example:
from bs4 import BeautifulSoup
html = """
<div class="b-branches__item"><i class="icon fa"><b>Firm</b> </i>RJT Roadlines</div>
"""
soup=BeautifulSoup(html,'html.parser')
tag = soup.find('div', class_='b-branches__item')
print(tag.contents[1])
Output:
RJT Roadlines

how to extract the contents of a strong tag using beautiful soup?

<li class="bg-blue">
<img alt="Active Status" src="assets/images/icon-infected.png"/>
<strong>83004</strong>
<span>Active Cases</span>
I need to extract the values inside the strong tag , how should I do that?

With beautifulsoup you can use contents to get that, just like this.
html_doc = """
<li class="bg-blue">
<img alt="Active Status" src="assets/images/icon-infected.png"/>
<strong>83004</strong>
<span>Active Cases</span>
"""
from bs4 import BeautifulSoup
soup = BeautifulSoup(html_doc, 'html.parser')
soup.strong.contents
# 83004

Firstly you find the strong element and use text for getting the value. You can try it:
html_doc = """
<li class="bg-blue">
<img alt="Active Status" src="assets/images/icon-infected.png"/>
<strong>83004</strong>
<span>Active Cases</span>
"""
soup = BeautifulSoup(html_doc, 'lxml')
value = soup.find('strong').text
print(value)

replace tag name on xml or html with python

i'd like to change some html tag name.
let me know the method with python code.:
As-is
<div class="title-title-1">hello</div>
<div class="text-body">i like you</div>
<div class="p">hehe</div>
To be
<title-title-1>hello</title-title-1>
<text-body>i like you</text-body>
<p>hehe</p>
somebody help me!!

Using BeautifulSoup
Demo:
from bs4 import BeautifulSoup
s = """<div class="title-title-1">hello</div>
<div class="text-body">i like you</div>
<div class="p">hehe</div>"""
soup = BeautifulSoup(s, "html.parser")
title = soup.find("div", class_="title-title-1").text
body = soup.find("div", class_="text-body").text
p = soup.find("div", class_="p").text
toFrame = """<title-title-1>{0}</title-title-1>
<text-body>{1}</text-body>
<p>{2}</p>""".format(title, body, p)
print(toFrame)
Output:
<title-title-1>hello</title-title-1>
<text-body>i like you</text-body>
<p>hehe</p>

i want to scrape data using python script

I have written python script to scrape data from http://www.cricbuzz.com/cricket-stats/icc-rankings/batsmen-rankings
It is a list of 100 players and I successfully scraped this data. The problem is, when i run script instead of scraping data just one time it scraped the same data 3 times.
<div class="cb-col cb-col-100 cb-font-14 cb-lst-itm text-center">
<div class="cb-col cb-col-16 cb-rank-tbl cb-font-16">1</div>
<div class="cb-col cb-col-50 cb-lst-itm-sm text-left">
<div class="cb-col cb-col-33">
<div class="cb-col cb-col-50">
<span class=" cb-ico" style="position:absolute;"></span> –
</div>
<div class="cb-col cb-col-50">
<img src="http://i.cricketcb.com/i/stats/fw/50x50/img/faceImages/2250.jpg" class="img-responsive cb-rank-plyr-img">
</div>
</div>
<div class="cb-col cb-col-67 cb-rank-plyr">
<a class="text-hvr-underline text-bold cb-font-16" href="/profiles/2250/steven-smith" title="Steven Smith's Profile">Steven Smith</a>
<div class="cb-font-12 text-gray">AUSTRALIA</div>
</div>
</div>
<div class="cb-col cb-col-17 cb-rank-tbl">906</div>
<div class="cb-col cb-col-17 cb-rank-tbl">1</div>
</div>
And here is python script which i write scrap each player data.
import sys,requests,csv,io
from bs4 import BeautifulSoup
from urllib.parse import urljoin
url = "http://www.cricbuzz.com/cricket-stats/icc-rankings/batsmen-rankings"
r = requests.get(url)
r.content
soup = BeautifulSoup(r.content, "html.parser")
maindiv = soup.find_all("div", {"class": "text-center"})
for div in maindiv:
print(div.text)
but instead of scraping the data once, it scrapes the same data 3 times.
Where can I make changes to get data just one time?

Select the table and look for the divs in that:
maindiv = soup.select("#batsmen-tests div.text-center")
for div in maindiv:
print(div.text)
Your original output and that above gets all the text from the divs as one line which is not really useful, if you just want the player names:
anchors = soup.select("#batsmen-tests div.cb-rank-plyr a")
for a in anchors:
print(a.text)
A quick and easy way to get the data in a nice csv format is to just get text from each child:
maindiv = soup.select("#batsmen-tests div.text-center")
for d in maindiv[1:]:
row_data = u",".join(s.strip() for s in filter(None, (t.find(text=True, recursive=False) for t in d.find_all())))
if row_data:
print(row_data)
Now you get output like:
# rank, up/down, name, country, rating, best rank
1,–,Steven Smith,AUSTRALIA,906,1
2,–,Joe Root,ENGLAND,878,1
3,–,Kane Williamson,NEW ZEALAND,876,1
4,–,Hashim Amla,SOUTH AFRICA,847,1
5,–,Younis Khan,PAKISTAN,845,1
6,–,Adam Voges,AUSTRALIA,802,5
7,–,AB de Villiers,SOUTH AFRICA,802,1
8,–,Ajinkya Rahane,INDIA,785,8
9,2,David Warner,AUSTRALIA,772,3
10,–,Alastair Cook,ENGLAND,770,2
11,1,Misbah-ul-Haq,PAKISTAN,764,6
As opposed to:
PositionPlayerRatingBest Rank
Player
1    –Steven SmithAUSTRALIA9061
2    –Joe RootENGLAND8781
3    –Kane WilliamsonNEW ZEALAND8761
4    –Hashim AmlaSOUTH AFRICA8471
5    –Younis KhanPAKISTAN8451
6    –Adam VogesAUSTRALIA8025

The reason you get output three times is because the website has three categories you have to select it and then accordingly you can use it.
Simplest way of doing it with your code would be to add just one line
import sys,requests,csv,io
from bs4 import BeautifulSoup
url = "http://www.cricbuzz.com/cricket-stats/icc-rankings/batsmen- rankings"
r = requests.get(url)
r.content
soup = BeautifulSoup(r.content, "html.parser")
specific_div = soup.find_all("div", {"id": "batsmen-tests"})
maindiv = specific_div[0].find_all("div", {"class": "text-center"})
for div in maindiv:
print(div.text)
This will give similar reuslts with just test batsmen, for other output just change the "id" in specific_div line.