I'm trying to make an automation script for tests. My issue is that I can't select a value from drop down menu. I've try a lot of things , but can't make it. My goal is the script to choose every time different value from the menu. When I click the hidden menu , it create a 'ul class' with about 100 'li classes' . There are no id, name or class. I don't know how to reach the element there and click it.
Things that I've tried...
elem = driver.find_element_by_xpath('/html/body/div[3]/div[3]')
all_li = elem.find_elements_by_tag_name("li")
gg = random.choice(all_li)
gg = driver.find_element_by_css_selector("ul > li:nth-child(15)").click()
html code, this is what generate html when hit menu
This is my code:
driver.find_element_by_xpath("/html/body/div[1]/div/main/div/div[2]/form/div[2]/div[1]/div/div/div").click()
Simple html:
<div class="MuiPaper-root MuiMenu-paper MuiPaper-elevation8 MuiPopover-paper MuiPaper-rounded" role="document" tabindex="-1" style="opacity: 1; transform: none; min-width: 491px; transition: opacity 381ms cubic-bezier(0.4, 0, 0.2, 1) 0ms, transform 254ms cubic-bezier(0.4, 0, 0.2, 1) 0ms; top: 80px; left: 16px; transform-origin: -1px 478.513px;">
<ul class="MuiList-root MuiMenu-list MuiList-padding" role="listbox" tabindex="-1" style="padding-right: 17px; width: calc(100% + 17px);">
<li class="MuiButtonBase-root MuiListItem-root MuiMenuItem-root MuiMenuItem-gutters MuiListItem-gutters MuiListItem-button" tabindex="-1" role="option" aria-disabled="false" variant="outlined" data-value="testOne">Test One<span class="MuiTouchRipple-root"></span></li>
<li class="MuiButtonBase-root MuiListItem-root MuiMenuItem-root MuiMenuItem-gutters MuiListItem-gutters MuiListItem-button" tabindex="-1" role="option" aria-disabled="false" variant="outlined" data-value="testTwo">Test Two<span class="MuiTouchRipple-root"></span></li>
Related
I'd like to extract and use the red alphabet of the code below through 'Selenium', so please give me some advice on how to do it
The alphabet changes randomly on every try
<td>
<input type="text" name="WKey" id="As_wkey" value="" maxlength="10" class="inputType" style="width: 300px;" title="password" />
<span id="myspam" style="padding: 2px;">
<span style="font-size: 12pt; font-weight: bold; color: red;">H</span>123
<span style="font-size: 12pt; font-weight: bold; color: red;">R</span>
<span style="font-size: 12pt; font-weight: bold; color: red;">8</span>6789
</span>
(type red word.)
</td>
here is my code
red_characters_element = driver.find_element(By.ID, 'myspam')
red_characters_elements = red_characters_element.find_elements(by = By.CSS_SELECTOR, value="span[style='font-size: 12pt; font-weight: bold; color: red;']")
print(red_characters_elements)
result []
Given all the Red colored alphabets are inside of the <span> tag. You can retrieve it using tag.
red_characters_element = driver.find_element(By.ID, 'myspam')
red_characters_elements = red_characters_element.find_elements(By.TAG_NAME, 'span')
for red_character in red_characters_elements:
print(red_character.text)
Results :
H
R
8
If you need only red letters, you can try using the java script inside the selenium:
driver.execute_script('return document.querySelectorAll("[style*=red]")')
You get an array of objects where the style has the attribute "red", with the for loop you can get the values or anything else
This is a link to HTML I want to scrape
https://pk.khaadi.com/unstitched/r20206-red-r20206-red-pk.html
<div class="swatch-attribute-options clearfix">
<div class="swatch-option color selected" option-type="1" option-
id="61" option-label="RED" option-tooltip-thumb="" option-tooltip-
value="#ee0000" "="" style="background: #ee0000 no-repeat center;
background-size: initial;">
</div>
<div class="swatch-option color selected" option-type="1" option-
id="73" option-label="YELLOW" option-tooltip-thumb="" option-tooltip-
value="#feed00" "="" style="background: #feed00 no-repeat center;
background-size: initial;">
</div>
</div>
Color = S_Driver.find_elements_by_xpath( '//*[#id="product-options-wrapper"]/div/div/div[1]/div' )
The Xpath is of the outer div in which both color div are present
for c in Color:
n_Color.append(c.get_attribute( 'option-label' ))
print( n_Color + '\n' )
This how i tried to extract the color through 'option-label' attribute
Change the xpath with:
//div[#class='swatch-option color']
Created based on the provided screenshot, hope that there are no other matches on page based on this one. If so, change it with:
//div[#class='swatch-option color' and #option-type='1']
html=
"""<div class="slick-list"><div class="slick-track" style="width: 1380px; opacity: 1; transform: translate3d(0px, 0px, 0px);"><div data-index="0" class="slick-slide slick-active slick-current" tabindex="-1" aria-hidden="false" style="outline: none; width: 230px;"><div><div data-courseid="567828" class="course-discovery-unit--card-margin--2TVw4 merchandising-course-card--card--2UfMa"><a href="/course/complete-python-bootcamp/" data-purpose="merchandising-course-card-body-567828" target="_self" class="merchandising-course-card--mask--2-b-d"><div class="merchandising-course-card--card-header--89z8L"><img class="merchandising-course-card--course-image--3G7Kh" alt="" width="240" height="135" src="https://img-a.udemycdn.com/course/240x135/567828_67d0.jpg" srcset="https://img-a.udemycdn.com/course/240x135/567828_67d0.jpg 1x, https://img-a.udemycdn.com/course/480x270/567828_67d0.jpg 2x"></div><div class="merchandising-course-card--card-body--3OpAH"><div><div class="merchandising-course-card--course-title--2Ob4m" data-purpose="course-card-title">Complete Python Bootcamp: Go from zero to hero in Python 3</div>"""
I want to extract link and title output:
title=Complete Python Bootcamp: Go from zero to hero in Python 3
link=/course/complete-python-bootcamp/
Here is my code:
data=soup.findAll("div",{"class":"slick-list"})
print(data)
for link in data:
for a in link.findAll("a"):
print(a.title,a.href)
from bs4 import BeautifulSoup
html="""<div class="slick-list"><div class="slick-track" style="width: 1380px; opacity: 1; transform: translate3d(0px, 0px, 0px);"><div data-index="0" class="slick-slide slick-active slick-current" tabindex="-1" aria-hidden="false" style="outline: none; width: 230px;"><div><div data-courseid="567828" class="course-discovery-unit--card-margin--2TVw4 merchandising-course-card--card--2UfMa"><a href="/course/complete-python-bootcamp/" data-purpose="merchandising-course-card-body-567828" target="_self" class="merchandising-course-card--mask--2-b-d"><div class="merchandising-course-card--card-header--89z8L"><img class="merchandising-course-card--course-image--3G7Kh" alt="" width="240" height="135" src="https://img-a.udemycdn.com/course/240x135/567828_67d0.jpg" srcset="https://img-a.udemycdn.com/course/240x135/567828_67d0.jpg 1x, https://img-a.udemycdn.com/course/480x270/567828_67d0.jpg 2x"></div><div class="merchandising-course-card--card-body--3OpAH"><div><div class="merchandising-course-card--course-title--2Ob4m" data-purpose="course-card-title">Complete Python Bootcamp: Go from zero to hero in Python 3</div>"""
soup = BeautifulSoup(html, 'html.parser')
print('title='+soup.find("div",{"data-purpose":"course-card-title"}).text)
print('link='+soup.find("a").get('href'))
I hope this answers your question.
I working solution based on your code (and using findAll):
from bs4 import BeautifulSoup
html= """<div class="slick-list"><div class="slick-track" style="width: 1380px; opacity: 1; transform: translate3d(0px, 0px, 0px);"><div data-index="0" class="slick-slide slick-active slick-current" tabindex="-1" aria-hidden="false" style="outline: none; width: 230px;"><div><div data-courseid="567828" class="course-discovery-unit--card-margin--2TVw4 merchandising-course-card--card--2UfMa"><a href="/course/complete-python-bootcamp/" data-purpose="merchandising-course-card-body-567828" target="_self" class="merchandising-course-card--mask--2-b-d"><div class="merchandising-course-card--card-header--89z8L"><img class="merchandising-course-card--course-image--3G7Kh" alt="" width="240" height="135" src="https://img-a.udemycdn.com/course/240x135/567828_67d0.jpg" srcset="https://img-a.udemycdn.com/course/240x135/567828_67d0.jpg 1x, https://img-a.udemycdn.com/course/480x270/567828_67d0.jpg 2x"></div><div class="merchandising-course-card--card-body--3OpAH"><div><div class="merchandising-course-card--course-title--2Ob4m" data-purpose="course-card-title">Complete Python Bootcamp: Go from zero to hero in Python 3</div>"""
soup = BeautifulSoup(html, 'html.parser')
data=soup.findAll("div",{"class":"slick-list"})
#print(data)
for div in data:
for a in div.findAll("a"):
print(div.text,a.get('href'))
I need to grab the text (shown as a date) in the following piece of html code. The code a1v2v3 changes depending on the page, so I cannot use that as a reference or use a css selector.
Relevant HTML:
<div class="mvp-collapse-content-box">
<div data-v-a1v2v3="" class="mvp-row"><div data-v-a1v2v3="" class="mvp-tag mvp-tag-default mvp-tag-checked" style="margin-left: -16px; visibility: hidden;">
<!----> <span class="mvp-tag-text">LIVE</span>
<!----></div><span data-v-a1v2v3="" style="display: inline-block;">
2019.06.12 17:09
<br data-v-a1v2v3="">
Full HTML:
<div class="mvp-collapse-content-box">
<div data-v-a1v2v3="">
<div data-v-a1v2v3="" class="mvp-collapse mvp-collapse-simple">
<div data-v-a1v2v3="" class="mvp-collapse-item mvp-collapse-item-active" style="padding-left: 6px;">
<div class="mvp-collapse-header">
<!----> <div data-v-a1v2v3="" class="mvp-row-flex mvp-row-flex-middle">
<div data-v-a1v2v3="" class="mvp-col mvp-col-span-18 mvp-col-span-xs-16 mvp-col-span-sm-18 mvp-col-span-md-18 mvp-col-span-lg-18"><div data-v-a1v2v3="" class="mvp-tag mvp-tag-blue mvp-tag-checked" style="margin-left: -16px;">
<!----> <span class="mvp-tag-text mvp-tag-color-white">LIVE</span>
<!----></div><div data-v-a1v2v3="" class="versionAndMemo">
<span data-v-a1v2v3="" style="display: inline-block; line-height: 26px; vertical-align: middle; margin: 0px 1px; font-weight: bold; font-size: 14px;">1.2.3.44</span>
<!----></div></div>
<div data-v-a1v2v3="" class="mvp-col mvp-col-span-6 mvp-col-span-xs-8 mvp-col-span-sm-6 mvp-col-span-md-6 mvp-col-span-lg-6"><div data-v-a1v2v3="" style="display: inline-block; float: right; margin-right: 6px;"><i data-v-a1v2v3="" class="lal la-download" style="font-size: 1.8em; margin-top: 8px; display: table-cell; vertical-align: middle;"></i></div>
<div data-v-a1v2v3="" style="float: right; margin-right: 22px;"><i data-v-a1v2v3="" class="lal la-link" style="font-size: 1.8em; margin-top: 8px; display: table-cell; vertical-align: middle;"></i></div></div></div></div> <div class="mvp-collapse-content" style="" data-old-padding-top="" data-old-padding-bottom="" data-old-overflow="">
<div class="mvp-collapse-content-box">
<div data-v-a1v2v3="" class="mvp-row"><div data-v-a1v2v3="" class="mvp-tag mvp-tag-default mvp-tag-checked" style="margin-left: -16px; visibility: hidden;">
<!----> <span class="mvp-tag-text">LIVE</span>
<!----></div><span data-v-a1v2v3="" style="display: inline-block;">
2019.06.12 17:09
<br data-v-a1v2v3="">
Here is what I have so far:
page = requests.get(app, headers=headers, cookies=cookies).text
soup = BeautifulSoup(page, 'html.parser')
for spantime in soup.findAll("div", {"class": "mvp-collapse-content-box"}):
print(spantime)
But nothing is being printed. I have also tried adding the following:
page = requests.get(app, headers=headers, cookies=cookies).text
soup = BeautifulSoup(page, 'html.parser')
for spantime in soup.findAll("div", {"class": "mvp-collapse-content-box"}):
print(spantime.text)
for span in spantime.find_all('span', recursive=True):
print(span.text)
But the neither of them prints anything. I have a feeling that it might have something to do with the mvp-collapse-content-box class that I've used - some of the div tags with that class do not necessarily have span tags, as shown in the Full HTML.
Use find_next() to find the span tag and use text property.
from bs4 import BeautifulSoup
html='''<div class="mvp-collapse-content-box">
<div data-v-a1v2v3="" class="mvp-row"><div data-v-a1v2v3="" class="mvp-tag mvp-tag-default mvp-tag-checked" style="margin-left: -16px; visibility: hidden;">
<!----> <span class="mvp-tag-text">LIVE</span>
<!----></div><span data-v-a1v2v3="" style="display: inline-block;">
2019.06.12 17:09
<br data-v-a1v2v3="">'''
soup=BeautifulSoup(html,'html.parser')
div=soup.find('div',class_="mvp-collapse-content-box")
print(div.find_next('span').find_next('span').text.strip())
Output:
2019.06.12 17:09
You can always select a child tag like this:
div = soup.find("div", { "class" : "mvp-collapse-content-box" })
spans = div.findChildren("span" , recursive=False)
for span in spans:
print span
`
How to write code to print a tooltip message in python?
I am using the following code to move my pointer to the element on the web page:
element = driver.find_element_by_css_selector("locator")
webdriver.ActionChains(driver).move_to_element(element).perform()
time.sleep(3)
Also, our developer is using the same class for all the tool tip messages on that page so I don't know how to print that particular tooltip message.
HTML code:
<div class="d3-tip n" style="position: absolute; opacity: 0; pointer-events: none; top: 1108px; left: 116px;"><div><strong>Table Name:</strong> <span style="color:#1F77B4">lineitem</span></div><div><strong>Number of Queries:</strong> <span style="color:#1F77B4">13</span></div></div>
I want to print the "lineitem" and "13" from the above div tag
But as I mentioned this is another div tag with the same class that has a different tool tip message:
<div class="d3-tip n" style="position: absolute; opacity: 0; pointer-events: none; top: 469px; left: -180.23684692382813px;">select s_suppkey, s_name, s_address, s_phone, total_revenue from supplier s join revenue r on s.s_suppkey = r.supplier_no join (select max(total_revenue) as m from revenue) mr on r.total_revenue = mr.m order by s_suppkey </div>
According to the info you've provided, you can get the desired div by the text inside.
For example, you can check for Table Name: text using the xpath:
element = driver.find_element_by_xpath("//div[./div/strong, 'Table Name:']")