<div>
<div class="alk_dvImage"><img class="alk_prImg" src="https://a random photo" alt="a random product">
</div>
<div class="product-score"></div>
<a href="/products/" class="alk_prName alk_pr" title="Products Title">Strong Graphic Card
</a>
</div>
Lets assume we have a html as given above. I want to extract the title of the 'a' element which is nested in a div. And also i want the class of this same element how ever when i try this code
browser.find_element_by_css_selector('a.alk_prName alk_pr')
this does not respond anything. Btw i couldnt do anything to get tite of a element.
What happens?
Your not chaining the classes by dot in your selector, try the following:
browser.find_element_by_css_selector('a.alk_prName.alk_pr').get_attribute("title")
Example:
from selenium import webdriver
browser = webdriver.Chrome('C:\Program Files\ChromeDriver\chromedriver.exe')
html_content = """
Strong Graphic Card
"""
browser.get("data:text/html;charset=utf-8,{html_content}".format(html_content=html_content))
browser.find_element_by_css_selector('a.alk_prName.alk_pr').get_attribute("title")
To print the value of the title attribute i.e. Products Title you can use either of the following Locator Strategies:
Using css_selector:
print(driver.find_element(By.CSS_SELECTOR, "a.alk_prName.alk_pr[href='/products/']").get_attribute("title"))
Using xpath:
print(driver.find_element(By.XPATH, "//a[#class='alk_prName alk_pr' and #href='/products/'][contains(., 'Strong Graphic Card')]").get_attribute("title"))
Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a.alk_prName.alk_pr[href='/products/']"))).get_attribute("value"))
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[#class='alk_prName alk_pr' and #href='/products/'][contains(., 'Strong Graphic Card')]"))).get_attribute("value"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
I'm trying to scrape the audience score from rotten tomatoes. I was able to get reviews but not sure how use selenium to get the "audiencescore"
Source:
<score-board
audiencestate="upright"
audiencescore="96"
class="scoreboard"
rating="R"
skeleton="panel"
tomatometerstate="certified-fresh"
tomatometerscore="92"
data-qa="score-panel"
>
<h1 slot="title" class="scoreboard__title" data-qa="score-panel-movie-title">Pulp Fiction</h1>
<p slot="info" class="scoreboard__info">1994, Crime/Drama, 2h 33m</p>
<a slot="critics-count" href="/m/pulp_fiction/reviews?intcmp=rt-scorecard_tomatometer-reviews" class="scoreboard__link scoreboard__link--tomatometer" data-qa="tomatometer-review-count">110 Reviews</a>
<a slot="audience-count" href="/m/pulp_fiction/reviews?type=user&intcmp=rt-scorecard_audience-score-reviews" class="scoreboard__link scoreboard__link--audience" data-qa="audience-rating-count">250,000+ Ratings</a>
<div slot="sponsorship" id="tomatometer_sponsorship_ad"></div>
</score-board>
Code:
from selenium import webdriver
driver = webdriver.Firefox()
url = 'https://www.rottentomatoes.com/m/pulp_fiction'
driver.get(url)
print(driver.find_element_by_css_selector('a[slot=audience-count]').text)
The attribute value of audiencescore which is not any text nodes value that's why we can't invoke .text method to grab that value. So you have to call get_attribute() after selecting the right locator. The following expression is working.
print(driver.find_element(By.CSS_SELECTOR,'#topSection score-board').get_attribute('audiencescore'))
#import
from selenium.webdriver.common.by import By
You were close enough. To extract the value of the audiencescore attribute i.e. the text 96 ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://www.rottentomatoes.com/m/pulp_fiction")
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "score-board.scoreboard"))).get_attribute("audiencescore"))
Using XPATH:
driver.get("https://www.rottentomatoes.com/m/pulp_fiction")
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//score-board[#class='scoreboard']"))).get_attribute("audiencescore"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Console Output:
96
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
Try this:
1- Get element score-board
2- Get audiencescore attribute from element
audiencescore = driver.find_element_by_css_selector('score-board').get_attribute('audiencescore')
How to locate value inside a div using xpath \ css_selector
I have this html:
<div class="catalog-products view-tile" data-catalog-products="" data-slider-available="" data-primary-as-icon=""><div data-id="product" class="catalog-product ui-button-widget" data-product="5a7b4c6d-0bc3-11ec-a2b0-00155dfc8232" data-code="4862447" data-preview-slider-inited="1"><div class="catalog-product__image"><a class="catalog-product__image-link" href="/product/5a7b4c6d0bc3c823/videokarta-msi-geforce-210-n210-1gd3lp/" data-toggle-slider=""><picture><source type="image/webp" media="(min-width: 768px)" etc
So I need to get data-code value 4862447
Tried to access via xpath:
/html/body/div[1]/div/div[2]/div[2]/div[3]/div/div[1]/div[1]
Got highlighted in Chrome console that part:
<div data-id="product" class="catalog-product ui-button-widget" data-product="5a7b4c6d-0bc3-11ec-a2b0-00155dfc8232" data-code="4862447" data-preview-slider-inited="1">
Don't know how to get data-code value.
Tried css_selector:
div[data-id='product']
Got same line:
<div data-id="product" class="catalog-product ui-button-widget" data-product="5a7b4c6d-0bc3-11ec-a2b0-00155dfc8232" data-code="4862447" data-preview-slider-inited="1">
And no idea again.
To print the value of the data-code attribute you have to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.catalog-products.view-tile > div.catalog-product.ui-button-widget[data-id='product']"))).get_attribute("data-code"))
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='catalog-products view-tile']/div[#data-id='product' and #class='catalog-product ui-button-widget'][.//a[#class='catalog-product__image-link' and #href and #data-toggle-slider]]"))).get_attribute("data-code"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
There is such HTML code on the page, I need to get its text for check, or rather 740 using Selenium.
<button data-v-3630a784="" class="currency-btn mr10"><div data-v-3630a784="" class="progress"><div data-v-3630a784="" class="progress-bg anim" style="transform: scaleX(0.458333);"></div></div> <div data-v-3630a784="" class="num flex align-center icon-mana">740</div> <!----></button>
To print the text 740 you can use either of the following locator strategies:
Using css_selector and get_attribute("innerHTML"):
print(driver.find_element(By.CSS_SELECTOR, "button[class*='currency-btn'] div.num.flex.align-center.icon-mana").get_attribute("innerHTML"))
Using xpath and text attribute:
print(driver.find_element(By.XPATH, "//button[contains(#class, 'currency-btn')]//div[#class='num flex align-center icon-mana']").text)
To extract the text 740 ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR and text attribute:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "button[class*='currency-btn'] div.num.flex.align-center.icon-mana"))).text)
Using XPATH and get_attribute("innerHTML"):
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//button[contains(#class, 'currency-btn')]//div[#class='num flex align-center icon-mana']"))).get_attribute("innerHTML"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
References
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium
I'm using Selenium and I want to grab the 'a' tag so I can navigate into my user profile. What is the best way to do this?
Here is the HTML:
<div class="parent-cont">
<div class="akd3 dafk4 dfan4">...</div>
<div class="avndkd dakdf">...</div>
<div class="fjkad fdadj dfakees">
<a aria-label="tag" class="oa2 g5iad jhuo" href="/profile.php?id=1792" role="link" tabindex="0">
<div class="dsks dssks">...</div>
<div class="dka dk2 fdakdd">...</div>
</a>
</div>
</div>
To locate the element you can use either of the following Locator Strategies:
Using css_selector:
element = driver.find_element_by_css_selector("a[aria-label='tag'][href='/profile.php?id=1792']")
Using xpath:
element = driver.find_element_by_xpath("//a[#aria-label='tag' and #href='/profile.php?id=1792']")
Ideally, to locate the element you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "a[aria-label='tag'][href='/profile.php?id=1792']")))
Using XPATH:
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[#aria-label='tag' and #href='/profile.php?id=1792']")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
you can use following code as mentioned:
enter code here
a = driver.find_elements_by_tag_name("a")
for a in x:
print(x.get_attribute('arial-label')
The following is the HTML structure:
<div class='list'>
<div>
<p class='code'>12345</p>
<p class='name'>abc</p>
</div>
<div>
<p class='code'>23456</p>
<p class='name'>bcd</p>
</div>
</div>
And there is a config.py for user input. If the user input 23456 to config.code, how can the selenium python select the second object? I am using find_by_css_selector() to locate and select the object, but it can only select the first object, which is Code='12345'. I tried to use find_by_link_text(), but it is a <p> element not <a> element. Anyone can help.....
To locate the element with respect to the input by the user using Selenium and python you need to to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using variable in XPATH:
user_input = '23456'
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='list']//div/p[#class='code' and text()='" +user_input+ "']")))
Using %s in XPATH:
user_input = '23456'
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='list']//div/p[#class='code' and text()='%s']"% str(user_input))))
Using format() in XPATH:
user_input = '23456'
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='list']//div/p[#class='code' and text()='{}']".format(str(user_input)))))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Try the below xpath:
code = '23456'
element = driver.find_element_by_xpath("//p[#class='code' and text()='" +code +"']")