Python + Selenium: I can't get print text from this div:
<div id="modal-content-18" class="modal-content" data-role="content">
<div>
SignUp Failed. Please Try Again.
</div>
</div>
I tried this:
resp = browser.find_element_by_class_name("modal-content").text
print resp
But it does not work.
Please help me.
I personally prefer xpaths because of cases like these. They can tackle many complex cases as well. Try the following:
resp = browser.find_element_by_xpath('//div[#class="modal-content"]/div').text
print resp
In case the element isn't visible on the screen. The text method will be none. In that case you need the textContent attribute. Use the following then:
resp = browser.find_element_by_xpath('//div[#class="modal-content"]/div').get_attribute("textContent")
print resp
Let me know if it works for you. Also make sure there is only one modal-content on the page. In case there are more than one, your css_selector is insufficient to identify this element. To check this you can run the following.
l = len(browser.find_elements_by_xpath('//div[#class="modal-content"]/div'))
print l
if it returns a number greater than 1, then the modal-content class alone isn't enough and you will need to expand on your selection criteria.
Induce WebDriverWait and visibility_of_element_located() and following locator strategy.
Using CLASS_NAME:
print(WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.CLASS_NAME,"modal-content"))).text)
Using XPATH:
print(WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='modal-content' and #data-role='content']"))).text)
You need to import followings.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
EDITED
Check the textContent attribute value.
print(WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.CLASS_NAME,"modal-content"))).get_attribute("textContent"))
OR
print(WebDriverWait(driver,20).until(EC.visibility_of_element_located((By.XPATH,"//div[#class='modal-content' and #data-role='content']"))).get_attribute("textContent"))
The desired text SignUp Failed. Please Try Again. is within the child <div> so you have to induce WebDriverWait for the desired visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.modal-content[id^='modal-content-'][data-role='content']>div"))).get_attribute("innerHTML"))
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[#class='modal-content' and starts-with(#id, 'modal-content-')][#data-role='content']/div"))).text)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the title attribute through Selenium using Python?
Outro
As per the documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium
Related
I am trying to get the following count of an Instagram account. I believe that the XPATH is correct an exists. Here's a screenshot showing it exists when I search for it:
This is my code:
wait = WebDriverWait(driver, 30)
followers = wait.until(EC.presence_of_element_located((By.XPATH, "/html/body/div[1]/div/div/div/div[1]/div/div/div/div[1]/div[1]/section/main/div/ul/li[2]/button/div/span")))
print(followers.get_attribute("title"))
I have even looked at similar projects that find the following count and our code is almost exactly the same.
The desired element is a dynamic element, so to locate the element instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategy:
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(., 'followers')]//span[#class and #title and text()]"))).get_attribute("title"))
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I'm trying to scrape the audience score from rotten tomatoes. I was able to get reviews but not sure how use selenium to get the "audiencescore"
Source:
<score-board
audiencestate="upright"
audiencescore="96"
class="scoreboard"
rating="R"
skeleton="panel"
tomatometerstate="certified-fresh"
tomatometerscore="92"
data-qa="score-panel"
>
<h1 slot="title" class="scoreboard__title" data-qa="score-panel-movie-title">Pulp Fiction</h1>
<p slot="info" class="scoreboard__info">1994, Crime/Drama, 2h 33m</p>
<a slot="critics-count" href="/m/pulp_fiction/reviews?intcmp=rt-scorecard_tomatometer-reviews" class="scoreboard__link scoreboard__link--tomatometer" data-qa="tomatometer-review-count">110 Reviews</a>
<a slot="audience-count" href="/m/pulp_fiction/reviews?type=user&intcmp=rt-scorecard_audience-score-reviews" class="scoreboard__link scoreboard__link--audience" data-qa="audience-rating-count">250,000+ Ratings</a>
<div slot="sponsorship" id="tomatometer_sponsorship_ad"></div>
</score-board>
Code:
from selenium import webdriver
driver = webdriver.Firefox()
url = 'https://www.rottentomatoes.com/m/pulp_fiction'
driver.get(url)
print(driver.find_element_by_css_selector('a[slot=audience-count]').text)
The attribute value of audiencescore which is not any text nodes value that's why we can't invoke .text method to grab that value. So you have to call get_attribute() after selecting the right locator. The following expression is working.
print(driver.find_element(By.CSS_SELECTOR,'#topSection score-board').get_attribute('audiencescore'))
#import
from selenium.webdriver.common.by import By
You were close enough. To extract the value of the audiencescore attribute i.e. the text 96 ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://www.rottentomatoes.com/m/pulp_fiction")
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "score-board.scoreboard"))).get_attribute("audiencescore"))
Using XPATH:
driver.get("https://www.rottentomatoes.com/m/pulp_fiction")
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//score-board[#class='scoreboard']"))).get_attribute("audiencescore"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Console Output:
96
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
Try this:
1- Get element score-board
2- Get audiencescore attribute from element
audiencescore = driver.find_element_by_css_selector('score-board').get_attribute('audiencescore')
HTML:
<iframe allowpaymentrequest="true" allowtransparency="true" src="https://shopify.wintopay.com/
cd_frame_id_="ca9e4ad6a1559de159faff5c1f563d59"
name="WinCCPay"
id="win-cc-pay-frame"
I'm trying to input text in a CC field. Apparently its in an iframe I picked the last one in the HTML and tried to select it from the identifiers above but I keep getting the element couldn't be found
iframe= wd.find_element_by_id("win-cc-pay-frame")
wd.switch_to.frame(iframe)
The frame is currently being shown in the browser so no need for implicit wait.
To identify the <iframe> so you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
You can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"iframe#win-cc-pay-frame[name='WinCCPay'][src^='https://shopify.wintopay.com']")))
Using XPATH:
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='win-cc-pay-frame' and #name='WinCCPay'][starts-with(#src, 'https://shopify.wintopay.com')]")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
The problem can be that the name and id of the element are dynamic and change for each unique checkout window? Can you check if adding class attribute at iframe tag and find element by this attribute?
It must be similar to:
iframe = wd.find_element_by_class_name('card-pay-iframe')
wd.switch_to.frame(iframe)
...
wd.switch_to.default_content()
good coding! ¯_(ツ)_/¯
I need to get the number "3" from this HTML with python selenium
<div class="number">3</div>
This is the XPATH:
//*[#id="roulette-recent"]/div/div[1]/div[1]/div/div
I tried something like
number = navegador.find_element_by_xpath('//*[#id="rouletterecent"]/div/div[1]/div[1]/div/div').get_attribute('class')
If this xpath
//*[#id="rouletterecent"]/div/div[1]/div[1]/div/div
represent the node:
<div class="number">3</div>
and you want to extract the text from it, you should use either:
number = navegador.find_element_by_xpath('//*[#id="rouletterecent"]/div/div[1]/div[1]/div/div').get_attribute('innerText')
print(number)
or
number = navegador.find_element_by_xpath('//*[#id="rouletterecent"]/div/div[1]/div[1]/div/div').text
print(number)
I think you're looking for:
number = navegador.find_element_by_xpath('//*[#id="rouletterecent"]/div/div[1]/div[1]/div/div').text
To print the text 3 you can use either of the following Locator Strategies:
Using css_selector and get_attribute("innerHTML"):
print(navegador.find_element(By.CSS_SELECTOR, "#roulette-recent div.number").get_attribute("innerHTML"))
Using xpath and text attribute:
print(navegador.find_element(By.XPATH, "//*[#id="roulette-recent"]//div[#class='number' and text()]").text)
Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR and text attribute:
print(WebDriverWait(navegador, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#roulette-recent div.number"))).text)
Using XPATH and get_attribute("innerHTML"):
print(WebDriverWait(navegador, 20).until(EC.visibility_of_element_located((By.XPATH, "//*[#id="roulette-recent"]//div[#class='number' and text()]"))).get_attribute("innerHTML"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python
References
Link to useful documentation:
get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium
I need help,
I'm trying to print the link from a website,
Here is how the code looks like:
<div class="tabla_szoveg">
<div style="cursor:pointer" onclick="konyvjelzo('1523442');" class="torrent_konyvjelzo2"></div>
I'm trying to print the number inside "konyvjelzo('1523442');"
Using selenium
Also tried:
linkgettr= driver.find_element_by_class("box_torrent_all")
but getting NONE
Thanks
To print the partial value of onclick event i.e. 1523442 you have to induce WebDriverWait for the desired visibility_of_element_located() and you can use the following solution:
Using CSS_SELECTOR:
print(WebDriverWait(browser, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.tabla_szoveg>div.torrent_konyvjelzo2"))).get_attribute("onclick").split("'")[1])
Using XPATH:
print(WebDriverWait(browser, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[class='tabla_szoveg']/div[#class='torrent_konyvjelzo2']"))).get_attribute("onclick").split("'")[1])
Please check if torrent_konyvjelzo2 is dynamic ,if not then you can also replace below torrent text with it otherwise you can use below code as it is.
split("'")[1] is used to split your konyvjelzo('1523442'); text
konyvjelzo( --> item 0
1523442 --> item 1
); --> item 2
the first item index starts from 0. so we can return 1 item.
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//div[contains(#class, 'torrent')]")))
attribute=element.get_attribute("onclick")
print attribute.split("'")[1]
i think you could just use simple regex to find this easily
ids = re.findall("onclick=\"konyvjelzo\('(.*?)'\);",page_text)