I’m trying to retrieve text from a webpage marked as a SPAN within a CLASS. I’ve tried this with Xpath, but this won’t work because the tag is encountered multiple times. I use Jupyter Notebook to write the program.
Here is an example from Instagram:
<div class="C4VMK">
<a class="FPmhX notranslate TlrDj" title="henkbrinkman1994"
href="/henkbrinkman1994/">henkbrinkman1994</a>
<span>Awesome!</span>
</div>
In this case i want to get the text 'Awesome!' in the SPAN tag.
How can I do this in Selenium Python?
Example
I don't have an Instagram account, nor do I have permission to use automation to collect information from their site (see their terms of service), so I can't really test this. The idea is that you would use find_element_by_xpath() to find the particular post (or find_elements_by_xpath() to get all of them).
my_post = driver.find_element_by_xpath('/xpath/to/a/post')
Then for each post use the same method to get the list of comments:
post_comments = my_post.find_elements_by_xpath('./relative/xpath/to/comments')
You can then loop through the objects in post_comments to get the text.
for post in post_comments:
print post.text
[there are probably more efficient ways of doing this, but this will get you started]
The desired element looks to be a dynamic element so to get the comment with text as Awesome! you need to induce WebDriverWait for the element to be visible and you can use either of the following solutions:
XPATH#1:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(#class,'notranslate') and contains(.,'henkbrinkman1994')]//following::span[1]"))).get_attribute("innerHTML"))
XPATH#2:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//a[contains(#class,'notranslate') and #title='henkbrinkman1994']//following::span[1]"))).get_attribute("innerHTML"))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can try alternative locator strategy because its not 100% surety that your xpath will work every time.
Use below css Selector
div[class='C4VMK'] span
OR
.C4VMK span
Make sure your element has been loaded and visible enough if not then try different ExplicitWait conditions to make them work.
Related
I have an issue with navigating in the listbox made of UL list using Selenium 4.3 and Python 3. The problem is that I don't know how to switch value from default to any other and when I'm trying to use click method on those <li> elements I'm getting only errors.
The screen shot of the html code below:
First I was trying to open whole list by clicking on it and after that choosing the element by the second click. I was trying to access that by ID, like this:
driver.find_element(By.ID, "__list1").click()
But it was an error that the asked element doesn't exist.
Do you have any idea how to navigate to different element from this kind of dropdown list? I know how to deal with this problem if the HTML tag would be a <select>.
As per the HTML:
The element identified through (By.ID, "__list1") is a <ul> tag which are generally not clickable.
Solution
Just like the usecase within the discussion you need to locate a <div> or a <span> element, which being clicked expands the <li> elements within either a <ul> or <ol>.
Your effective code block will be:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "div_or_span_which_expands_li"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='sapUiSimpleFixFlexContent']/ul//li[text()='All Cases']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I was making a webscraper with selenium and bs4 to keep some shopping items' stock in track, there is a load more button I want to click which looks like this
button
and here is the HTML code for it, the website is https://www.nvidia.com/en-us/shop/geforce/?page=1&limit=9&locale=en-us
HTML code of the button
Whenever I try to find the element with bs4 it works but it never works with
driver.find_element_by_class_name("buy-link load-more-btn")
and I need it to click the button, can someone help me out
Am I was passing too many classes at once?
find_element_by_class_name accepts only one class. You are passing multiple classes in it.
Try implementing it with one class only
driver.find_element_by_class_name("load-more-btn").click()
Sort of a combination of the other two suggestions. If I understand Selenium correctly, it basically converts any find_element_* into a css selector, so having a space in the class name messes things up if you put both. This worked for me.
# Removes the Accept Cookies Banner
driver.find_element_by_css_selector("div.green-box-btn").click()
# Performs the click you want
driver.find_element_by_css_selector("input.buy-link").click()
driver.find_element_by_class_name("buy-link load-more-btn").click()
Please add click() to the end.
The LOAD MORE element is located at the bottom of the DOM Tree so to click on it you need to induce WebDriverWait for the element_to_be_clickable() which will automatically scroll the element within the Viewport and invoke click() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.buy-link.load-more-btn[value='LOAD MORE']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#class='buy-link load-more-btn' and #value='LOAD MORE']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I have been trying to get an account balance text to print in python using selenium. I go to chrome, inspect the element, copy the xpath, put it in VS and it says cant find the xpath. I did the same thing for CSS selector, full xpath, tried to create my own xpath, tried by class name, etc. Ive tried every option it has. nothing is working. What do you do when you run into this situation? It is there on the website, its gotta be accessible in the background somewhere. The has to be a way to grab it. I have been able to get other text on the site to display properly.
This is just a stock simulator game, I have it open to the public if you want to log in and try your hand at getting the correct inspect element.
The xpath it generates:
/html/body/div[4]/div[3]/div[1]/div[2]/ul/li[1]/span
How I have it in VS:
driver.find_element_by_xpath('/html/body/div[4]/div[3]/div[1]/div[2]/ul/li[1]/span')
The error in VS:
Unable to locate element: {"method":"xpath","selector":"/html/body/div[4]/div[3]/div[1]/div[2]/ul/li[1]/span"}
To get the amount Net Worth Use WebDriverWait() and wait for visibility_of_element_located() and following xpath.
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//div[./span[text()='Net Worth']]/following::span[1]"))).text)
OR
print(WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.XPATH,"//ul[#class='list list--kv']/li[1]/span"))).text)
You need to import below libraries.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I am having trouble with a selenium send keys on discord. I am attempting to send a message to a user.
The error I get is:
selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable
The HTML is as follows:
The Object im trying to send_keys that is highlighted by the xpath is as follows:
My code is as follows
inputMessage = wait.until(EC.visibility_of_element_located((By.XPATH,"//div[contains(text(),'Message #')]/..")))
#inputMessage = driver.find_element_by_xpath("//div[contains(text(),'Message #')]/..")
inputMessage.send_keys(msg,Keys.ENTER)
I have attempted several ways to try to solve the error but have not succeeded. Any help would be appreciated. Thank you.
The ElementNotInteractableException error tells you that you can't use the send_keys() method on this webElement. I can't really tell you what could work, considering the lack of information considering the problem, but here are some clues :
Using the ActionsChains to try to input what you want in the field:
from selenium.webdriver.common.action_chains import ActionChains
actions = ActionChains(self.driver)
actions.send_keys('your_data')
actions.perform()
Try inputMessage.click() before you try to send keys
Otherwise, it happened to me too, and the thing was that i tried to send keys to the div, and not to the element which was inside it.
Hopes it help !
HTML contenteditable Attribute
By using some JavaScript event handlers you can transform your web page into a full and fast rich text editor only by setting the contenteditable attribute on nearly any HTML element to make it editable. As an example, to create a <div> element whose contents an user can edit would be:
<div contenteditable="true">
This text can be edited by the user.
</div>
When an HTML element has contenteditable set to true, the document.execCommand() method is made available. This lets you run commands to manipulate the contents of the editable region. However, there seem to be some difference in usage of contenteditable across different browsers. As an example, when you press Enter or Return to create a new line of text inside an editable element:
Firefox inserted <br> elements
IE/Opera used <p> elements
Chrome/Safari used <div> elements
Firefox 60 wraps the separate lines in <div> elements, matching the behavior of Chrome, modern Opera, Edge, and Safari.
This usecase
As per the HTML it seems the next <div> with attribute aria-label="Message #Ticketing" has the property contenteditable="true". So ideally you should be able to invoke send_keys() on this element.
Solution
Finally, to interact with an element you need to to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[aria-label='Message #Ticketing'][data-slate-editor='true'][role='textbox']"))).send_keys(msg)
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#aria-label='Message #Ticketing' and #data-slate-editor='true'][#role='textbox']"))).send_keys(msg)
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am pretty new to Python and Selenium, and I have got my script to do what I want so far, but I have this current coding from the website:
<a onclick="realPostBack('ctl00$ctl00$mainContent$ContentPlaceHolder1$ucHub$ucSearchExplorer$dgContents$ctl00$ctl04$lnkContent', ''); return false;" id="ctl00_ctl00_mainContent_ContentPlaceHolder1_ucHub_ucSearchExplorer_dgContents_ctl00_ctl04_lnkContent" class="hub-content-item" actiontype="Secondary" href="javascript:__doPostBack('ctl00$ctl00$mainContent$ContentPlaceHolder1$ucHub$ucSearchExplorer$dgContents$ctl00$ctl04$lnkContent','')"><span>How to Project Wirelessly in Philly</span></a>
and I can't get it to click the link.
I've tried:
driver.find_element_by_text("How to Project Wirelessly in Philly")
and partial text
driver.find_element_by_id("ctl00_ctl00_mainContent_ContentPlaceHolder1_ucHub_ucSearchExplorer_dgContents_ctl00_ctl04_lnkContent")
I've tried by tag but all returning errors. Looking up on here, I've seen stuff with Xpath but I have no clue how to do that, but if someone here does, then a little help with that, or any other simple code that hopefully allows me to click that link. (I know i will have to do .click() to eventually click it, but i cant even find the element yet)
The desired element is a dynamic element, so invoke click() on it you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using CSS_SELECTOR:
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a.hub-content-item[id*='SearchExplorer'][actiontype='Secondary']>span")))
Using XPATH:
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='hub-content-item' and contains(#id,'SearchExplorer')][#actiontype='Secondary']/span[text()='How to Project Wirelessly in Philly']")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC