Running these 2 lines together in Colab returns []:
wd.get("https://wetransfer.com/")
wd.find_elements(By.CSS_SELECTOR, 'input[type=file]')
However, running one, followed by the other returns the expected result:
[<selenium.webdriver.remote.webelement.WebElement (session="3cdfb3afbb591862e909cd406b6ac523", element="19fd31e8-710a-4b6e-8284-9a7409f12718")>,
<selenium.webdriver.remote.webelement.WebElement (session="3cdfb3afbb591862e909cd406b6ac523", element="837097d1-5735-4b24-9cb2-9d3ded3a0311")>]
Get is supposed to be blocking so not sure what is going on here.
This is how basically Selenium works.
It can access web elements only after they are completely loaded.
This is why we use implicitly and explicitly waits here.
The explicitly waits are much more recommended.
So instead of
wd.get("https://wetransfer.com/")
wd.find_elements(By.CSS_SELECTOR, 'input[type=file]')
You should use something like this:
wd.get("https://wetransfer.com/")
wait = WebDriverWait(wd, 20)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "input[type=file]")))
wd.find_elements(By.CSS_SELECTOR, 'input[type=file]')
To use it you will have to import these imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
On invoking get() as soon as the browser sends 'document.readyState' == "complete" to the driver, Colab executes the next line of code which doesn't find any match as the DOM Tree have completely not loaded. Hence you see Colab returning []
To locate the visible elements you need to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "input[type=file]")))
Using XPATH:
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//input[#type=file]")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You can use the following code as an example, to wait until your the site is loaded completly and ready for manipulations.
myElem = WebDriverWait(driver, delay).until(EC.element_to_be_clickable((By.CLASS_NAME , 'bs_btn_buchen')))
Related
Trying to automate one thing for my work, which was choosing one option from the dropdown list on the website below:
https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ
The Selenium automation didn't work in that case. After writing such code:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chrome_driver_path = r"C:/Users/.../Projects/chromedriver.exe"
service = Service(executable_path=chrome_driver_path)
driver = webdriver.Chrome(service=service)
driver.get("https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ")
wait = WebDriverWait(driver, 10)
abo_button = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="combobox-button-53"]')))
After executing I've got a message:
TimeoutException
In case of finding element by tag name or any other options, the following message pops up:
Message: no such element: Unable to locate element
The list is built on button tags and has a structure of lightning-basecombobox. It looks like there is no possibility to click on the dropdown list and choose the required option automatically.
Is it needed to do something different with such stuff?
What I expect is to use Selenium to choose between the options in the list.
The combobox is a <button> element.
Additionally the id i.e. combobox-button-53 is dynamically generated and is bound to chage sooner/later. They may change next time you access the application afresh or even while next application startup. So can't be used in locators.
Solution
To click element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Using CSS_SELECTOR:
driver.get("https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[name='subscriptionsTypeId']"))).click()
Using XPATH:
driver.get("https://interparking-pl.my.site.com/abonament/s/?id=a0A58000000D7pZ")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#name='subscriptionsTypeId']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot;
When I load the page, I get the element ID (XPATH) as following. //*[#id="combobox-button-51"]
Maybe the number 51 isn't always the same? In that case, try:
//*[starts-with(#id,'combobox-button-5')]
Or just use //*[#name="subscriptionsTypeId"] , as another answer here allready mentioned.
I'm automating a bot to book a certain time (preferred time and second preferred time).
I need help to use if else statement to allow the browser to search for the preferred time and if not present to search for the second preferred time.
Below is my code:
preferred_time=driver.find_element(By.XPATH,'//div[#class="booking-start-time-label"][contains(., "12:33pm")]')
preferred_time.click()
second_preferd_time= driver.find_element(By.XPATH,'//div[#class="booking-start-time-label"][contains(., "1:33pm")]')
second_preferd_time.click()
An ideal approach will be to try to click on the initial desired element inducing WebDriverWait for element_to_be_clickable() and incase it fails catch the TimeoutException and attempt to click on the second desired element as follows:
try:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="booking-start-time-label"][contains(., "12:33pm")]'))).click()
except TimeoutException:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="booking-start-time-label"][contains(., "1:33pm")]'))).click()
Note: You have to add the following imports :
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I am trying to get the following count of an Instagram account. I believe that the XPATH is correct an exists. Here's a screenshot showing it exists when I search for it:
This is my code:
wait = WebDriverWait(driver, 30)
followers = wait.until(EC.presence_of_element_located((By.XPATH, "/html/body/div[1]/div/div/div/div[1]/div/div/div/div[1]/div[1]/section/main/div/ul/li[2]/button/div/span")))
print(followers.get_attribute("title"))
I have even looked at similar projects that find the following count and our code is almost exactly the same.
The desired element is a dynamic element, so to locate the element instead of presence_of_element_located() you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following locator strategy:
Using XPATH:
print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[contains(., 'followers')]//span[#class and #title and text()]"))).get_attribute("title"))
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
How to access an element on pic: 'captcha__human__submit-description' in a frame using xpath. I need to check the presence of this element, and depending on its presence, continue or terminate the program.
this combination does not find this element:
driver.switch_to(0)
driver.find_element_by_xpath("//dic[#class='captcha__human__submit-description']").text
xml data
try to switch it like this :
driver.switch_to.frame(driver.find_element_by_xpath("//iframe[#onload='iframeOnload()']"))
driver.find_element_by_xpath("//div[#class='captcha__human__submit-description']").text
or with explicit waits :
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[#onload='iframeOnload()']")))
some_text = wait.until(EC.visibility_of_element_located((By.XPATH, "//div[#class='captcha__human__submit-description']"))).text
print(some_text)
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
As already given by CruisePandey, you can switch to frame and then access element using the XPaths. However, I believe you can also use CSS Selectors to do the same thing.
Using Selenium 3 -
driver.find_element_by_css_selector("div.captcha__human_submit-description")
or
driver.find_element_by_css_selector("div.captch__human__container > div:nth-child(2)")
Using Selenium 4 Beta version
driver.find_element(By.CSS_SELECTOR, "div.captcha__human_submit-description")
or
driver.find_element(By.CSS_SELECTOR,"div.captch__human__container > div:nth-child(2)")
I am facing inconsistencies in Selenium execution.
Last line in the code snippet I pasted below doesn't execute consistently. Sometimes it works, sometimes it throws an error saying that element is not found. Doesn't Selenium "block" for the element to appear before attempting to execute the click? I generated it using Selenium IDE. What I am missing here?
self.driver.find_element(By.CSS_SELECTOR, ".dx-ellipsis:nth-child(2)").click()
self.driver.switch_to.default_content()
self.driver.find_element(By.CSS_SELECTOR, "#PageContentPlaceHolder_TimeControlSplitter_TimeControlContent_TimesheetEntrySplitter_TimesheetDetailsMenu_DXI0_T > .dxm-contentText").click()
Selenium may not find elements if they happen to be loaded dynamically by JS and if you search for them before they are loaded.
You can try either an implicit wait or an explicit wait.
In case of implicit waiting, the docs say:
An implicit wait tells WebDriver to poll the DOM for a certain amount of time when trying to find any element (or elements) not immediately available.
You could do with something like:
from selenium import webdriver
driver = webdriver.Chrome()
driver.implicitly_wait(10) #wait and poll for 10 seconds
Whereas the explicit waiting means to explicitly specify the element which is to be waited for it to be available. As per the docs:
An explicit wait is a code you define to wait for a certain condition to occur before proceeding further in the code.
You can do this with something like:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
element1 = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".dx-ellipsis:nth-child(2)")))
element1.click()
self.driver.switch_to.default_content()
element2 = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.CSS_SELECTOR, "#PageContentPlaceHolder_TimeControlSplitter_TimeControlContent_TimesheetEntrySplitter_TimesheetDetailsMenu_DXI0_T > .dxm-contentText")))
element2.click()
As you are using the line of code:
self.driver.switch_to.default_content()
Presumably you are switching Selenium's focus from a frame or iframe to the Top Level Content. Hence you need to induce WebDriverWait for the desired element to be clickable and you can use the following Locator Strategy:
self.driver.switch_to.default_content()
WebDriverWait(self.driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#PageContentPlaceHolder_TimeControlSplitter_TimeControlContent_TimesheetEntrySplitter_TimesheetDetailsMenu_DXI0_T > .dxm-contentText"))).click()
Note:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
References
You can find a couple of relevant detailed discussions in:
How to send text to the Password field within https://mail.protonmail.com registration page?
How to switch between iframes using Selenium and Python?
Wait for the element to be loaded
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
WebDriverWait(self.driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".dx-ellipsis:nth-child(2)"))).click()
self.driver.switch_to.default_content()
WebDriverWait(self.driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#PageContentPlaceHolder_TimeControlSplitter_TimeControlContent_TimesheetEntrySplitter_TimesheetDetailsMenu_DXI0_T > .dxm-contentText"))).click()
The number is how long the driver should spend looking for the element before moving on.