Develop Reference

Selenium find_elements only works if time passes since get()

Running these 2 lines together in Colab returns []:
wd.get("https://wetransfer.com/")
wd.find_elements(By.CSS_SELECTOR, 'input[type=file]')
However, running one, followed by the other returns the expected result:
[<selenium.webdriver.remote.webelement.WebElement (session="3cdfb3afbb591862e909cd406b6ac523", element="19fd31e8-710a-4b6e-8284-9a7409f12718")>,
<selenium.webdriver.remote.webelement.WebElement (session="3cdfb3afbb591862e909cd406b6ac523", element="837097d1-5735-4b24-9cb2-9d3ded3a0311")>]
Get is supposed to be blocking so not sure what is going on here.

This is how basically Selenium works.
It can access web elements only after they are completely loaded.
This is why we use implicitly and explicitly waits here.
The explicitly waits are much more recommended.
So instead of
wd.get("https://wetransfer.com/")
wd.find_elements(By.CSS_SELECTOR, 'input[type=file]')
You should use something like this:
wd.get("https://wetransfer.com/")
wait = WebDriverWait(wd, 20)
wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "input[type=file]")))
wd.find_elements(By.CSS_SELECTOR, 'input[type=file]')
To use it you will have to import these imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

On invoking get() as soon as the browser sends 'document.readyState' == "complete" to the driver, Colab executes the next line of code which doesn't find any match as the DOM Tree have completely not loaded. Hence you see Colab returning []
To locate the visible elements you need to induce WebDriverWait for the visibility_of_all_elements_located() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "input[type=file]")))
Using XPATH:
elements = WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//input[#type=file]")))
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can use the following code as an example, to wait until your the site is loaded completly and ready for manipulations.
myElem = WebDriverWait(driver, delay).until(EC.element_to_be_clickable((By.CLASS_NAME , 'bs_btn_buchen')))

grabbing dynamic element in a table using python selenium

the link I'm trying to scrape is https://hcad.org/property-search/real-property/real-property-search-by-account-number/
I'm using this method currently
driver.get('https://hcad.org/property-search/real-property/real-property-search-by-account-number/')
driver.implicitly_wait(2)
driver.find_element_by_xpath('/html/body/div[1]/div[3]/div/h1').click()
actions.send_keys(Keys.TAB * 2 )
actions.send_keys(account_num)
actions.send_keys(Keys.TAB)
actions.send_keys(Keys.RETURN)
actions.perform()
to type the account number in search box and search it which gives me a new form.
I tried to find the input area using full xpath but it never works.
Is there is anyway I can write account num using element xpath or I can excess the form elements other then actions method.
I tried different methods.
like searching input area using (id,full xpath) using driver implicity wait function but none of them work please help me here so that I can grab elements using your method on the form I get after typing account number I'm lost.

You are using wrong locator.
Also, I don't understand why are you using Actions instead of Selenium.send_keys()?
I would do this as following:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
wait = WebDriverWait(driver, 20)
driver.get('https://hcad.org/property-search/real-property/real-property-search-by-account-number/')
wait.until(EC.frame_to_be_available_and_switch_to_it(driver.find_element_by_xpath("//iframe")))
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'input[name="searchval"]'))).send_keys(account_num)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, 'input[type="submit"]'))).click()

The search field is in iframe, you would need to switch the driver focus :
code :
driver.maximize_window()
wait = WebDriverWait(driver, 10)
driver.get('https://hcad.org/property-search/real-property/real-property-search-by-account-number/')
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[src='https://public.hcad.org/records/Real.asp']")))
driver.find_element_by_id('acct').send_keys("12345")
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input[value='Search']"))).click()
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Failed to locate the webdriver element using Selenium |

Using selenium automation webdriver i'm unable to locate the text box element on a travel website using python. Using locator present in the webdriver such as Id/name/css_selector/class_name or xpath/full xpath.
Below is the screenshot of the python code:
[Code_1][1]
While the first one is located the second one isn't. The corresponding HTML code is
[text_box2][2]
How can i fill(automate) both fileds corresponding flight destinations i.e leaving and going
[1]: https://i.stack.imgur.com/gpoIr.jpg
[2]: https://i.stack.imgur.com/Q7mGs.jpg

Here is a slightly different approach for locating things. I am using waits borrowed from CruisePandey in this thread. I used firefox, but that is adaptable. Some notes of what was hard:
I had to make sure to be on the Flights tab.
I had to click in the from and to fields, which were buttons after all, and then wait to be able to type into the revealed input fields.
Finally, I had to choose the first element from the dropdown list that resulted.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox(executable_path='/usr/bin/geckodriver')
driver.maximize_window()
driver.get('https://www.expedia.co.in')
wait = WebDriverWait(driver, 15)
wait.until(EC.presence_of_element_located((By.LINK_TEXT, "Flights")))
driver.find_element_by_link_text('Flights').click()
wait.until(EC.presence_of_element_located((By.ID, "location-field-leg1-origin")))
driver.find_element(By.CLASS_NAME,'uitk-faux-input').click()
fieldInput = driver.find_element_by_id('location-field-leg1-origin')
wait.until(EC.visibility_of(fieldInput))
fieldInput.send_keys("SFO")
wait.until(EC.presence_of_element_located((By.TAG_NAME, "strong")))
driver.find_element_by_tag_name('strong').click()
driver.find_elements(By.CLASS_NAME,'uitk-faux-input')[1].click()
fieldInput = driver.find_element_by_id('location-field-leg1-destination')
wait.until(EC.visibility_of(fieldInput))
fieldInput.send_keys("BOS")
wait.until(EC.presence_of_element_located((By.XPATH,"//strong[contains(text(), 'Boston')]")))
driver.find_element_by_xpath("//strong[contains(text(), 'Boston')]").click()

You may want to use the below xpath for going To input field :
//label[text()='Going to']//following-sibling::input[#name]
Code :
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//label[text()='Going to']//following-sibling::input[#name]"))).send_keys('something')
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

if you want to automate from 1 place holder to another, i have use this set of code & it works for me
wait.until(EC.element_to_be_clickable((By.XPATH,"//*[#id='location-field-leg1-origin-menu']/div[2]/ul/li[1]/button"))).click()
b = wait.until(EC.element_to_be_clickable((By.XPATH, ".//*[#id='location-field-leg1-destination-menu']/div[1]/button"))).send_keys("NYC")
b = wait.until(EC.element_to_be_clickable((By.XPATH,".//*[#id='location-field-leg1-destination-menu']/div[2]/ul/li[1]/button"))).click()
c = wait.until(EC.element_to_be_clickable((By.XPATH, ".//*[#id='d1-btn']"))).click()

How to create an automation for tplink pharos cpe520 using xpath with selenium and python for log in?

I'm trying to create an automation for my tplink pharos cpe520
This is the full xpath
"/html/body/div[4]/div/div[4]/div/div/div/div/div[1]/div[2]/div[1]/div[2]/div[2]/div[2]/div[1]/span[2]/input" it never change. I have to use xpath because every time the id is changed.
this is the xpath
//*[#id="widget--95952b3d-c134-3cfe-dd46-1a85b70c6882"]/div[2]/div[1]/span[2]/input
this is a new xpath
//*[#id="widget--059a411f-7134-3cfe-ec40-3c71bd80af37"]/div[2]/div[1]/span[2]/input
as you can see it changed
1

Try below xpath :
driver.find_element_by_xpath("//input[#type='text']")
or
wait = WebDriverWait(driver, 30)
wait.until(EC.element_to_be_clickable((By.XPATH, "//input[#type='text']']"))).send_keys("Test")
Note : add below impports to your solution
from selenium import webdriver
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By

The problem was solved with "implicitly_wait(5)" before I get the link. It was not a problem from xpath.

How to find the Xpath of the element as per the html through Selenium and Python

Tried something like this in python unable I want to click on cross using Selenium.
driver.find_element_by_xpath("//span[contains(#onclick, 'parent.$WZRK_WR.closeIframe('60005','intentPreview');')]").click()

As per the HTML you have shared to click on the element depicted as X, first you need to induce WebDriverWait while switching to the <iframe> and again induce WebDriverWait for the desired element to be clickable and you can use the following solution:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//iframe[#id='wiz-iframe-intent']")))
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='CT_Interstitial']//span[#class='CT_InterstitialClose']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

Open below URL and Click on Add to Chrome button.
Xpath Helper- https://chrome.google.com/webstore/detail/xpath-helper/hgimnogjllphhhkhlmebbmlgjoejdpjl?hl=en
Once plugin added to Chrome. Open your application/webpage and press SHIFT+CTRL+X. Black window will appear on top then you can play with xpath.
If xpath is invalid- then you will get xpath error and if no matches found then it will display NULL.
Note- To check element attribute still you can use F12 that is nothing but default inspect element for all browser then check attribute and create your own xpath and try.
Xpath Syntax
// tagname[#attribute-name=’value1′] and if you are not sure about tag name then no worry you can try with * also
//*[#attribute-name='value1']

You are dealing with an iframe. Follow below steps :
You'll need to switch control to iframe.
Then perform your action (in this case 'Click close').
Switch the control back to default frame.

You can try with this css_selector :
div.CT_InterstitialContents+span.CT_InterstitialClose[onclick]
Xpath would be :
//div[#class='CT_InterstitialContents']/following-sibling::span[#class='CT_InterstitialClose' and onclick]

Try to use this xPath:
//div[#id = 'contentDiv']/div/div/span[#class = 'CT_InterstitialClose']
Code:
close_btn = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[#id = 'contentDiv']/div/div/span[#class = 'CT_InterstitialClose']")))
close_btn.click()
Imports:
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.by import By
Explanation:
WebDriverWait is used to wait until element will be clickable, and only then clicks on it. In this example WebDriverWait will wait at least 10 seconds until element will be clickable.
PS: as I see in the screenshot your element is probably in iframe. That means you have to switch to this iframe first, to be able to interact with the elements in it. The code sample for it would be like this:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "XPATH_TO_FRAME")))
# do stuff
close_btn = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//div[#id = 'contentDiv']/div/div/span[#class = 'CT_InterstitialClose']")))
close_btn.click()
# switch back to default content
driver.switch_to.default_content()

You can write xpath using class name of span. Please refer an example below.
//span[#class='amountCharged']