I'm trying to find the search box on tripadvisor but sometimes it is not able to find it. (I get timeout error). I have added WebDriverWait and time.sleep but to no avail. I am still in the midst of testing so there is a part of my code requiring human intervention below.
Edit: Changing to time.sleep(10) instead of 5 seconds helps. But it makes the code very slow. Hoping to use WebDriverWait instead...
searchbox = WebDriverWait(driver,20).until(EC.presence_of_element_located((By.XPATH, "//input[#placeholder='Where to?']")))
Below is my full code, where I have to find the searchbox multiple times for each country.
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome(executable_path=r'C:\\Users\\user\\Downloads\\chromedriver.exe')
driver.get("https://www.tripadvisor.com.sg/Hotels-g293951-Malaysia-Hotels.html")
Human intervention here: Click on a random check-in date, check-out date and then click "Update"
countries3 = ["France","Qatar","Brunei Darussalam","Hong Kong"]
countries4 = ["Germany","The Netherlands","Nigeria","South Africa","Russia"]
countries2 = ["New York","Saudi Arabia","Sri Lanka","Switzerland","Turkey"]
countries1 = ["United Arab Emirates","New Zealand","Thailand","India"]
countries5 = ["Vietnam","Australia","Malaysia","Cambodia","Indonesia","Laos","Myanmar","Philippines","Thailand","Vietnam","Australia", "Canada","China","Japan","Taiwan","United Kingdom","United States","Bangladesh","Belgium","Egypt","Finland"]
countries = countries1+countries2+countries3+countries4+countries5
for country in countries:
driver.get(URL)
driver.implicitly_wait(10)
time.sleep(5)
searchbox = WebDriverWait(driver,20).until(EC.presence_of_element_located((By.XPATH, "//input[#placeholder='Where to?']")))
searchbox.send_keys(country) #select your destination here
time.sleep(2)
possible = driver.find_elements_by_class_name("_3a_rGDNB")
for i in possible:
if i.text == country:
i.click()
break
time.sleep(5)
hotels = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//*[#title="Hotels"]')))
driver.execute_script("arguments[0].click();", hotels)
time.sleep(3)
To find the search box on Tripadvisor instead of presence_of_element_located() you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
searchbox = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='q'][title='Search']")))
Using XPATH:
searchbox = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#name='q' and #title='Search']")))
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
To entered value in search input box
Use WebDriverWait() and wait for element_to_be_clickable() and following xpath.
searchbox = WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.XPATH, "//input[#aria-label='Search']")))
Related
I would like to scrape data from this page: https://www.investing.com/equities/nvidia-corp-financial-summary.
There are two buttons that I'd like to click:
Accept button.
Checking the XPath of the button:
XPath = //*[#id="onetrust-accept-btn-handler"]
Replicating the steps performed here: Clicking a button with selenium using Xpath doesn't work
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
wait = WebDriverWait(driver, 5)
link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id="onetrust-accept-btn-handler")))
I got the error: SyntaxError: invalid syntax
Annual button
there is a toggle between Annual and Quarterly (default is quarterly)
XPath is //*[#id="leftColumn"]/div[9]/a[1]
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id="leftColumn"]/div[9]/a[1]")))
also returned invalid Syntax.
Updated Code
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
company = 'nvidia-corp'
driver = webdriver.Chrome(path)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
wait = WebDriverWait(driver, 2)
accept_link= wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="onetrust-accept-btn-handler"]')))
accept_link.click()
scrollDown = "window.scrollBy(0,500);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll
driver.maximize_window()
time.sleep(10)
wait = WebDriverWait(driver, 2)
scrollDown = "window.scrollBy(0,4000);"
driver.execute_script(scrollDown)
#scroll down to get the page data below the first scroll
try:
close_popup_link= wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="PromoteSignUpPopUp"]/div[2]/i')))
close_popup_link.click()
except NoSuchElementException:
print('No such element')
wait = WebDriverWait(driver, 3)
try:
annual_link = wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="leftColumn"]/div[9]/a[1]')))
annual_link()
# break
except NoSuchElementException:
print('No element of that id present!')
The first accept button was successfully clicked,
but clicking the Annual button returns Timeout Exception error.
Annual button
At least for me I saw we need to use another locator to access that element.
I used scrolling until I can click that element.
The following code works for me:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("--start-maximized")
s = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=s)
company = 'nvidia-corp'
wait = WebDriverWait(driver, 5)
driver.get(f"https://www.investing.com/equities/{company}-financial-summary")
try:
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="PromoteSignUpPopUp"]/div[2]/i'))).click()
except:
pass
while True:
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[#class='float_lang_base_1']//a[#data-ptype='Annual']"))).click()
break
except:
driver.execute_script("window.scrollBy(0, arguments[0]);", 1000)
You need to take care of a couple of things here as follows:
If you are supplying the xpath within double qoutes, i.e. "..." then the attribute values needs to be within single quotes, i.e. '...'
Similarly, if you are supplying the xpath within single qoutes, i.e. '...' then the attribute values needs to be within double quotes, i.e. "..."
This take care of both the SyntaxError: invalid syntax
Effectively, the lines of code will be:
link= wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id='onetrust-accept-btn-handler')))
and
wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id='leftColumn']/div[9]/a[1]")))
Solution
To click on the clickable elements you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:
Clicking on I Accept:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#onetrust-accept-btn-handler"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#id='onetrust-accept-btn-handler']"))).click()
Clicking on Annual:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[data-ptype='Annual']"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#data-ptype='Annual']"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
I'm new to Python. To enter the Tradingview.com site with Selenium library.
I wrote the following code and used Xpath and CSS selector to give the address, and the Click method, but it does not work properly. Has anyone solved this problem?
import time
from selenium import webdriver
from webdriver_manager.firefox import GeckoDriverManager
driver = webdriver.Firefox(executable_path=GeckoDriverManager().install())
longInfo = ["xxx.gmail.com", "11"]
try:
driver.get("https://www.tradingview.com/#signin")
driver.set_page_load_timeout(20)
driver.maximize_window()
# click email Button for logging page
driver.find_element_by_xpath("/html/body/div[7]/div/div[2]/div/div/div/div/div/div/div[1]/div[4]/div/span").click()
time.sleep(5)
driver.find_element_by_css_selector(
"#email-signin__user-name-input__e07a4b49-2f94-4b3e-a3f8-934a5744fe02").send_keys(longInfo[0])
driver.find_element_by_css_selector(
"#email-signin__password-input__e07a4b49-2f94-4b3e-a3f8-934a5744fe02").send_keys(longInfo[1])
# click sign in Button
driver.find_element_by_xpath(
"/html/body/div[7]/div/div[2]/div/div/div/div/div/div/form/div[5]/div[2]/button/span[2]").click()
input("type for exit")
driver.quit()
except Exception as e:
print(e)
driver.quit()
It seems the locator you have used its dynamic, you need to identify the element with better approach.
Its required synchronization time while navigating.
Use explicit wait and wait for element to be clickable.
longInfo = ["xxx.gmail.com", "11"]
driver.get("https://www.tradingview.com/#signin")
wait=WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Email']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "[name='username']"))).send_keys(longInfo[0])
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "[name='password']"))).send_keys(longInfo[1])
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[.//span[contains(., 'Sign in')]]"))).click()
Use following libraries.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
These values of the id attribute:
email-signin__user-name-input__e07a4b49-2f94-4b3e-a3f8-934a5744fe02
email-signin__password-input__e07a4b49-2f94-4b3e-a3f8-934a5744fe02
are dynamically generated and would change everytime you access the webpage afresh. Instead you need to use locator strategies based on static attributes.
Solution
To login within Trading View website you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following locator strategies:
driver.get("https://www.tradingview.com/#signin")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[text()='Email']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#name='username']"))).send_keys("matin_mhz#stackoverflow.com")
driver.find_element(By.XPATH, "//input[#name='password']").send_keys("matin_mhz" + Keys.RETURN)
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
On twitch.tv/esl_csgo I want to click on the channel points button/icon but it keeps giving me the Error
Message: no such element: Unable to locate element
I have searched and tried various methods of finding the element for over 4 hours and I have not found a way to click on the element
This is my code but it can not click on the button I want to, help would really be appreciated.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import time
options = Options()
options.add_argument("--user-data-dir=C:\\Users\\Me\\Desktop\\UserData")
options.page_load_strategy = 'normal'
driver = webdriver.Chrome(options=options)
driver.get("https://twitch.tv/esl_csgo")
time.sleep(10)
element = driver.find_element_by_xpath('//*[#id="c7037441c8fd58e7e0ac6326babcf03d"]/div/div[1]/div/div/div/div/div/section/div/div[5]/div[2]/div[2]/div[1]/div/div/div/div[1]/div[2]/button/div/div/div/div[2]/span')
element.click()
To click on the first channel points button/icon you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div[class^='InjectLayout-sc'] > div span[data-test-selector]"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[starts-with(#class, 'InjectLayout-sc')]/div//span[#data-test-selector]"))).click()
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://twitter.com/i/flow/signup")
driver.implicitly_wait(10)
setname = driver.find_element_by_name("name")
setname.click()
setname.send_keys("NAME SURNAME")
driver.implicitly_wait(10)
emailoption = driver.find_element_by_xpath("/html/body/div/div/div/div[2]/main/div/div/div/div[2]/div[2]/div/div[4]") #selenium can not find the element XPATH (I found it manually)
emailoption.click() #need to click in the element :)
driver.close()
Selenium can not find the element XPATH (I found it manually) ---Use email instead---.
Tried to find this element other ways... same result.
Maybe there is another way to click in it?
Add a Selenium expected condition (element_to_be_clickable) to your code and use relative XPath. To input your name, click on the link and input your email, you can use :
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='name']"))).send_keys('name')
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='tel']/following::span[1]']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='email']"))).send_keys('email')
Be sure to add the following imports :
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
If it still fails you can use Javascript :
name = "your_name"
email = "your_email"
elem = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='name']")))
self.driver.execute_script("arguments[0].setAttribute('value', '" + name +"')", elem)
elem2 = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='tel']/following::span[1]']")))
self.driver.execute_script("arguments[0].click();", elem2)
elem3 = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#autocomplete='email']")))
self.driver.execute_script("arguments[0].setAttribute('value', '" + email +"')", elem3)
The elements within Twitter Signup page are React elements. So to send a character sequence to the Name field you have to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
Using CSS_SELECTOR:
driver.get("https://twitter.com/i/flow/signup")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[name='name']"))).send_keys("Manoel Augusto")
Using XPATH:
driver.get("https://twitter.com/i/flow/signup")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[#name='name']"))).send_keys("Manoel Augusto")
Note: You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
Reference
You can find a detailed relevant discussion in:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element error sending text to Email field in twitter with Selenium Python
Try below code -
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Chrome()
action = ActionChains(driver)
wait = WebDriverWait(driver, 20)
driver.get('https://twitter.com/i/flow/signup')
time.sleep(5) # Add wait here so that twitter login page will load.
NameElement = driver.find_element_by_xpath("//input[#name='name']")
action.move_to_element(NameElement).click().perform()
NameElement.send_keys("Hello")
EmailOption = driver.find_element_by_xpath('//span[text()="Use email instead"]/parent::div')
EmailOption.click()
driver.close()
I ran your code on my computer and It worked perfectly, finding the email option and clicking on it. I also found the same xpath you provided, have you tried running your program in a debugger, and What IDE/text editor are you using? Also, are you leaving the window open when you are running your program? If you close the window it will throw an error. If you have any more details on what is happening when you run your code let me know I'd love to help.
Try below code :
wait = WebDriverWait(driver, 10)
driver.get("https://twitter.com/i/flow/signup")
driver.implicitly_wait(10)
setname = driver.find_element_by_name("name")
setname.click()
setname.send_keys("NAME SURNAME")
driver.implicitly_wait(10)
element1 = wait.until(EC.element_to_be_clickable((By.XPATH, "//body//div[4]")))
element1.click()
Output::
I am new to the world of python and I am trying to select a couple of options on the following website and then click the search button to update results. However, I cannot get the button to respond.
I tried using search button.click() and .submit() and I have tried to implicitly wait. I have also used the code below to wait until the button is clickable. When executing the code, it highlights the button but doesn't seem to release the click; almost like a half click.
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Safari()
driver.get('https://leasing.com/personal/car-leasing/')
element = driver.find_element_by_id('selUpfront')
select = Select(element)
select.select_by_value("3")
element = driver.find_element_by_id('selMileage')
select = Select(element)
select.select_by_value("8000")
searchbutton = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.ID, "search-button")))
searchbutton.click()
I would expect the search results to be updated with the conditions above.
Seems you were close. To click() on the element you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using CSS_SELECTOR:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button#search-button>i.fa.fa-search#search-button-icon"))).click()
Using XPATH:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#id='search-button']/i[#class='fa fa-search' and #id='search-button-icon']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Browser Snapshot:
There are 2 elements with the search-button you need to use the xpath to locate specific
searchbutton = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//button[#id='search-button']")))
searchbutton.click()