I'm working on a web-scraping project , I encounter a problem that I couldn't locate the element(1H) by using find_element_by_xpath/id/css-selector/class_name and perform click()on it. Does anyone have any ideas how to make it work ? Thanks in advance!
Here's the part of my code
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
from selenium import webdriver
from datetime import timedelta, date
import time
import datetime
import re
import mouse
def scrape():
website = 'https://www.binance.com/en/trade-margin/BTC_USDT'
path = '/Users/admin/Downloads/chromedriver'
driver = webdriver.Chrome(path)
driver.get(website)
driver.maximize_window()
#click on 1H
actions = ActionChains(driver)
one_hour = driver.find_element_by_xpath('//div[#class="css-e2pgpg"][#id='1h']')
actions.click(one_hour).perform()
Here's the html
#it is flexbox
<div class="css-e2pgpg">
<div id="Time" class="css-1pj8e72">Time</div>
<div id="15m" class="css-1pj8e72">15m</div>
<div id="1h" class="css-ktyfxp">1H</div> #i want to locate this element and perform click on it
<div id="4h" class="css-1pj8e72">4H</div>
<div id="1d" class="css-1pj8e72">1D</div>
<div id="1w" class="css-1pj8e72">1W</div>
</div>
I went to see the webpage myself. In my opinion, you don't have to locate the flexbox itself.
Just try with:
driver.find_element_by_id('1h').click();
Also, I would add some kind of wait until the page loads fully. You can use Thread.sleep, implicit waits or waiting for an element.
If you are just looking to click on 1H web element, you can do it by using the below code. We have to induce explicit wait to get the job done.
driver.maximize_window()
wait = WebDriverWait(driver, 30)
driver.get("https://www.binance.com/en/trade-margin/BTC_USDT")
try:
wait.until(EC.element_to_be_clickable((By.XPATH, "//div[text()='1H']"))).click()
print("Clicked")
except:
pass
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
Related
I'm trying to click on button that contains this code :
<div style="text-align:center" id="plus_res">
<img src="https://www.tunlancer.com/theme/images/plus_resv2.png" align="plus d'elements" title="plus d'elements" style="cursor:pointer" onclick="plus_resultat()">
</div>
i've tried :
image = driver.find_element_by_xpath("//img[contains(#src,'https://www.tunlancer.com/theme/images/plus_resv2.png')]")
driver.execute_script("arguments[0].click();", image)
driver.find_element_by_id('plus_res').click()
driver.find_element_by_xpath('//*[#id="plus_res"]/img').click()
But it's showing the NoSuchElementException .
How can i fix it please?
Possibly you have to add delay.
Something like this should work :
time.sleep(5)
driver.find_element_by_css_selector('img[src="https://www.tunlancer.com/theme/images/plus_resv2.png"]').click()
I guess you are missing a delay / wait before accessing that element.
Please try this:
wait = WebDriverWait(driver, 20)
wait.until(EC.visibility_of_element_located((By.XPATH, '//img[#onclick="plus_resultat()"]'))).click()
You will need the following imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Also possibly the element is inside an iframe.
I am trying to automate the location selection process, however I am struggle with it.
So for, I can only open the menu and select the first item.
And my code is:
import bs4
from bs4 import BeautifulSoup
import selenium
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(ChromeDriverManager().install())
url = 'https://www.ebay.com/b/Food-Beverages/14308/bn_1642947?listingOnly=1&rt=nc'
driver.get(url)
soup = BeautifulSoup(driver.page_source, 'html.parser')
button = driver.find_element_by_id('gh-shipto-click') //find the location button
button.click()
button2 = driver.find_element_by_id('gh-shipto-click-body-cnt') //open the menu
button2.click()
driver.find_element(By.XPATH,"//div[#role='menuitemradio']").click() //choose the first location
I believe the attribute "data-makeup-index" (show in the pic) would help, but I don't know how to use it.
Sine some of you may not able to find the "ship to" button. Here is the html code I copied from the web.
<li id="gh-shipto-click" class="gh-eb-li">
<div class="gh-menu">
<button _sp="m570.l46241" title="Ship to" class="gh-eb-li-a gh-icon" aria-expanded="false" aria-controls="gh-shipto-click-o" aria-label="Ship to Afghanistan"><i class="flgspr gh-flag-bg flaaf"></i><span>Ship to</span></button>
<i class="flgspr gh-flag-bg flaaf"></i>
I have found my answer as below:
driver.find_element(By.XPATH,"//div[#role='menuitemradio'][{an integer}]").click()
Although the Ship to was not visible to my location, I've inspected it from different Geo location and guess what I was able to find this elementstyle="display: none;
I've removed the none temporarily, and it got displayed.
<li id="gh-shipto-click" class="gh-eb-li gh-modal-active" style="display:none;" aria-hidden="false">
You could find the element by these XPath and handle the dropdown
#This XPath is taking you to the specific div where you need to provide the element index [1][2][3] the ID of this XPath is dynamic
driver.find_element_by_xpath("((((//*[contains(#id,'nid-')])[2])//child::div/div")
driver.find_element_by_xpath("//*[#class='cn']")
I know you got your solution, but this what I've tried It might help
I want to retrieve from a website every div class='abcd' element using Selenium together with 'waiter' and 'XPATH' classes from Explicit.
The source code is something like this:
<div class='abcd'>
<a> Something </a>
</div>
<div class='abcd'>
<a> Something else </a>
...
When I run the following code (Python) I get only 'Something' as a result. I'd like to iterate over every instance of the div class='abcd' appearing in the source code of the website.
from explicit import waiter, XPATH
from selenium import webdriver
driver = webdriver.Chrome(PATH)
result = waiter.find_element(driver, "//div[#class='abcd']/a", by=XPATH).text
Sorry if the explanation isn't too technical, I'm only starting with webscraping. Thanks
I've used like this. You can also use if you like this procedure.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(PATH)
css_selector = "div.abcd"
results = WebDriverWait(driver, 10).until((expected_conditions.presence_of_all_elements_located((By.CSS_SELECTOR, css_selector))))
for result in results:
print(result.text)
Using selenium to load and page and need to click load more button, but after it clicks load more than 100 times
i m getting this error of element click intercepted. because after 100 times the page takes time to load. and code doesnt know where to click.
Tried increasing sleep time to 20 seconds also but at some points if page takes more than 20 seconds the code returns error
error :
ElementNotInteractableException: Message: element not interactable (Session info: chrome=75.0.3770.100)
code :
from selenium import webdriver
import time
import pandas as pd
driver = webdriver.Chrome('/Users/1/chromedriver.exe')
driver.get('https://simpletire.com/catalog?select=1&brand=61&query=catalog')
click_more=True
while click_more:
time.sleep(2)
driver.find_element_by_css_selector(".btn.btn-primary.btn-lg").click();
Consider introducing Explicit Wait to ensure that the button is there prior to attempting clicking it
Example code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('/Users/1/chromedriver.exe')
driver.get('https://simpletire.com/catalog?select=1&brand=61&query=catalog')
click_more=True
while click_more:
WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CSS_SELECTOR, ".btn.btn-primary.btn-lg"))).click()
More information:
Python Selenium: Wait Support
Python Selenium: Waits
How to use Selenium to test web applications using AJAX technology
In these situations I typically stop the page load, perhaps scroll a couple of times and use screen coordinates with a macro tool like AppRobotic to click the button. Something like this should work for you:
import win32com.client
from win32com.client import Dispatch
x = win32com.client.Dispatch("AppRobotic.API")
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('/Users/1/chromedriver.exe')
driver.get('https://simpletire.com/catalog?select=1&brand=61&query=catalog')
# wait 20 seconds per question
x.Wait(20000)
# scroll down a couple of times in case page javascript is waiting for user interaction
x.Type("{DOWN}")
x.Wait(2000)
x.Type("{DOWN}")
x.Wait(2000)
# forcefully stop pageload at this point
driver.execute_script("window.stop();")
# if clicking with Selenium still does not work here, use screen coordinates
x.MoveCursor(xCoord, yCoord)
x.MouseLeftClick
x.Wait(2000)
It would appear that you might be running into the google ad at the bottom of the page.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
driver = webdriver.Chrome()
driver.get('https://simpletire.com/catalog?select=1&brand=61&query=catalog')
click_more=True
wait = WebDriverWait(driver, 30)
while click_more:
try:
elem = wait.until(EC.visibility_of_element_located((By.XPATH, '//button[#class="btn btn-primary btn-lg"]')),
"Cannot find 'Load More' button")
elem.click()
except TimeoutException:
click_more = False
Using xpath and visibility_of_element_located I get the following exception:
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <button class="btn btn-primary btn-lg">...</button> is not clickable at point (591, 797). Other element would receive the click: <iframe id="google_ads_iframe_/21692090825/ST_FlexFooter_0" title="3rd party ad content" name="google_ads_iframe_/21692090825/ST_FlexFooter_0" width="970" height="90" scrolling="no" marginwidth="0" marginheight="0" frameborder="0" srcdoc="" style="border: 0px; vertical-align: bottom;" data-google-container-id="1" data-load-complete="true"></iframe>
You either need to close the google add or scroll the page down a little before attempting to click the button. Once that is done, the while loop should work.
Try the script below to keep clicking on that button until there is none left to click.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.support import expected_conditions as EC
with webdriver.Chrome() as driver:
wait = WebDriverWait(driver, 10)
driver.get('https://simpletire.com/catalog?select=1&brand=61&query=catalog')
while True:
try:
item = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#load_button > button")))
except TimeoutException:
item = ""
if not item: break
driver.execute_script("arguments[0].scrollIntoView();", item)
item.click()
Im using this code to explore tripadvisor (Portuguese comments)
from selenium import webdriver
from bs4 import BeautifulSoup
driver=webdriver.Firefox()
driver.get("https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-TAP-Portugal#review_425811350")
driver.set_window_size(1920, 1080)
Then Im trying to click the google-translate link
driver.find_element_by_class_name("googleTranslation").click()
But getting this error :-
WebDriverException: Message: Element is not clickable at point (854.5, 10.100006103515625). Other element would receive the click: <div class="inner easyClear"></div>
So the div class="inner easyClear" is getting the click. I tried exploring it
from bs4 import BeautifulSoup
page=driver.page_source
for i in page.findAll("div","easyClear"):
print i
print "================="
But was unable to get any intuition from this as in what changes to incorporate now to make the "Google Translate" clickable. Please help
===============================EDIT===============================
Ive also tried these
driver.execute_script("window.scrollTo(0, 1200);")
driver.find_element_by_class_name("googleTranslation").click()
Resizing the browser to full screen etc..
What worked for me was to use an Explicit Wait and the element_to_be_clickable Expected Condition and get to the inner span element:
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
driver.get("https://www.tripadvisor.com.br/ShowUserReviews-g1-d8729164-r425802060-TAP_Portugal-World.html")
wait = WebDriverWait(driver, 10)
google_translate = wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".googleTranslation .link")))
actions = ActionChains(driver)
actions.move_to_element(google_translate).click().perform()
You may also be getting into a "survey" or "promotion" popup - make sure to account for those.