This is the webpage -- link. I am going to crawl (In Persian). I have a problem when I am going to click on the next page button. The XPath is:
nextpage = '//*[#id="ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom"]/a[1]/img'
page = driver.find_element_by_xpath(nextpage)
page.click()
After the page.click() I got the following error:
selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
Some answers say there might be a duplicate XPath, but I could not find such a thing in the source of the webpage.
The complete code:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.keys import Keys
from selenium.webdriver import ActionChains
driver = webdriver.Chrome(executable_path='./chromedriver')
url = 'http://hmi.mrud.ir/sabaa/SABAA/Home/Default.aspx?strTownship=0101&g=false'
driver.get(url)
time.sleep(10)
nextpage = '//*[#id="ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom"]/a[1]/img'
page = driver.find_element_by_xpath(nextpage)
page.click()
Thanks.
Use the following before clicking on the element:
driver.execute_script("arguments[0].scrollIntoView(true);", page)
This is scrolling to the element.
Hope it helps you!
As per the Website if you want to click on the Next Page Button you have to induce WebDriverWait for the WebElement to be clickable following either of the code block as follows :
nextpage = WebDriverWait(driver, 10).until(EC.element_to_be_clickable(By.XPATH,"//div[#id='ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom']/a[#class='dxp-lead dxp-button dxp-bi']/img[#class='dxWeb_pPrev_Aqua']"))
nextpage.click()
Or
nextpage = WebDriverWait(driver, 10).until(lambda x: x.find_element_by_xpath("//div[#id='ctl00_ContentPlaceHolder1_ASPxSplitter1_CallbackPaneldgd_dgd_DXPagerBottom']/a[#class='dxp-lead dxp-button dxp-bi']/img[#class='dxWeb_pPrev_Aqua']"))
nextpage.click()
Related
I'm trying to use selenium to open up this webpage (https://app.powerbi.com/view?r=eyJrIjoiMzE2ZDIyN2YtODY1Yy00ZGY0LWE4YTktNDcxOTcwYWQyMjM5IiwidCI6IjcyMmVhMGJlLTNlMWMtNGIxMS1hZDZmLTk0MDFkNjg1NmUyNCJ9) and click on the Tram icon to navigate to the web page I want to scrape.
This is what I've tried up to now
from selenium import webdriver
driver=webdriver.Chrome()
driver.get("https://app.powerbi.com/view?r=eyJrIjoiMzE2ZDIyN2YtODY1Yy00ZGY0LWE4YTktNDcxOTcwYWQyMjM5IiwidCI6IjcyMmVhMGJlLTNlMWMtNGIxMS1hZDZmLTk0MDFkNjg1NmUyNCJ9")
driver.maximize_window()
x=driver.find_element("class name", "imageBackground")
print(x)
#driver.find_element("class name", "imageBackground").click()
However, the element returns none. I'm not sure how to find the element and click the div to navigate to the next page as well.
I've updated your code.
I added a WebDriverWait that pauses until the tram element is clickable, then it clicks on the Tram element and goes to the next page.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
if __name__ == "__main__":
driver=webdriver.Chrome()
driver.get("https://app.powerbi.com/view?r=eyJrIjoiMzE2ZDIyN2YtODY1Yy00ZGY0LWE4YTktNDcxOTcwYWQyMjM5IiwidCI6IjcyMmVhMGJlLTNlMWMtNGIxMS1hZDZmLTk0MDFkNjg1NmUyNCJ9")
driver.maximize_window()
wait = WebDriverWait(driver, 30)
tramElementXpath = '//*[#aria-label="Tram Bookmark . Click here to follow link"]'
x = wait.until(EC.element_to_be_clickable((By.XPATH, tramElementXpath)))
x.click()
I am using selenium to try to scrape data from a website (https://www.mergentarchives.com/), and I am attempting to get the innerText from this element:
<div class="x-paging-info" id="ext-gen200">Displaying reports 1 - 15 of 15</div>
This is my code so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.support.ui import WebDriverWait
driver = webdriver.Firefox()
driver.maximize_window()
search_url = 'https://www.mergentarchives.com/search.php'
driver.get(search_url)
assert 'Mergent' in driver.title
company_name_input = '//*[#id="ext-comp-1009"]'
search_button = '//*[#id="ext-gen287"]'
driver.implicitly_wait(10)
driver.find_element_by_xpath(company_name_input).send_keys('3com corp')
driver.find_element_by_xpath(search_button).click()
driver.implicitly_wait(20)
print(driver.find_element_by_css_selector('#ext-gen200').text)
basically I am just filling out a search form, which works, and its taking me to a search results page, where the number of results is listed in a div element. When I attempt to print the text of this element, I simply get a blank space, there is nothing written and no error.
[Finished in 21.1s]
What am I doing wrong?
I think you may need explicit Wait :
wait = WebDriverWait(driver, 10)
info = wait.until(EC.visibility_of_element_located((By.XPATH, "//div[#class = 'x-paging-info' and #id='ext-gen200']"))).get_attribute('innerHTML')
print(info)
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You may need to put a condition by verifying if search results loaded or not and once its loaded you can use below code
print(driver.find_element_by_id('ext-gen200').text)
im quite noob in python and right now building up a web scraper in Selenium that would take all URL's for products in the clicked 'tab' on web page. But my code take the URL's from the first 'tab'. Code below. Thank you guys. Im starting to be kind of frustrated lol.
Screenshot
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from lxml import html
PATH = 'C:\Program Files (x86)\chromedriver.exe'
driver = webdriver.Chrome(PATH)
url = 'https://www.alza.sk/vypredaj-akcia-zlava/e0.htm'
driver.get(url)
driver.find_element_by_xpath('//*[#id="tabs"]/ul/li[2]').click()
links = []
try:
WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.CLASS_NAME, 'blockFilter')))
link = driver.find_elements_by_xpath("//a[#class='name browsinglink impression-binded']")
for i in link:
links.append(i.get_attribute('href'))
finally:
driver.quit()
print(links)
To select current tab:
current_tab = driver.current_window_handle
To switch between tabs:
driver.switch_to_window(driver.window_handles[1])
driver.switch_to.window(driver.window_handles[-1])
Assuming you have the new tab url as TAB_URL, you should try:
from selenium.webdriver.common.action_chains import ActionChains
action = ActionChains(driver)
action.key_down(Keys.CONTROL).click(TAB_URL).key_up(Keys.CONTROL).perform()
Also, apparently the li doesn't have a click event, are you sure this element you are getting '//*[#id="tabs"]/ul/li[2]' has the aria-selected property set to true or any of these classes: ui-tabs-active ui-state-active?
If not, you should call click on the a tag inside this li.
Then you should increase the timeout parameter of your WebDriverWait to guarantee that the div is loaded.
I am currently trying to learn selenium in Python and I am having an issue clicking the "Accept All Cookies" button.
I am using:
Python v3.9
Chrome v87
This is the HTML page i am trying to scrape
https://www.currys.co.uk/gbuk/tv-and-home-entertainment/televisions/televisions/samsung-ue75tu7020kxxu-75-smart-4k-ultra-hd-hdr-led-tv-10213562-pdt.html
Here is my code currently
# Selenium Tutorial #1
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
import time
driver = webdriver.Chrome(r"C:\Users\Ste1337\Desktop\chromedriver\chromedriver.exe")
driver.get("https://www.currys.co.uk/gbuk/tv-and-home-entertainment/televisions/televisions/samsung-ue75tu7020kxxu-75-smart-4k-ultra-hd-hdr-led-tv-10213562-pdt.html")
#search = driver.find_element_by_id(ContentPlaceHolder1_NotifyBtn)
driver.implicitly_wait(10)
link = driver.find_element_by_id("onetrust-accept-btn-handler")
link.click
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "onetrust-accept-btn-handler"))
)
element.click
except:
driver.quit()
The "( )" is missing after the click.
Try this:
link = driver.find_element_by_id("onetrust-accept-btn-handler")
link.click()
Simply wait and click.
wait = WebDriverWait(driver, 10)
driver.get("https://www.currys.co.uk/gbuk/tv-and-home-entertainment/televisions/televisions/samsung-ue75tu7020kxxu-75-smart-4k-ultra-hd-hdr-led-tv-10213562-pdt.html")
wait.until(EC.element_to_be_clickable((By.ID, "onetrust-accept-btn-handler"))).click()
I've written a script in python to click on some categories in a webpage. I could manage to click on the first two categories but got stuck when it comes to initiate the final click. I've given a link leading to the two images in I have marked where to click.
This is the first link where there is a sign (marked with pencil) to click on to enter the second portion.
This is the second link where I get stuck when I try to click on the names (I've marked those names with pencil)
This is the site link.
Script I've tried with so far:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("replace_with_above_link")
wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "i4ewOd-pzNkMb-ornU0b-b0t70b-Bz112c"))).click()
post = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div[role='checkbox']")))[1]
post.click()
for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".HzV7m-pbTTYe-JNdkSc .suEOdc"))):
item.click()
driver.quit()
My intention is to click the names cyclically. Thanks in advance.
Try below code to click each item in list:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(URL)
wait.until(EC.element_to_be_clickable((By.CLASS_NAME, "i4ewOd-pzNkMb-ornU0b-b0t70b-Bz112c"))).click()
post = wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "div[role='checkbox']")))[1]
post.click()
for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR,".HzV7m-pbTTYe-JNdkSc .suEOdc")))[1:]:
item.click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".HzV7m-tJHJj-LgbsSe-Bz112c.qqvbed-a4fUwd-LgbsSe-Bz112c"))).click()
wait.until(EC.invisibility_of_element_located((By.CSS_SELECTOR, ".qqvbed-p83tee")))
driver.quit()