main url :https://www.kaplanpathways.com/degree-finder/#/search-result?status=7&institution_short_name=Arizona-State-University-Downtown-Phoenix-Campus&subject_area_name=&university=38°ree_level=20
try:
for i in range(1, 20):
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
driver.find_element_by_xpath("/html/body/div[5]/main/div/div/app-root/app-search-result/div/div[2]/div[2]/div[2]/button[1]").click()
except:
pass
course_list = driver.find_elements_by_xpath("//*[#class='wrap-result']")
print("Total courses: ", len(course_list))
Problem:
When you are moving to the bottom of the page, you actually move out of the clickable area for the "Show more" button. selenium does not click the button if that is not clickable (i.e. out of the screen, or behind some div like accept cookies div in your case).
Solution:
Try clicking by injecting javascript.
driver.execute_script("""document.querySelector(".dgf-show-more-button").click()""")
Note: Also don't forgot to click the "I accept" button for cookies.
Related
I am trying to scrape user reviews from the IMDB website using selenium. To load more reviews, I instantiated a click event which clicks the "load more" button on the review page . The issue is that the button gets clicked on the review pages of some movies but it does not work on review pages of other movies.
To give an example, the code given below works when the button is to be clicked on the "Iron Man" user review webpage but on the webpage of "Armageddon", it gives the error: NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".ipl-load-more__button"}
I find this a bit odd because the web element is the same and so if it works on one page, it should work for all the pages. However, I am not able to resolve this issue.
Here is the code which instantiates the click event:
def extend_page(wd): # wd is the webdriver
count = 20
while(count>0):
button = wd.find_element_by_class_name('ipl-load-more__button')
button.click()
wd.execute_script('window.scrollTo(0,document.body.scrollHeight);')
time.sleep(3)
new_height = wd.execute_script('return document.body.scrollHeight')
prev_height = new_height
count-=1
page = wd.page_source
return page
I have tried different methods for instantiating the click event such as find_element_by_name and find_element_by_xpath. I also tried both implicit and explicit wait to give some time for the webpage to load but to no avail.
Can anyone please tell me what could be the reason for this?
I am creating a downloader from Instagram. This program gets URLs from top Instagram posts in a given hashtag and inputs them into a downloader. The issue is that a popup iframe ad always appears over the download button when the website is first loaded. This throws an error that the button cannot be clicked because the iframe will be clicked.
This is for Python Selenium running Chrome driver. I have tried to run a filter that finds iframes and goes back to the main page:
all_iframes = self.browser.find_elements_by_tag_name("iframe")
if len(all_iframes) > 0:
self.browser.switch_to.default_content()
This did not work, I also tried to get the XPath to the X-button on the ad, but the ID changes every time, so I cannot be clicked or identified.
#get the website
for link in self.links:
self.browser.get('https://downloadgram.com/')
time.sleep(5)
#this is the x button click iframe I tried, but the XPath changes
all_iframes = self.browser.find_elements_by_tag_name("iframe")
if len(all_iframes) > 0:
xButton = self.browser.find_element_by_xpath('//div[#id="id3019a64023cross3019a64023"]')
xButton.click()
#inputs the URL from array links[] into download box
input = self.browser.find_element_by_xpath('//input[#name="url"]')
input.clear()
input.send_keys(link)
time.sleep(1)
#clicks download button
download = self.browser.find_element_by_xpath("//input[#type='submit']")
download.click()
time.sleep(1)
#clicks confirm button
actuallyDownload = self.browser.find_element_by_xpath("//a[#target='_blank']")
actuallyDownload.click()
time.sleep(1)
I expect the code to download the pictures at the url, but I get:
selenium.common.exceptions.WebDriverException: Message: unknown error: Element <input type="submit" value="Download" class="button"> is not clickable at point (451, 446). Other element would receive the click:
the website (turn off adblock to see add) https://downloadgram.com/
I am able to close the pop up with following code.Please try that.
browser.get('https://downloadgram.com/')
time.sleep(5)
element=browser.find_element_by_xpath("//div[starts-with(#id,'id')]" and "//div[starts-with(#style,'position:absolute !important;height:20px !important;width:20px !important;top:3px !important;left:3px !important;background-image:url(data:image/png;')]")
arrt=element.get_attribute("id")
print(arrt)
browser.execute_script("arguments[0].click();", element)
Let me know if it works.Good Luck.
I need to click on a button using selenium in python. This is what I have:
read_more_buttons = responses[0].find_elements_by_class_name("read-more")
if len(read_more_buttons) > 0:
read_more_buttons[0].click()
It works fine most of the time but sometimes there's an overlay on the bottom of the page, which can not be dismissed. Therefore I'd get this error:
[element] is not clickable at point (665.7333145141602,883.4666748046875) because another element <div class="raq-module js-raq-module"> obscures it
I tried to scroll down the page with this code right before calling click():
driver.execute_script("window.scrollTo(0, " + str(read_more_buttons[0].location["y"] + 120) + ")")
However, I'm still receiving the same error. It seems like by calling .click() the element is scrolled to the very bottom of the page, which is right under the overlay. How can I move up the page and then click?
Those dang overlays!
Here, let's try and use JS to scroll into view and then click:
read_more_buttons = responses[0].find_elements_by_class_name("read-more")
if len(read_more_buttons) > 0:
driver.execute_script("arguments[0].scrollIntoView(true);", read_more_buttons[0])
driver.execute_script("arguments[0].click()", read_more_buttons[0])
I'm trying to click on this button: browser.find_element_by_id('btnSearch')
But this button is blocking by this div tag: <div id="actionSearch" class="row pull-right">
How do I go around to click this button with id='btnSearch" while it's blocking by the actionSearch div?
I tried the following:
browser.find_element_by_id('btnSearch').click()
browser.implicitly_wait(10)
el = browser.find_element_by_xpath('//*[#id="btnSearch"]')
ActionChains(browser).move_to_element_with_offset(el, 1827, 270)
ActionChains(browser).click()
ActionChains(browser).perform()
element = browser.find_element_by_id('btnSearch')
browser.execute_script("arguments[0].click();", element)
wait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, //*[#id="btnSearch"]'))).click()
none of these work.
Can anyone help me with this? I've spent two days trying to click this button!! Please help!
Considering provided image of HTML source (tip: do not provide it as image, but as text instead) I can assume that required element located in the bottom of page and you might need to scroll page down to be able to handle it.
Try below code
link = browser.find_element_by_id("btnSearch")
browser.execute_script("arguments[0].scrollIntoView()", link)
link.click()
Note that link is not a pseudo-element (is not located inside ::before/::after pseudo-element), so it can not be the cause of your problem
As for your code:
ActionChains(browser).move_to_element_with_offset(el, 1827, 270)
ActionChains(browser).click()
ActionChains(browser).perform()
Here you're trying to make scroll to link with huge offset and then you make click on current mouse position - not on link
You can try to modify it as
ActionChains(browser).move_to_element(el)
ActionChains(browser).click(el) # Pass WebElement you want to click as argument to `click()`
ActionChains(browser).perform()
I am trying to click the "more" button for getting next set of pages.
Next page link Htmltags:
<span id="direct_moreLessLinks_listingDiv" class="more_less_links_container" data-num-items="15" offset="45" data-type="listing">
<a id="button_moreJobs" rel="nofollow" href="#">More</a>
</span>
Note: when I am clicking the "more" button. It is loading the next page in the same page.
I am trying to click the next page using selenium:
Code part:
self.driver.get('example.com')
while True:
more_btn = WebDriverWait(self.driver, 10).until(
EC.visibility_of_element_located((By.ID, "More"))
)
More.click()
# stop when we reach the desired page
Find the link by ID and click it:
more_btn = WebDriverWait(self.driver, 10).until(
EC.visibility_of_element_located((By.ID, "button_moreJobs"))
)
more_btn.click()
Since this is a while True loop, you need to figure out a way to stop the pagination and break the loop. This is something I can only make guesses about since you haven't provided an actual page you are working with.