I have seen posts on using the sleep function before scrolling to the element but I was wondering if its possible to actually show the process of scrolling, instead of hard cutting down to the element. I want the python script to scroll as if were using a mouse to scroll through a page.
So far I have the scroll line:
driver.implicitly_wait(5)
driver.execute_script("window.scrollTo(0, 1000)")
which works, but just hard cuts down the page.
The answer was a lot simple than I thought.
from selenium import webdriver
path = 'D:\coding\chromedriver.exe'
driver = webdriver.Chrome(path)
driver.get('https://selenium-python.readthedocs.io/getting-started.html#simple-usage') #ANY WEBSITE INSIDE BRACKETS
for i in range(1000):
browser.execute_script("window.scrollBy(0, 1)")
This seems to work perfectly, the only downside being that you cannot specify the length of time scrolling, only adjust the pixels.
Related
I am trying to automate a process to download data from a website. The code works if I run it step by step but if I run it all at once it fails. Giving the error
ElementNotInteractableException: Message: element not interactable
I have got around this using time.sleep(x amount of time) but it still seems to fail intermittently. I am having trouble implementing implicit waits. Any help would be appreciated. Code below.
import selenium
browser = webdriver.Chrome(executable_path=r'path\to\chromedriver.exe')
browser.get("https://map.sarig.sa.gov.au/")
browser.maximize_window()
browser.switch_to.frame(browser.find_element_by_id('MapViewer'))
browser.find_element_by_xpath('//*[#id="TourWidget"]/div[1]/span').click()
browser.find_element_by_xpath('//*[#id="menuAllMapLayers"]/div[2]/p').click()
browser.find_element_by_xpath('//*[#id="238"]/li[1]/div/div/span[1]').click()
time.sleep(3)
browser.find_element_by_xpath('//*[#id="238"]/li[1]/div/div/span[1]').click()
browser.find_element_by_xpath('//*[#id="238"]/li[3]/div/div/label/span').click()
browser.find_element_by_xpath('//*[#id="239"]/li[1]/div/div/span[1]').click()
browser.find_element_by_xpath('//*[#id="239"]/li[3]/div/div/label/span').click()
browser.find_element_by_xpath('//*[#id="menuActiveLayers"]').click()
browser.find_element_by_xpath('//*[#id="groupOptions238"]/span').click()
time.sleep(3)
browser.find_element_by_xpath('//*[#id="238"]/li[2]/div/div[3]/div[2]/span').click()
browser.find_element_by_xpath('//*[#id="groupOptions239"]/span').click()
time.sleep(3)
browser.find_element_by_xpath('//*[#id="239"]/li[2]/div/div[3]/div[2]/span').click()
Use ActionChains and get access to pause(3) instead of using sleep(3) but it could also help to use Waits and checking if your elements actually are "visible" rather than "present" (see expected_conditions)
It's a lot of dropdowns so maybe there are not visible all the time, but you can run these checks after doing a move_to_element() so it would actually be present.
I use Selenium with Chrome WebDriver on mobile mode, I got an element and I need to click on it.
When I click, A pop-up is visible, which is good for me, but the click function is still going.
It's stuck and doesn't finish.
Also, when I click the element, there is no new screen on the browser, just an object visible, this is why the click button is not getting any response and doesn't finish.
I want to wrap this click function with a timeout to finish after a couple of seconds.
This must run on a Windows and I can not open a new process/thread for it, just finish the call function and continue with the code.
This is an idea of the code:
from selenium import webdriver
driver= webdriver.Chrome()
driver.get('https://webstie.com')
toggle_to_mobile() # THIS IS PRIVATE METHOD
element = driver.find_element_by_name("my_element")
element.click() # THIS IS STUCK
.
.
Thanks!
I've got this code to bypass captcha basically:
#!/usr/bin/python
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
import time
import sys
try:
driver = webdriver.Chrome()
driver.get(sys.argv[1])
time.sleep(2)
slider = driver.find_element_by_id('nc_2_n1z')
move = ActionChains(driver)
move.click_and_hold(slider).move_by_offset(400, 0).release().perform()
time.sleep(5)
driver.close()
except:
pass
Everything works but when I execute this code, it moves the slider very fast (probably less than 1 second) so I can't bypass the Slide to verify captcha. From start to finish moving the slider, I want it to take 3-5 seconds so it'll act more like a human when moving the slider. Is it possible to adjust the speed when moving the slider ?
You can try this by splitting the below line
move.click_and_hold(slider).move_by_offset(400, 0).release().perform()
You have to click and hold for the desired seconds then release
move.click_and_hold(slider).perform()
sleep(2)
move.move_by_offset(400, 0).release().perform()
However i am not sure the ask can handle captcha as most of the modern captcha can figure out if you are running script
I'm using Selenium to scrape data from a website. The website requires window focus in order to display certain elements that I need.
I want to be able to run my program in the background, without having to focus the window while it's running.
Is there any way to trick the site into thinking it's focused on?
I'm using the selenium chrome driver.
Edit: Here's a quick and dirty test I built.
Check out the code on GitHub
The website background color will turn black when the window.onblur event is recieved, and turn back white when the window.onfocus event is recieved.
I want to fake those events, to make the browser think it's recieved a focus event.
Since the page gets focus information through the onfocus and onblur callbacks in the window object, you can just call them yourself.
browser.execute_script("window.onfocus()")
browser.execute_script("window.onblur()")
For fun, try this script out:
from selenium import webdriver
import time
chromedriver = "./chromedriver"
browser = webdriver.Chrome(executable_path = chromedriver)
browser.get('http://anubiann00b.github.io/FocusTest/')
while True:
browser.execute_script("window.onfocus()")
time.sleep(1)
browser.execute_script("window.onblur()")
time.sleep(1)
And of course, if you want to make the browser think it's always focused, make the onblur method call the onfocus method:
browser.execute_script("window.onblur = function() { window.onfocus() }")
I'm trying to scrape contents from this page on my linux machine. I want to display all the list of wines by clicking the show more button [around 600] until no show more buttons appear. I'm using selenium and PhantomJS for handling javascripts. I'm using time.sleep() show that once i click the show more button, it sleeps for some short time until another appears. The problem i'm facing is, initially the program clicks the show more button quickly but once it reaches around 100-150 clicks, the time taken to detect the show more button increases at an alarming rate, taking too much time. Below is the code that detects the show more button and clicks it.
def parse(self,response):
sel = Selector(self.driver.get(response.url))
self.driver.get(response.url)
click = self.driver.find_elements_by_xpath("//*[#id='btn-more-wines']")
try:
while click:
click[0].click()
time.sleep(2)
except Exception:
print 'no clicks'
An Explicit Wait (instead of time.sleep()) can make a positive impact here:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
click = wait.until(EC.element_to_be_clickable((By.XPATH, "//*[#id='btn-more-wines']")))
This would basically wait for the "Show More" button to become clickable.
Another possible improvement could be achieved by switching to Chrome in a headless mode (with a virtual display):
How do I run Selenium in Xvfb?