On the mentioned website, After searching for the token, a slider captcha appears.
An example of the captcha:
I want to bypass the slider captcha. I took reference from the first solution in Unable to let my script slide a button to the right
My approach would be to slowly move until the slider is in the right place and after that, the new page opens.
The website is: https://www.ems.com.cn/english/
My approach:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import time
from selenium.webdriver import ActionChains
driver = webdriver.Chrome(ChromeDriverManager().install())
actions = ActionChains(driver)
url = 'https://www.ems.com.cn/english/'
driver.get(url)
token = 'CY008445045CN'
token_space = driver.find_element_by_xpath("//input[#class='el-input__inner']")
token_space.send_keys(token)
driver.find_element_by_xpath("//i[#class='el-icon-search']").click()
time.sleep(4)
slider_container = driver.find_element_by_xpath("//div[#class='slide-verify-slider']")
slider = driver.find_element_by_xpath("//div[#class='slide-verify-slider-mask-item']")
# Perform sliding action
actions.move_to_element(slider).click_and_hold().move_by_offset(slider_container.size['width'], 0).release().perform()
find_element_by_xpath
is deprecated, you should use:
find_element(By.XPATH, value="")
Here is updated version of your code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import time
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
actions = ActionChains(driver)
url = 'https://www.ems.com.cn/english/'
driver.get(url)
token = 'CY008445045CN'
token_space = driver.find_element(By.XPATH, value="//input[#class='el-input__inner']")
token_space.send_keys(token)
driver.find_element(By.XPATH, value="//i[#class='el-icon-search']").click()
time.sleep(8)
slider_container = driver.find_element(By.XPATH,value="//div[#class='slide-verify-slider']")
slider = driver.find_element(By.XPATH, value="//div[#class='slide-verify-slider-mask-item']")
# Perform sliding action
for x in range(10000):
actions.move_to_element(slider).click_and_hold().move_by_offset(x, 0).release().perform()
time.sleep(0.1)
To resolve captcha you must release button in correct place, not just slide.
Much better way to get data is to use REST API
REST API
Related
not very familiar with interacting with page source code, I was trying to get the table on a stock screening website, but I just can't seem to get the table in the #document from inspect element to show up in any of my attempts.
the table I'm trying to reach
the inspect element
just the table it seems
Then I tried with selenium:
from time import sleep
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
username = 'yemingsang'
password = MY_PASSWORD
url = 'https://www.esignal.com/members/login'
options = Options()
options.add_experimental_option("detach",True)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()),options=options)
driver.get(url)
sleep(1)
driver.find_element(By.NAME,'user').send_keys(username)
driver.find_element(By.NAME,'password').send_keys(password)
driver.find_element(By.CSS_SELECTOR,'input[type=\"SUBMIT\" i]').click()
sleep(1)
get_source = driver.page_source
print(get_source)
does not have the #document element either using selenium
I'm trying to send keys to the Youtube search box with Selenium, I've tried search ID and Xpath and consistently get errors. These errors include "Message: element not interactable", where am I going wrong?
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("https://youtube.com")
searchbox = driver.find_element("xpath", '//*[#id="search-input"]')
searchbox.send_keys('Jack West')
There are two problems:
First:
Your xpath gives <div> but you can't send keys to <div>
You have to get <input> which is inside <div>
'//*[#id="search-input"]//input'
And later it needs to click button to start searching.
Second:
When I run code browser displays popup window and asks to accept cookies, etc. And this popup blocks access to search field. It needs to add code which will accept it.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from webdriver_manager.chrome import ChromeDriverManager
#from webdriver_manager.firefox import GeckoDriverManager
import time
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
#driver = webdriver.Firefox(service=Service(GeckoDriverManager().install()))
driver.get("https://youtube.com")
time.sleep(5)
print('cookies')
cookies = driver.find_elements(By.XPATH, '//div[contains(#class,"eom-button-row")]//a')
cookies[1].click()
time.sleep(5)
print('searchbox')
searchbox = driver.find_element(By.XPATH, '//*[#id="search-input"]//input')
searchbox.send_keys('Jack West')
print('searchbox button')
searchbox_button = driver.find_element(By.XPATH, '//button[#id="search-icon-legacy"]')
searchbox_button.click()
But it could be simpler to use url
https://www.youtube.com/results?search_query=Jack+West
and it doesn't need to click any keys or buttons.
YouTube has API for developers/programmers and this should be preferred method.
I am currently working on a scraper for aniworld.to.
My goal is it to enter the anime name and get all of the Episodes downloaded.
I have everything working except one thing...
The websites has a Watch button. That Button redirects you to https://aniworld.to/redirect/SOMETHING and that Site has a captcha which means the link is not in the html...
Is there a way to bypass this/get the link in python? Or a way to display the captcha so I can solve it?
Because the captcha only appears every lightyear.
The only thing I need from that page is the redirect link. It looks like this:
https://vidoza.net/embed-something.html
My very very wip code is here if it helps: https://github.com/wolfswolke/aniworld_scraper
Mitchdu showed me how to do it.
If anyone else needs help here is my code: https://github.com/wolfswolke/aniworld_scraper/blob/main/src/logic/captcha.py
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.support.ui import WebDriverWait
from threading import Thread
import os
def open_captcha_window(full_url):
working_dir = os.getcwd()
path_to_ublock = r'{}\extensions\ublock'.format(working_dir)
options = webdriver.ChromeOptions()
options.add_argument("app=" + full_url)
options.add_argument("window-size=423,705")
options.add_experimental_option('excludeSwitches', ['enable-logging'])
if os.path.exists(path_to_ublock):
options.add_argument('load-extension=' + path_to_ublock)
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get(full_url)
wait = WebDriverWait(driver, 100, 0.3)
wait.until(lambda redirect: redirect.current_url != full_url)
new_page = driver.current_url
Thread(target=threaded_driver_close, args=(driver,)).start()
return new_page
def threaded_driver_close(driver):
driver.close()
I'm trying to make a script that logs into my online grade book to look for any changes (new grades, etc). This is my code so far.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
import time
def main():
options = Options()
# options.add_argument("--headless")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.maximize_window()
# goes to the desired website
driver.get('https://portal.librus.pl/rodzina')
# searches for and clicks a button that drops down a menu in which link for login form is visible
button = driver.find_element(By.CLASS_NAME, 'btn.btn-third.btn-synergia-top.btn-navbar.dropdown-toggle')
button.click()
# searches and clicks login link
agree = driver.find_element(By.CLASS_NAME, 'zmdi.zmdi-account.dropdown-item__icon')
agree.click()
time.sleep(10)
driver.quit()
if __name__ == '__main__':
main()
And there is a problem, I cannot seem to find a way to make webdriver see what I see. What I mean is that I see the webpage like this and webdriver sees the same webpage like this also the source code is different. I've tried using undetected ChromeDriver with no success. This is my code using UC.
import undetected_chromedriver as uc
import time
from selenium.webdriver.common.by import By
def main():
driver = uc.Chrome()
driver.maximize_window()
# goes to the desired website
driver.get('https://portal.librus.pl/rodzina/home')
# searches for and clicks a button that drops down a menu in which link for login form is visible
button = driver.find_element(By.CLASS_NAME, 'btn.btn-third.btn-synergia-top.btn-navbar.dropdown-toggle')
button.click()
# searches and clicks login link
agree = driver.find_element(By.CLASS_NAME, 'zmdi.zmdi-account.dropdown-item__icon')
agree.click()
time.sleep(5)
driver.execute_script("window.print();")
if __name__ == '__main__':
main()
Has anyone had a similar problem and managed to solve it?
Code:
from lib2to3.pgen2 import driver
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
s=Service("/usr/local/bin/chromedriver")
driver = webdriver.Chrome(service=s)
driver.implicitly_wait(0.5)
driver.maximize_window().
driver.get("https://greenhillsschool.myschoolapp.com/app#login")
driver.find_element(By.XPATH, "//*.
[#id="Username"]").send_keys("rpatel#greenhillsschool.org")
'''
usernamebox.send_keys("email")
next = driver.find_element_by_xpath("Next")
next.click()
'''