First off - no experience with Python, Sublime Text or Selenium - please be kind..
I am having an issue building a scraper in Python - using Sublime text, Selenium and Chrome. I have updated to the latest Python, Pip and downloaded the correct Chrome Driver. The webpage pops up fine, but get errors.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
#from selenium.webdriver.common.ui import WebDriverWait - commented out due to error
#from selenium.webdriver.common import expected_conditions as EC - commented out due to error
import time
driver = webdriver.Chrome("C:\Program Files (x86)\chromedriver.exe")
driver.get ("https://www.royalcaribbean.com/account/cruise-planner/category/pt_beverage/product/3222?bookingId=1429192&shipCode=NV&sailDate=20220907")
print(driver.find_element(by=By.CLASS_NAME, value='ng-tns-c210-4 text-promo-1').text)***
wait = WebDriverWait(driver, 20)
driver.get("https://www.royalcaribbean.com/account/cruise-planner/category/pt_beverage/product/3222?bookingId=1429192&shipCode=NV&sailDate=20220907")
elem=wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, '.ng-tns-c210-4.text-promo-1')))
print(elem.text)
Your class name is actually multiple class names so using css selector.
Outputs:
$80.99
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Related
I am new to python and selenium. Trying to automate download of files from the dropdown menu on the below link for the first time. The code gets stuck after page loads i.e the drop down does not work and gives me an error. Sorry if my code looks clunky. Any help is appreciated.
Thanks
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.select import Select
import time
s=Service("C:\\python\\chromedriver.exe")
driver = webdriver.Chrome(service=s)
driver.maximize_window()
driver.get('http://www.amfiindia.com/research-information/aum-data/classified-average-aum')
#monthname=driver.find_element(By.XPATH,"//*[#id='AaumDate']")
monthdd=Select(monthname)
monthdd.select_by_value('01-Jan-22')
time.sleep(3)
typename=driver.find_element(By.XPATH,"//*[#id='AumType']")
typedd=Select(typename)
typedd.select_by_visible_text('Scheme category wise')
time.sleep(3)
mfname=driver.find_element(By.XPATH,"//*[#id='AumMFName']")
mfdd=Select(mfname)
mfdd.select_by_visible_text('All')
time.sleep(3)
driver.find_element("class name","sprite-inter go-btn").click()
wait=WebDriverWait(driver,20)
driver.get('http://www.amfiindia.com/research-information/aum-data/classified-average-aum')
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"#divAumPeriod > span > a > span.ui-button-text"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH,"//*[#class='ui-menu-item']/a[.='January - 2022']"))).click()
Most of these tags open like so and have another list with them that's not using the select tag.
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I am trying to get some data from this website link
This is my code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Edge("C:/Users/Hamita/Documents/msedgedriver.exe")
driver.get("https://www.bountou1x2.com/sport.jsp?ispt=null")
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//*[#id=\"iframe-alto-sinistra\"]")))
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH,"//*[#id=\"left-menu-container\"]/div[1]/div[1]"))).click()
But it's giving me this error:
selenium.common.exceptions.TimeoutException
The button i want to click is to select Today, it's located in the left side
Help is appreciated, thanks to all of you
The <iframe> you are trying to switch to is located by the following ID receiver
So you should use:
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"receiver")))
instead of
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"//*[#id=\"iframe-alto-sinistra\"]")))
I am using Python selenium chromedriver and I want to add Capmonster to my code. Basically if my Code gets the URL, a ReCaptcha appears and I want to get it solved.
My Code
from threading import Thread
from selenium.webdriver.chrome.options import Options
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
from selenium.webdriver.support.ui import Select
from selenium.common.exceptions import NoSuchElementException
from selenium.common.exceptions import TimeoutException
import os
chromedriver = 'C:\\Users\\yvesb\\OneDrive\\Desktop\\chromedriver\\chromedriver.exe'
options = webdriver.ChromeOptions()
options.add_argument('headless')
options.add_argument('window-size=1200x600') # optional
driver = webdriver.Chrome(executable_path=chromedriver, chrome_options=options)
def test(driver, url):
driver.get(url)
try:
driver.save_screenshot('C:\\Users\\yvesb\\Downloads\\headless_chrome_test7.png')
finally:
print("screenshot done")
url = "https://www.snipes.com/login"
Thread(target=test, args=(driver, url)).start()
And when it gets the URL a ReCaptcha appears, which I want to get solved with my Capmonster Key.
I suggest you to try this library I've developed some time ago. If you have a set of labelled captchas that service would fit you. Take a look: https://github.com/punkerpunker/captcha_solver
In README there is a section "Train model on external data" that you might be interested in.
I am using Selenium, inside a Django Project. I am using it in views.py, inside to my app, and I link it by url.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
def allyears(br):
global browser
path = os.environ['PATH']
browser = webdriver.Chrome(executable_path="/"+path+"/chromedriver)
browser.get('https://www.address.com/recipes/))
I am trying to install Webdriver Chrome in a Django project. I linked to path environment, but not working. Anyway, when I will update it to a server, I can not link to a physical route.
So, how can import webdriver.Chrome or webdriver.Firefox in Django?
I am trying to web scrape this website using PYTHON selenium.
www.shareinvestor.com/fundamental/factsheet.html?counter=Z74.SI
But the selenium seem to hang and not loading.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium import webdriver as web
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r'C:\Program Files (x86)\Mozilla Firefox\firefox.exe')
driver = webdriver.Firefox(firefox_binary=binary)
import time
driver.get("http://www.shareinvestor.com/fundamental/factsheet.html?counter=Z74.SI")
WebDriverWait(driver, 20)
time.sleep(5)
print(driver.page_source)
I am interested in the table as shown below.
What is wrong with my code? I am not able to get any output.