I'm following a scraping tutorial from GeeksForGeeks below:
https://colab.research.google.com/drive/14kXYQCSGVye4bBKKExtuHOeq32bc4xbW?usp=sharing#scrollTo=UjB8J1L8s6kT
I'm working on a Macbook Pro in Google Colab via Chrome.
When I get to the 4th command block where the path and driver are defined I get an error saying
<ipython-input-33-bb59bfe38590>:2: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(PATH)
And
During handling of the above exception, another exception occurred:
WebDriverException Traceback (most recent call last)
/usr/local/lib/python3.8/dist-packages/selenium/webdriver/common/service.py in _start_process(self, path)
211 except OSError as err:
212 if err.errno == errno.ENOENT:
--> 213 raise WebDriverException(
214 f"'{os.path.basename(self.path)}' executable needs to be in PATH. {self.start_error_message}"
215 )
WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home
What does this mean? Do I need to link to webdriver on my local mac drive? Currently the command in the tutorial is:
PATH = "/Users/Edu/Desktop/VENV/chromedriver"
driver = webdriver.Chrome(PATH)
driver.get(url)
Do I need to replace this with the path to chromedriver in my files? I tried replacing the path above with the below but it doesn't change anything:
/Users/KD/Downloads/chromedriver
Tried the command below from the tutorial linked here: https://colab.research.google.com/drive/14kXYQCSGVye4bBKKExtuHOeq32bc4xbW?usp=sharing#scrollTo=UjB8J1L8s6kT
PATH = "/Users/Edu/Desktop/VENV/chromedriver"
driver = webdriver.Chrome(PATH)
driver.get(url)
You need to use Service.
The following works:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = 'your_link'
driver.get(url)
Related
I got this error when running my code. BTW, I am running this behind a VPN.
Can anyone tell me how can I fix it?
BTW, I try --headless, but weird that only the default profile is started with headless.
selenium.common.exceptions.WebDriverException: Message: unknown error:
Chrome failed to start: exited normally. (unknown error:
DevToolsActivePort file doesn't exist) (The process started from
chrome location C:\Program Files\Google\Chrome\Application\chrome.exe
is no longer running, so ChromeDriver is assuming that Chrome has
crashed.)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
import os
os.environ['WDM_SSL_VERIFY'] = '0'
os.environ["HTTPS_PROXY"] = "http://127.0.0.1:4780"
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
# options.add_argument('--headless')
options.add_argument(r"--user-data-dir=C:\Users\nuc\AppData\Local\Google\Chrome\User Data") #e.g. C:\Users\You\AppData\Local\Google\Chrome\User Data
options.add_argument(r'--profile-directory=Profile 5') #e.g. Profile 3
# options.binary_location
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.switch_to.new_window('tab')
driver.get("https://www.youtube.com/")
driver.save_screenshot('ss.png')
I want to save element data to an excel file via python. I have the code below, I need some help why the line where
element.click()
gives an error. Even though I put the click() method upper line, but i need it to be in line below.
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(r"C:\Users\Admin\Downloads\chromedriver_win32 (1)\chromedriver.exe")
driver.get("https://www.nba.com/schedule?pd=false®ion=1")
driver.implicitly_wait(30)
element_to_click=driver.find_element(By.ID,"onetrust-accept-btn-handler").click()
element_to_click.click() 'error
element_to_save=driver.find_element(By.XPATH,"//div/div/div/div/h4")
#element_to_save.to_excel("3row,3column)")
driver.quit()
This is one way to reject/accept cookies on that website:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
chrome_options = Options()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument('disable-notifications')
chrome_options.add_argument("window-size=1280,720")
webdriver_service = Service("chromedriver/chromedriver") ## path to where you saved chromedriver binary
browser = webdriver.Chrome(service=webdriver_service, options=chrome_options)
wait = WebDriverWait(browser, 20)
url = 'https://www.nba.com/schedule?pd=false®ion=1'
browser.get(url)
try:
wait.until(EC.element_to_be_clickable((By.ID, "onetrust-accept-btn-handler"))).click()
print('accepted cookies')
except Exception as e:
print('no cookie button!')
Setup is selenium/chrome on linux - just observe the imports and the part after defining the browser/driver.
Selenium documentation can be found at https://www.selenium.dev/documentation/
So when I am execute my code the website will be open but the other steps which I described in my code below are not be execute, why? I even tried time.sleep() after the website is loaded in order to execute the remaining code but it is not working.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
driver = webdriver.Chrome('/Users/User/Desktop/it_projects/python-google-automation/Neuer Ordner/chromedriver')
#open certain website
driver.get('https://www.nike.com/de/?cp=58194921917_search_%7cnike%7c10594878138%7c107792850434%7ce%7cc%7cDE%7cpure%7c452291007809&ds_rl=1252249&gclid=EAIaIQobChMIod-_o8jD-QIVE4XVCh1-FggsEAAYASAAEgJSYfD_BwE&gclsrc=aw.ds')
#go through cookie process
evade_cookie = driver.find_element_by_id('hf_cookie_text_moreInformation') #search for cookie-button (more informations)
evade_cookie.send_keys(Keys.RETURN) #click on button
select_cookie = driver.find_element_by_id('hf_cookie_label_done') #search for done-button
select_cookie.send_keys(Keys.RETURN) #click on done-button
#search for Sneakers
search = driver.find_element_by_id('VisualSearchInput') #search for input-area
search.send_keys('Nike Dunk Low') #insert input
search.send_keys(Keys.RETURN) #enter or return respectively
Error
/Users/user/Desktop/it_projects/python-google-automation/Neuer Ordner/main.py:6: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome('/Users/user/Desktop/it_projects/python-google-automation/Neuer Ordner/chromedriver')
Traceback (most recent call last):
File "/Users/user/Desktop/it_projects/python-google-automation/Neuer Ordner/main.py", line 14, in <module>
evade_cookie = driver.find_element_by_id('hf_cookie_text_moreInformation') #search for cookie-button (more informations)
AttributeError: 'WebDriver' object has no attribute 'find_element_by_id'
user#MacBook-Air-von-Sami python-google-automation %
It seems that the syntax you are using is incorrect.
Try the below code
#go through cookie process
evade_cookie = driver.find_element('id','hf_cookie_text_moreInformation') #search for cookie-button (more informations)
evade_cookie.send_keys(Keys.RETURN) #click on button
select_cookie = driver.find_element('id','hf_cookie_label_done') #search for done-button
select_cookie.send_keys(Keys.RETURN) #click on done-button
The following code will properly wait for the elements to be clickable, and execute correctly. The setup is Firefox/geckodriver on Linux, but you can adapt it to your own, just observe the imports, and the part after defining the browser/driver:
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.firefox.options import Options as Firefox_Options
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support import expected_conditions as EC
import time as t
firefox_options = Firefox_Options()
driverService = Service('chromedriver/geckodriver')
browser = webdriver.Firefox(service=driverService, options=firefox_options)
url = 'https://www.nike.com/de/?cp=58194921917_search_%7cnike%7c10594878138%7c107792850434%7ce%7cc%7cDE%7cpure%7c452291007809&ds_rl=1252249&gclid=EAIaIQobChMIod-_o8jD-QIVE4XVCh1-FggsEAAYASAAEgJSYfD_BwE&gclsrc=aw.ds'
browser.get(url)
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,'//span[text()="Mehr Informationen"]'))).click()
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH,'//span[text()="Fertig"]'))).click()
search_box = WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.ID,'VisualSearchInput')))
search_box.send_keys('Nike Dunk Low')
t.sleep(1)
search_box.send_keys(Keys.RETURN)
Selenium documentation: https://www.selenium.dev/documentation/
As per the error stacktrace there are two errors:
DeprecationWarning: executable_path has been deprecated, please pass in a Service object: Implies that the default argument executable_path is deprecated now and you have to use an instance of the Service() class. See: DeprecationWarning: executable_path has been deprecated selenium python
AttributeError: 'WebDriver' object has no attribute 'find_element_by_id': Implies that the find_element_by_* commands are deprecated in the latest Selenium Python libraries. Instead you have to use find_element(). See: find_element_by_* commands are deprecated in selenium
Your effective code block will be:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
s = Service('/Users/User/Desktop/it_projects/python-google-automation/Neuer Ordner/chromedriver')
driver = = webdriver.Chrome(service=s)
driver.get('https://www.nike.com/de/?cp=58194921917_search_%7cnike%7c10594878138%7c107792850434%7ce%7cc%7cDE%7cpure%7c452291007809&ds_rl=1252249&gclid=EAIaIQobChMIod-_o8jD-QIVE4XVCh1-FggsEAAYASAAEgJSYfD_BwE&gclsrc=aw.ds')
evade_cookie = driver.find_element(By.ID, "hf_cookie_text_moreInformation")
I was doing an availability checker for some Amazon products, it was working but after I went for sometime I got back and I found this error. I don't know if I've missed something or editing something.
I hope the following information helps:
This is the error:
Exception has occurred: AttributeError 'WebDriver' object has no attribute 'link'
and this is my code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import webbrowser
import time
import os
urls = [
"URL 1",
"URL 2"
]
PATH = "C:\Program Files (x86)\chromedriver.exe"
options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')
driver = webdriver.Chrome(PATH, chrome_options=options)
def main():
# stuff
while True:
for i in range(0, len(urls)):
url = urls[i]
time.sleep(2)
print(url)
wait = WebDriverWait(driver, 10)
driver.link(url)
wait.until(EC.element_to_be_clickable((By.XPATH, '//*[#id="availability"]/span')))
print(driver.title)
availablility = driver.find_element_by_xpath('//*[#id="availability"]/span')
print(availablility.text)
main()
I am using Python 3.7.9 64-bit in Visual Studio Code.
There is no webdriver.link. You probably meant to use webdriver.get:
So just change driver.link(url)
into driver.get(url)
I am trying to install chrome extension using Python Selenium.
When I click Add to chrome button a pop up(don't know whether it is a java scripted) is generated asking: "Add extension", "Cancel". I want to click "Add extension" but I am getting following error:
selenium.common.exceptions.NoAlertPresentException: Message: no alert open
My code:
from selenium import webdriver
import time
driver=webdriver.Chrome()
driver.implicitly_wait(30)
driver.get("https://chrome.google.com/webstore/detail/buyhatke/jaehkpjddfdgiiefcnhahapilbejohhj?hl=en")
time.sleep(15)
element=driver.find_element_by_css_selector("body > div.F-ia-k.S-ph.S-Rc-qa > div.h-F-f-k.F-f-k > div > div > div.e-f-o > div.h-e-f-Ra-c.e-f-oh-Md-zb-k >
div.dd-Va.g-c-wb.g-eg-ua-Uc-c-za.g-c-Oc-td-jb-oa.g-c")
element.click()
alert = driver.switch_to.alert
alert.accept()
help me to install it.
Updated code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import os
executable_path = "C:\\Users\\SACHIN\\AppData\\Local\\Programs\\Python\\Python36\\chromedriver"
os.environ["webdriver.chrome.driver"] = executable_path
chrome_options = Options()
chrome_options.add_extension("C:\\Users\\SACHIN\\AppData\\Local\\Google\\chrome\\User Data\\Default\\Extensions\\jaehkpjddfdgiiefcnhahapilbejohhj\\
3.4.143_0")
driver = webdriver.Chrome(executable_path=executable_path,chrome_options=chrome_options)
driver.get("http://stackoverflow.com")
driver.quit()
This is because the download option pop up that you are trying to select using switch_to.alert is not actually a JS alert pop up. You can not select is using Selenium's switch_to methods. You need to use the DesiredCapabilities class to select this option and then use it in your code.
As per the ChromeDriver documentation, given here, please use the packed( the one with .crx extension ) or unpacked extension file (directory containing the extension, including a manifest.json file) and load it using DesiredCapabilities. This can be done using
from selenium.webdriver.chrome.options import Options
executable_path = "path_to_webdriver"
os.environ["webdriver.chrome.driver"] = executable_path
chrome_options = Options()
chrome_options.add_extension('path_to_extension')
driver = webdriver.Chrome(executable_path=executable_path,chrome_options=chrome_options)
driver.get("http://stackoverflow.com")
driver.quit()