How to initiate Chrome Canary in headless mode through Selenium and Python - python

from selenium import webdriver
options = webdriver.ChromeOptions()
options.binary_location = 'C:\Users\mpmccurdy\Desktop\Google Chrome Canary.lnk'
options.add_argument('headless')
options.add_argument('window-size=1200x600')
driver = webdriver.Chrome(chrome_options=options)
driver.get("https://www.python.org")

If you are using Chrome Canary as a basic Requirement the server still expects you to have Chrome installed in the default location as per the underlying OS architecture as follows:
You can also override the default Chrome Binary Location following the documentation Using a Chrome executable in a non-standard location as follows:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.binary_location = r'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
options.add_argument('--headless')
options.add_argument('window-size=1200x600')
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', chrome_options=options)
driver.get("https://www.python.org")

chrome_options = Options()
chrome_options.add_argument("--headless")
path = os.getcwd() +'\\chromedriver.exe' #needs to be in your current working directory
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path=path)

Related

Website not opening with chrome driver but opening with geckodriver

I am trying to scrape the website. First of all, it is not working with Beautifulsoup but when I am trying to open it with selenium chrome driver it's not opening. It's opening with firefox but it's very slow and gives an error on element click. Here is my code:
from selenium import webdriver
opt = webdriver.ChromeOptions()
opt.add_argument("--disable-xss-auditor")
opt.add_argument("--disable-web-security")
opt.add_argument("--allow-running-insecure-content")
opt.add_argument("--no-sandbox")
opt.add_argument("--disable-setuid-sandbox")
opt.add_argument("--disable-webgl")
opt.add_argument("--disable-popup-blocking")
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.get(f"http://app1.nmpa.gov.cn/data_nmpa/face3/base.jsp?tableId=25&tableName=TABLE25&title=%B9%FA%B2%FA%D2%A9%C6%B7&bcId=152904713761213296322795806604&CbSlDlH0=qGrYrAktn7.tn7.tnznJalIvVetjcXpaapSdKuqmmoVqqWL")
Possibly Selenium driven ChromeDriver initiated google-chrome Browsing Context is geting detected as bot and the arguments you have added can't bypass the bot detection mechanism effectively.
Solution
You can evade the detection by adding a few arguments and experimental_option as follows:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_argument("start-maximized")
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get("http://app1.nmpa.gov.cn/data_nmpa/face3/base.jsp?tableId=25&tableName=TABLE25&title=%B9%FA%B2%FA%D2%A9%C6%B7&bcId=152904713761213296322795806604&CbSlDlH0=qGrYrAktn7.tn7.tnznJalIvVetjcXpaapSdKuqmmoVqqWL")

Adding Extension to Chromedriver in Heroku Python

I have successfully set app chromedriver with the following set up on Heroku.
I would like to add some extensions (myextension.crx) through the webdriver options.
Can you help? (what path to use and how to steps please)
I.e
options = webdriver.ChromeOptions()
options.add_extension('./myextension.crx')
#Heroku Chromedriver Set Up
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
CHROMEDRIVER_PATH = "/app/.chromedriver/bin/chromedriver"
chrome_bin =os.environ.get('GOOGLE_CHROME_SHIM', None)
options = webdriver.ChromeOptions()
options.binary_location = chrome_bin
options.add_argument(" — disable-gpu")
options.add_argument(" — no-sandbox")
options.add_argument(" — headless")
options.add_argument('--disable-dev-shm-usage')
options.add_argument('--remote-debugging-port=9222')
options.add_argument('--disable-infobars')
driver = webdriver.Chrome(executable_path="chromedriver", chrome_options=options)
Also used the official buildpacks
https://github.com/heroku/heroku-buildpack-google-chrome
https://github.com/heroku/heroku-buildpack-chromedriver
And Variables
CHROMEDRIVER_PATH = /app/.chromedriver/bin/chromedriver
GOOGLE_CHROME_BIN = /app/.apt/usr/bin/google-chrome
I don't think you can run extensions on Chrome Driver in headless mode. Switching to Firefox with geckodriver is one potential solution though

How to correctly specify webdriver path with add_argument()?

This is my code:
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--no-sandbox")
options.add_argument("C:/webdrivers/chromedriver.exe")
driver = webdriver.Chrome(options=options)
driver.get("https://www.google.com")
But it did not use the webdriver that I am trying to specify and uses some different one. How to correctly specify path to webdriver in the code above?
So main point here is that I want to specify path to webdriver and also use it without sandbox. How can I do it?
This worked:
from selenium import webdriver
# start the browser
options = webdriver.ChromeOptions()
# options.add_argument("--headless")
options.add_argument("--no-sandbox")
# options.add_argument("--disable-dev-shm-usage")
# options.add_argument("--disable-gpu")
# options.add_argument("--window-size=1920,1080")
driver = webdriver.Chrome(executable_path=r"C:/webdrivers/chromedriver.exe", options=options)
driver.get("https://www.google.com")

Python Selenium Firefox - how to enable headless-mode as part of a class/object?

I have the following code:
options = Options()
options = options.set_headless( headless=True)
class Sel_Driver():
def __init__(self):
self.driver = webdriver.Firefox(firefox_options=options)
I can then use self.driver.get(url) as part of a method to open urls I feed in. This works - I can feed in and open the URLs, but they don't in headless mode.
(I initially defined the driver as self.driver = webdriver.Firefox(firefox_options=Options().set_headless(headless=True) - but that didn't work, so I tried it as above).
What am I missing? I don't understand why the driver is able to open pages, but the options aren't enabled.
Please try following code :
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(firefox_options=options)
This will work for you for sure. Try it.Please specify the path of the driver. It is for chrome change it to firefox.
from pyvirtualdisplay import Display
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--headless")
driver = webdriver.Chrome(chrome_options=options, executable_path="C:\\Users\\Username\\Downloads\\chromedriver.exe")
print("Firefox Headless Browser Invoked")
driver.get('https://www.facebook.com/')
jks = driver.find_element_by_id("email").get_attribute("class")
print(jks)
driver.quit()

Disable python chrome driver extensions without lose driver path

I started having issues running a python script which uses selenium and chrome driver, so I want to disable extensions of chrome driver using python without losing the path where the driver is located.
Currently I have the below:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
path_to_Ie = 'C:\\Python34\\ChromeDriver\\ChromeDriver.exe'
browser = webdriver.Chrome(executable_path = path_to_Ie)
url = 'https://wwww.test.com/'
browser.get(url)
and I would like to add the below lines:
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
browser = webdriver.Chrome(chrome_options=chrome_options)
Any idea how I can merge both codes to make it work properly?
Thanks a lot.
Just provide multiple keyword arguments:
chrome_options = Options()
chrome_options.add_argument("--disable-extensions")
browser = webdriver.Chrome(executable_path=path_to_Ie, chrome_options=chrome_options)

Categories

Resources