After an exhaustive amount of Googling, I'm at a loss as to how to connect to a CefPython (Chrome Embedded Framework) browser instance using Selenium.
I see two possible ways of going about this:
Use Selenium to launch a CefPython instance directly, or
Launch a CefPython instance independently, then connect to it with Selenium.
I've looked for similar questions but they either have non-working code (older versions?) or seem to be attempting to do other things, and I can't find any with actual working code snippets as answers. So as a starting point, here is working code for launching Chrome with Selenium, but using a standard non-CEF Chrome instance:
Option 1 (working; launch standard Chrome.exe with Selenium)
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--disable-gpu")
chromedriver_path = r"C:\Users\..\webdrivers\chromedriver_2_40\chromedriver_win32\chromedriver.exe"
chrome_path = r"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"
options.binary_location = chrome_path;
driver = webdriver.Chrome(executable_path=chromedriver_path, options=options)
time.sleep(1)
driver.get("https://www.google.com") # SUCCESS!
time.sleep(4)
driver.quit()
Option 2 (working; launch Chrome.exe then connect to it with Selenium)
In this example, "driver2" is the one connecting remotely to the already-running instance created by "driver."
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--disable-gpu")
chromedriver_path = r"C:\Users\..\webdrivers\chromedriver_2_40\chromedriver_win32\chromedriver.exe"
chrome_path = r"C:\Program Files (x86)\Google\Chrome\Application\chrome.exe"
options.binary_location = chrome_path;
driver = webdriver.Chrome(executable_path=chromedriver_path, options=options)
executor_url = driver.command_executor._url
session_id = driver.session_id
print(executor_url, session_id)
driver2 = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver2.close() # close the second session created by driver 2 (cannot pass a session_id to webdriver.Remote())
driver2.session_id = session_id # use the driver1 session instead
time.sleep(1)
driver2.get("https://www.google.com") # SUCCESS!
time.sleep(4)
driver2.quit()
But when I try to make this work with CefPython, I am at a loss for how to do it.
Option 1 (non-working; CefPython instance)
Attempting Option 1 with CefPython just hangs for a while before raising an exception. The only executable in the CefPython package I see that Selenium could possibly use to launch is the subprocess.exe file, but clearly this is NOT just a drop-in replacement for chrome.exe.
This code is identical to the "Option 1" code above, except it swaps the chrome_path for the subprocess.exe binary.
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("--disable-gpu")
chromedriver_path = r"C:\Users\..\webdrivers\chromedriver_2_40\chromedriver_win32\chromedriver.exe"
chrome_path = r"C:\Users\..\project-folder\pybin\Lib\site-packages\cefpython3\subprocess.exe"
options.binary_location = chrome_path;
driver = webdriver.Chrome(executable_path=chromedriver_path, options=options)
print('driver created...') # is never reached :( apparently hangs over socket waiting...
# after a while...
# selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist
time.sleep(1)
driver.get("https://www.google.com")
time.sleep(4)
driver.quit()
Option 2 (non-working; CefPython instance)
Here I try to launch CEFPython independently, then connecting to it with Selenium. Attempting this leaves me with needing an executor_url and a session Id, however I cannot for the life of me figure out how to get these from a running CefPython instance:
from cefpython3 import cefpython as cef
from selenium import webdriver
settings = {"windowless_rendering_enabled": False}
switches = {"remote-debugging-port": "22222",
'user-data-dir':r"C:\Users\..\..\mydatadir"}
cef.Initialize(settings, switches)
executor_url = None # how to get this?
session_id = None # how to get this?
driver2 = webdriver.Remote(command_executor=executor_url, desired_capabilities={})
driver2.close() # close the driver 1 session (cannot pass a session_id to webdriver.Remote())
driver2.session_id = session_id
time.sleep(30)
driver2.get("https://www.google.com")
time.sleep(4)
driver2.quit()
I'm using the 2.40 version of ChromeDriver, because the latest version of CefPython uses Chrome version 66, which in turn requires version 2.40 of the chromedriver.
Any assistance is appreciated.
Related
#TLDR I want to use brave browser with selenium written in python but can't find any current solutions that work.
This code works
from selenium import webdriver
option = webdriver.ChromeOptions()
option.binary_location = r'C:\Program Files\BraveSoftware\Brave-
Browser\Application\brave.exe'
driver = webdriver.Chrome(executable_path=r'C:\WebDrivers\chromedriver.exe',
options=option)
driver.get("https://www.google.com")
driver.quit()
but executable_path is deprecated:
C:\Users\USER\PycharmProjects\pythonProject\sol2.py:5:
DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(executable_path=r'C:\WebDrivers\chromedriver.exe', options=option)
Found this on youtube: https://www.youtube.com/watch?v=VMzmVFA-Gps
# import statements
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
# Declare variables and setup services
driverService = Service('C:/webdrivers/chromedriver.exe')
# 1. Passes the chromedriver path to the service object
# 2. stores the service object in the s variable
driver = webdriver.Chrome(service=driverService)
# 1. Passes service object driverSerice into the webdriver.Chrome
# 2. Stores object in driver variable
# Body (actually doing stuff)
driver.maximize_window() # maximizes the browser window
driver.get("https://www.google.com") # navigates to google.com
myPageTitle = driver.title
# gets the title of the web page stores in myPageTitle
print(myPageTitle) # prints myPageTitle to Console
assert "Google" in myPageTitle
# checks myPageTitle to ensure it contains Google
# clean up
driver.quit() # closes the browser
When I run this code I get:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot find Chrome binary
This code works as long as you allow Google Chrome onto your PC. I don't want Chrome on my PC.
The problem is that I can't figure out how to get selenium to use brave instead of Chrome.
As of this writing I am using the following:
Windows 11 Home
Selenium v4.0.0
Python v3.10
ChromeDriver 95.0.4638.69
Brave Browser Version 1.31.91 Chromium: 95.0.4638.69 (Official Build) (64-bit)
Can some one please explain how to make this work with the current (read nondeprecated) code on brave browser? Thanks for your time.
To initiate a brave browsing context you need to:
Use the binary_location attribute to point to the brave binary location.
Use the chromedriver executable to initiate the brave browser.
Code block:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
option = webdriver.ChromeOptions()
option.binary_location = r'C:\Program Files (x86)\BraveSoftware\Brave-Browser\Application\brave.exe'
driverService = Service('C:/Users/.../chromedriver.exe')
driver = webdriver.Chrome(service=driverService, options=option)
driver.get("https://www.google.com")
Note: The DeprecationWarning: executable_path has been deprecated is a harmless warning message which doesn't affects your test execution and you can still ignore it.
References
You can find a couple of relevant detailed discussions in:
How to use Brave web browser with python, selenium and chromedriver?
How to initiate Brave browser using Selenium and Python on Windows
you have to set your path to brave binary.
options.setBinary("Path to brave.exe");
Go through this website:
https://mundrisoft.com/tech-bytes/how-to-execute-selenium-script-in-brave-browser/
So basically i am working on a python script that loggs into a twitch account and stays there to generate a viewer.
But my main issue is how do i make this work for multiple accounts.
How to hide alle the Windows, and how can i handle multiple selenium windows ?
Is selenium even good for that or is there a other way ?
from selenium import webdriver
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--mute-audio")
driver = webdriver.Chrome("D:\Downloads\chromedriver_win32\chromedriver.exe", chrome_options=chrome_options)
driver.minimize_window()
driver.get('https://www.twitch.tv/login')
search_form = driver.find_element_by_id('login-username')
search_form.send_keys('user')
search_form = driver.find_element_by_id('password-input')
search_form.send_keys('password')
search_form.submit()
driver.implicitly_wait(10)
driver.get('https://www.twitch.tv/channel')
You are definitely able to use Selenium and Python to do this. To run multiple accounts, you will have to utilize multi-threading or create multiple driver objects to manage.
Multithreading example from this thread:
from selenium import webdriver
import threading
import time
def test_logic():
driver = webdriver.Firefox()
url = 'https://www.google.co.in'
driver.get(url)
# Implement your test logic
time.sleep(2)
driver.quit()
N = 5 # Number of browsers to spawn
thread_list = list()
# Start test
for i in range(N):
t = threading.Thread(name='Test {}'.format(i), target=test_logic)
t.start()
time.sleep(1)
print t.name + ' started!'
thread_list.append(t)
# Wait for all thre<ads to complete
for thread in thread_list:
thread.join()
print 'Test completed!'
Each driver will have to use a proxy connection to connect to Twitch on separate IP addresses. I suggest using Opera as it has a built in VPN, makes it a lot easier.
Example of Opera and Selenium from this thread:
from selenium import webdriver
from time import sleep
# The profile directory which Opera VPN was enabled manually using the GUI
opera_profile = '/home/user-directory/.config/opera'
options = webdriver.ChromeOptions()
options.add_argument('user-data-dir=' + opera_profile)
driver = webdriver.Opera(options=options)
driver.get('https://whatismyipaddress.com')
sleep(10)
driver.quit()
To hide the console for webdrivers you must run them with the "headless" option.
Headless for chrome driver.
from selenium import webdriver from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument("--headless")
Unfortunately headless is not supported in Opera driver, so you must use Chrome or Firefox for this.
Good luck!
hi you will not be able to create a bot with selenium because even if you manage to connect several accounts on the twitch account, twitch (like youtube) have a system that looks at your IP address and does not increase the number of views if the multiple connection come from the same computer.
def __init__(self):
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=bot_data")
options.add_argument("--headless") # Runs Chrome in headless mode.
options.add_argument('--no-sandbox') # Bypass OS security model
options.add_argument('--disable-gpu') # applicable to windows os only
options.add_argument('start-maximized') #
options.add_argument('disable-infobars')
options.add_argument("--disable-extensions")
# self.driver = webdriver.Chrome(ChromeDriverManager().install(), options=options)
self.driver = webdriver.Chrome('chromedriver.exe',
options=options)
self.driver.get('https://www.google.com')
self.wait = WebDriverWait(self.driver, 10)
There is have my codes. I want to change it to headless browser. But i am getting an error.
I added screenshot to show error.
This error message...
ERROR:devtools_http_handler.cc(288)] Error writing DevTools active port to file
...implies that there was an error while writing DevTools active port to the required file.
As per the discussion in How to open a Chrome Profile through Python instead of specifying only the directory name through user-data-dir, you need to pass the absolute path of the user-data-dir.
Solution
So you need to replace the line of code:
options.add_argument("user-data-dir=bot_data")
With:
options.add_argument("user-data-dir=C:\\Users\\AtechM_03\\AppData\\Local\\Google\\Chrome\\User Data\\bot_data")
Reference
You can find a couple of relevant discussions in:
How to use Chrome Profile in Selenium Webdriver Python 3
Selenium: Point towards default Chrome session
Outro
A couple of relevant documentations:
Session isolation in Headless Chrome
headless: Introduce a browser context
Save and restore browser sessions
Headless maintains a different profile folder structure to headful
When I'm trying to open a web page, its opening in a new chrome window stripped of all the extensions and modules. I'm not able to emulate the certain behavior of the website using selenium chrome browser window but I'm able to do the same thing in a normal chrome window without any issues.
from selenium import webdriver
driver = webdriver.Chrome(r'C:\chromedriver.exe')
driver.get("remote_worksplace_link")
id_box = driver.find_element_by_id('Enter user name')
id_box.send_keys('123456')
pass_box = driver.find_element_by_id('passwd')
pass_box.send_keys('123abc')
login_button = driver.find_element_by_id('Log_On')
login_button.click()
driver.implicitly_wait(2)
launch_button = driver.find_element_by_class_name('storeapp-icon ui-sortable-handle')
launch_button.click()
driver.implicitly_wait(5)
driver.close()
all extentions has its .crx file just you need to add those path
chrome_options = Options()
chrome_options.add_extension('path_to_extension')
driver = webdriver.Chrome(executable_path=executable_path, chrome_options=chrome_options)
driver.get("url")
driver.quit()
I am running this code with python, selenium, and firefox but still get 'head' version of firefox:
binary = FirefoxBinary('C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe', log_file=sys.stdout)
binary.add_command_line_options('-headless')
self.driver = webdriver.Firefox(firefox_binary=binary)
I also tried some variations of binary:
binary = FirefoxBinary('C:\\Program Files\\Nightly\\firefox.exe', log_file=sys.stdout)
binary.add_command_line_options("--headless")
To invoke Firefox Browser headlessly, you can set the headless property through Options() class as follows:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.headless = True
driver = webdriver.Firefox(options=options, executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("http://google.com/")
print ("Headless Firefox Initialized")
driver.quit()
There's another way to accomplish headless mode. If you need to disable or enable the headless mode in Firefox, without changing the code, you can set the environment variable MOZ_HEADLESS to whatever if you want Firefox to run headless, or don't set it at all.
This is very useful when you are using for example continuous integration and you want to run the functional tests in the server but still be able to run the tests in normal mode in your PC.
$ MOZ_HEADLESS=1 python manage.py test # testing example in Django with headless Firefox
or
$ export MOZ_HEADLESS=1 # this way you only have to set it once
$ python manage.py test functional/tests/directory
$ unset MOZ_HEADLESS # if you want to disable headless mode
Steps through YouTube Video
Mozilla Firefox in Headless Mode through Selenium 3.5.2 (Java)
Login into Gmail Account using Headless Chrome through Selenium Java
Outro
How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?
The first answer does't work anymore.
This worked for me:
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium import webdriver
options = FirefoxOptions()
options.add_argument("--headless")
driver = webdriver.Firefox(options=options)
driver.get("http://google.com")
My answer:
set_headless(headless=True) is deprecated.
https://seleniumhq.github.io/selenium/docs/api/py/webdriver_firefox/selenium.webdriver.firefox.options.html
options.headless = True
works for me
Used below code to set driver type based on need of Headless / Head for both Firefox and chrome:
// Can pass browser type
if brower.lower() == 'chrome':
driver = webdriver.Chrome('..\drivers\chromedriver')
elif brower.lower() == 'headless chrome':
ch_Options = Options()
ch_Options.add_argument('--headless')
ch_Options.add_argument("--disable-gpu")
driver = webdriver.Chrome('..\drivers\chromedriver',options=ch_Options)
elif brower.lower() == 'firefox':
driver = webdriver.Firefox(executable_path=r'..\drivers\geckodriver.exe')
elif brower.lower() == 'headless firefox':
ff_option = FFOption()
ff_option.add_argument('--headless')
ff_option.add_argument("--disable-gpu")
driver = webdriver.Firefox(executable_path=r'..\drivers\geckodriver.exe', options=ff_option)
elif brower.lower() == 'ie':
driver = webdriver.Ie('..\drivers\IEDriverServer')
else:
raise Exception('Invalid Browser Type')
To the OP or anyone currently interested, here's the section of code that's worked for me with firefox currently:
opt = webdriver.FirefoxOptions()
opt.add_argument('-headless')
ffox_driver = webdriver.Firefox(executable_path='\path\to\geckodriver', options=opt)
from selenium.webdriver.firefox.options import Options
if __name__ == "__main__":
options = Options()
options.add_argument('-headless')
driver = Firefox(executable_path='geckodriver', firefox_options=options)
wait = WebDriverWait(driver, timeout=10)
driver.get('http://www.google.com')
Tested, works as expected and this is from Official - Headless Mode | Mozilla
Nowadays with this code:
options = Options()
options.headless = True
driver = webdriver.Firefox(executable_path=GeckoDriverManager().install(),options=options)
We have a warning:
DeprecationWarning: executable_path has been deprecated, please pass in a Service object
Changing to this one, works perfectly:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# selenium drivers: https://www.selenium.dev/documentation/webdriver/getting_started/install_drivers/
# pip3 install selenium
# pip3 install webdriver-manager
# for custom firefox installation: link firefox to /usr/bin/firefox, example: ln -s /opt/firefox/firefox-bin /usr/bin/firefox
from selenium import webdriver
from webdriver_manager.firefox import GeckoDriverManager
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.service import Service
options = Options()
options.headless = True
service = Service(executable_path=GeckoDriverManager().install())
driver = webdriver.Firefox(service=service, options=options)
driver.get("http://google.com/")
print("Headless Firefox Initialized")
driver.quit()