Using Python Selenium Webdriver to open Electron Application

Using Python Selenium Webdriver to open Electron Application - python

I've been attempting to bypass using Spectron for End2End testing an electron application by leveraging my experience with Selenium Webdriver on Python.
Using a combination of the Chromedriver get started page, and several resources that seem to suggest its possible, this is what I came up with:
from selenium import webdriver
import selenium.webdriver.chrome.service as service
servicer = service.Service('C:\\browserDrivers\\chromedriver_win32\\chromedriver.exe')
servicer.start()
capabilities = {'chrome.binary': 'C:\\path\\to\\electron.exe'}
remote = webdriver.remote.webdriver.WebDriver(command_executor=servicer.service_url, desired_capabilities = capabilities, browser_profile=None, proxy=None, keep_alive=False
The issue is that instead of opening the electron application, it opens a standard instance of Chrome.
Most of resources I've seen have been several years old so something may have changed to make it no longer possible.
Does anyone know of a way to use Python Selenium WebDriver to test an Electron application?

Below works great for me
from selenium import webdriver
options = webdriver.ChromeOptions()
options.binary_location = "/Applications/Electron.app/Contents/MacOS/Electron"
driver = webdriver.Chrome(chrome_options=options)
driver.get("http://www.google.com")
driver.quit()

Related

Selenium either Custom Profile or get() method not working

I was trying to control an already open instance of Chrome, but when that didn't work, I started to try to make a new instance of Chrome on my profile. I found two stackoverflow posts detailing the problem (How to use Chrome Profile in Selenium Webdriver Python 3 and Selenium: get() not working with custom google profile) but neither solution worked. I have two code blocks, both of which has its own problem.
The first thing I tried was this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument(r"--user-data-dir=C:\\Users\\EliBa\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument(r'--profile-directory=Default')
driver = webdriver.Chrome(executable_path=r'C:\\Selenium Drivers\\chromedriver.exe', chrome_options=options)
driver.get("https://google.com")
While this opened my profile, it did not open google like I asked it to. The second thing I tried was this:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("user-data-dir=C:\\Users\\EliBa\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
driver = webdriver.Chrome(executable_path='C:\\Selenium Drivers\\chromedriver.exe', options=options)
driver.get("https://www.google.com")
While this opened Google, it did not open my custom profile. Does anyone know what I should be putting so that the custom profile opens AND it loads the site I ask it to?

Selenium python - How to load Default Chrome Profile

I need to load my own Default chrome profile where all my saved login info is, instead of selenium logging me into my websites.
I have taken this code from stackoverflow, cannot find the exact user. My issue with this code is selenium opens chrome but does not load the correct profile. Every time I open run the code it creates a new "scoped_dir" folder and runs the profile "default" from there chrome version
(C:\Users\farja\AppData\Local\Temp\scoped_dir[bunch of numbers]\Default).
I have tried closing chrome and then running my code which doesn't work
I am thinking there is a big flaw in my code but do not know what it is or how to find it. A relevant answer for 2022 would be very very much appreciated as I have literally been stuck on this for a week now and have tried multiple answers on stackoverflow, the web and youtube but nearly all give me a deprecation error.
Thank you for taking the time to read.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
ser = Service(r'C:\Users\farja\Documents\Instagram Programmes\Scheduler 2\chromedriver')
op = webdriver.ChromeOptions()
s = webdriver.Chrome(service=ser, options=op)
op.add_argument(r"--user-data-dir=C:\\Users\\farja\\AppData\\Local\\Google\\Chrome\\User Data")
op.add_argument("--profile-directory=Default")
s.get("chrome://version/")

As your usecase is to load the Default Chrome Profile you can use the argument user-data-dir and remove the argument --profile-directory as follows:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
op = webdriver.ChromeOptions()
op.add_argument(r"--user-data-dir=C:\\Users\\farja\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
ser = Service(r'C:\Users\farja\Documents\Instagram Programmes\Scheduler 2\chromedriver')
s = webdriver.Chrome(service=ser, options=op)
s.get("chrome://version/")
References
You can find a couple of relevant detailed discussions in:
How to open a Chrome Profile through Python
How to open a Chrome Profile through --user-data-dir argument of Selenium

Is there anyway to make selenium without headless mode on Firefox? [duplicate]

Is there a way to make your Selenium script undetectable in Python using geckodriver?
I'm using Selenium for scraping. Are there any protections we need to use so websites can't detect Selenium?

There are different methods to avoid websites detecting the use of Selenium.
The value of navigator.webdriver is set to true by default when using Selenium. This variable will be present in Chrome as well as Firefox. This variable should be set to "undefined" to avoid detection.
A proxy server can also be used to avoid detection.
Some websites are able to use the state of your browser to determine if you are using Selenium. You can set Selenium to use a custom browser profile to avoid this.
The code below uses all three of these approaches.
profile = webdriver.FirefoxProfile('C:\\Users\\You\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\something.default-release')
PROXY_HOST = "12.12.12.123"
PROXY_PORT = "1234"
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", PROXY_HOST)
profile.set_preference("network.proxy.http_port", int(PROXY_PORT))
profile.set_preference("dom.webdriver.enabled", False)
profile.set_preference('useAutomationExtension', False)
profile.update_preferences()
desired = DesiredCapabilities.FIREFOX
driver = webdriver.Firefox(firefox_profile=profile, desired_capabilities=desired)
Once the code is run, you will be able to manually check that the browser run by Selenium now has your Firefox history and extensions. You can also type "navigator.webdriver" into the devtools console to check that it is undefined.

The fact that selenium driven Firefox / GeckoDriver gets detected doesn't depends on any specific GeckoDriver or Firefox version. The Websites themselves can detect the network traffic and can identify the Browser Client i.e. Web Browser as WebDriver controled.
As per the documentation of the WebDriver Interface in the latest editor's draft of WebDriver - W3C Living Document the webdriver-active flag which is initially set as false, is set to true when the user agent is under remote control i.e. when controlled through Selenium.
Now that the NavigatorAutomationInformation interface should not be exposed on WorkerNavigator.
So,
webdriver
Returns true if webdriver-active flag is set, false otherwise.
where as,
navigator.webdriver
Defines a standard way for co-operating user agents to inform the document that it is controlled by WebDriver, for example so that alternate code paths can be triggered during automation.
So, the bottom line is:
Selenium identifies itself
However some generic approaches to avoid getting detected while web-scraping are as follows:
The first and foremost attribute a website can determine your script/program is through your monitor size. So it is recommended not to use the conventional Viewport.
If you need to send multiple requests to a website, you need to keep on changing the User Agent on each request. Here you can find a detailed discussion on Way to change Google Chrome user agent in Selenium?
To simulate human like behavior you may require to slow down the script execution even beyond WebDriverWait and expected_conditions inducing time.sleep(secs). Here you can find a detailed discussion on How to sleep webdriver in python for milliseconds

As per the current WebDriver W3C Editor's Draft specification:
The webdriver-active flag is set to true when the user agent is under remote control. It is initially false.
Hence, the readonly boolean attribute webdriver returns true if webdriver-active flag is set, false otherwise.
Further the specification further clarifies:
navigator.webdriver Defines a standard way for co-operating user
agents to inform the document that it is controlled by WebDriver, for
example so that alternate code paths can be triggered during
automation.
There had been tons and millions of discussions demanding Feature: option to disable navigator.webdriver == true ? and #whimboo in his comment concluded that:
that is because the WebDriver spec defines that property on the
Navigator object, which has to be set to true when tests are running
with webdriver enabled:
https://w3c.github.io/webdriver/#interface
Implementations have to be conformant to this requirement. As such we
will not provide a way to circumvent that.
Generic Conclusion
From the above discussions it can be concluded that:
Selenium identifies itself
and there is no way to conceal the fact that the browser is WebDriver driven.
Recommendations
However some users have suggested approaches which can conceal the fact that the Mozilla Firefox browser is WebDriver controled through the usage of Firefox Profiles and Proxies as follows:
selenium4 compatible python code
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Admin\AppData\Roaming\Mozilla\Firefox\Profiles\s8543x41.default-release'
options=Options()
options.set_preference('profile', profile_path)
options.set_preference('network.proxy.type', 1)
options.set_preference('network.proxy.socks', '127.0.0.1')
options.set_preference('network.proxy.socks_port', 9050)
options.set_preference('network.proxy.socks_remote_dns', False)
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = Firefox(service=service, options=options)
driver.get("https://www.google.com")
driver.quit()
Other Alternatives
It is observed that in some specific os variants a couple of diverse settings/configuration can bypass the bot detectation which are as follows:
selenium4 compatible code block
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = webdriver.Chrome(service=s, options=options)
Potential Solution
A potential solution would be to use the tor browser as follows:
selenium4 compatible python code
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
import os
torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile_path = r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default'
firefox_options=Options()
firefox_options.set_preference('profile', profile_path)
firefox_options.set_preference('network.proxy.type', 1)
firefox_options.set_preference('network.proxy.socks', '127.0.0.1')
firefox_options.set_preference('network.proxy.socks_port', 9050)
firefox_options.set_preference("network.proxy.socks_remote_dns", False)
firefox_options.binary_location = r'C:\Users\username\Desktop\Tor Browser\Browser\firefox.exe'
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = webdriver.Firefox(service=service, options=firefox_options)
driver.get("https://www.tiktok.com/")

It may sound simple, but if you look how the website detects selenium (or bots) is by tracking the movements, so if you can make your program slightly towards like a human is browsing the website you can get less captcha, such as add cursor/page scroll movements in between your operations, and other actions which mimics the browsing. So between two operations try to add some other actions, Add some delay etc. This will make your bot slower and could get undetected.
Thanks

How to initiate a Chromium based Vivaldi browser session using Selenium and Python

I am trying to use the vivaldi browser with Selenium. It is a chromium browser that runs very similar to chrome. I have Selenium working with Firefox (geckodriver), and Google Chrome(chromedriver), but I can't seem to find a way with Vivaldi. Any help would be appreciated.

If the vivaldi binary by default is located at C:\Users\levir\AppData\Local\Vivaldi\Application\vivaldi.exe you can use the following solution:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("start-maximized")
options.binary_location=r'C:\Users\levir\AppData\Local\Vivaldi\Application\vivaldi.exe'
driver = webdriver.Chrome(executable_path=r'C:\path\to\chromedriver.exe', options=options)
driver.get('http://google.com/')

For future reference:
to make Vivaldi work with selenium you need to make sure of three things :
The correct version of ChromeDriver
Set selenium's driver to use Vivaldi's binary via webdriver.ChromeOptions()
Make sure you're getting the correct url (don't forget "https://")
All of the above is explained step by step with screenshots in this blog post

The key executable_path will be deprecated in the upcoming releases of Selenium.
This post has the solution. I'm posting a copy of said solution with the path to Vivaldi, where the username is fetched by the script so you don't have to hard code it.
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
import os
current_user = os.getlogin()
s = Service(rf"C:\Users\{current_user}\AppData\Local\Vivaldi\Application\vivaldi.exe")
driver = webdriver.Chrome(service=s)
driver.get("http://duckduckgo.com") # or your website of choice

You can use ChromeOptions and supply binary.
from selenium.webdriver.chrome.options import Options
opt = Options()
opt.binary_location = chromium_path//path to chromium binary
driver = webdriver.Chrome(options=opt, executable_path="path_to_chromedriver")

How can I make a Selenium script undetectable using GeckoDriver and Firefox through Python?

Is there a way to make your Selenium script undetectable in Python using geckodriver?
I'm using Selenium for scraping. Are there any protections we need to use so websites can't detect Selenium?

There are different methods to avoid websites detecting the use of Selenium.
The value of navigator.webdriver is set to true by default when using Selenium. This variable will be present in Chrome as well as Firefox. This variable should be set to "undefined" to avoid detection.
A proxy server can also be used to avoid detection.
Some websites are able to use the state of your browser to determine if you are using Selenium. You can set Selenium to use a custom browser profile to avoid this.
The code below uses all three of these approaches.
profile = webdriver.FirefoxProfile('C:\\Users\\You\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\something.default-release')
PROXY_HOST = "12.12.12.123"
PROXY_PORT = "1234"
profile.set_preference("network.proxy.type", 1)
profile.set_preference("network.proxy.http", PROXY_HOST)
profile.set_preference("network.proxy.http_port", int(PROXY_PORT))
profile.set_preference("dom.webdriver.enabled", False)
profile.set_preference('useAutomationExtension', False)
profile.update_preferences()
desired = DesiredCapabilities.FIREFOX
driver = webdriver.Firefox(firefox_profile=profile, desired_capabilities=desired)
Once the code is run, you will be able to manually check that the browser run by Selenium now has your Firefox history and extensions. You can also type "navigator.webdriver" into the devtools console to check that it is undefined.

As per the current WebDriver W3C Editor's Draft specification:
The webdriver-active flag is set to true when the user agent is under remote control. It is initially false.
Hence, the readonly boolean attribute webdriver returns true if webdriver-active flag is set, false otherwise.
Further the specification further clarifies:
navigator.webdriver Defines a standard way for co-operating user
agents to inform the document that it is controlled by WebDriver, for
example so that alternate code paths can be triggered during
automation.
There had been tons and millions of discussions demanding Feature: option to disable navigator.webdriver == true ? and #whimboo in his comment concluded that:
that is because the WebDriver spec defines that property on the
Navigator object, which has to be set to true when tests are running
with webdriver enabled:
https://w3c.github.io/webdriver/#interface
Implementations have to be conformant to this requirement. As such we
will not provide a way to circumvent that.
Generic Conclusion
From the above discussions it can be concluded that:
Selenium identifies itself
and there is no way to conceal the fact that the browser is WebDriver driven.
Recommendations
However some users have suggested approaches which can conceal the fact that the Mozilla Firefox browser is WebDriver controled through the usage of Firefox Profiles and Proxies as follows:
selenium4 compatible python code
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Admin\AppData\Roaming\Mozilla\Firefox\Profiles\s8543x41.default-release'
options=Options()
options.set_preference('profile', profile_path)
options.set_preference('network.proxy.type', 1)
options.set_preference('network.proxy.socks', '127.0.0.1')
options.set_preference('network.proxy.socks_port', 9050)
options.set_preference('network.proxy.socks_remote_dns', False)
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = Firefox(service=service, options=options)
driver.get("https://www.google.com")
driver.quit()
Other Alternatives
It is observed that in some specific os variants a couple of diverse settings/configuration can bypass the bot detectation which are as follows:
selenium4 compatible code block
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.chrome.service import Service
options = Options()
options.add_experimental_option("excludeSwitches", ["enable-automation"])
options.add_experimental_option('excludeSwitches', ['enable-logging'])
options.add_experimental_option('useAutomationExtension', False)
options.add_argument('--disable-blink-features=AutomationControlled')
s = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = webdriver.Chrome(service=s, options=options)
Potential Solution
A potential solution would be to use the tor browser as follows:
selenium4 compatible python code
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
import os
torexe = os.popen(r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile_path = r'C:\Users\username\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default'
firefox_options=Options()
firefox_options.set_preference('profile', profile_path)
firefox_options.set_preference('network.proxy.type', 1)
firefox_options.set_preference('network.proxy.socks', '127.0.0.1')
firefox_options.set_preference('network.proxy.socks_port', 9050)
firefox_options.set_preference("network.proxy.socks_remote_dns", False)
firefox_options.binary_location = r'C:\Users\username\Desktop\Tor Browser\Browser\firefox.exe'
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = webdriver.Firefox(service=service, options=firefox_options)
driver.get("https://www.tiktok.com/")

It may sound simple, but if you look how the website detects selenium (or bots) is by tracking the movements, so if you can make your program slightly towards like a human is browsing the website you can get less captcha, such as add cursor/page scroll movements in between your operations, and other actions which mimics the browsing. So between two operations try to add some other actions, Add some delay etc. This will make your bot slower and could get undetected.
Thanks

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.