chromedriver wont get website python selenium - python

I want my program to open up the default chrome profile and then get youtube.
I can make it either open youtube (in a new chrome browser), or open the default chrome profile, but not both.
(And no im not running both driver variables)
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
print('starting')
print('getting driver')
exec_path= "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe" # exec path from chrome://version
profilePath= 'C:\\Users\\MyName\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1' #profile path from chrome://version
chromePath= 'C:\\Users\\MyName\\OneDrive\\Documents\\Python programming\\chromedriver.exe' #path to driver
options= webdriver.ChromeOptions()
options.add_argument(profilePath)
print('options add argument...')
### Run one or the other ###
driver = webdriver.Chrome(executable_path= chromePath , options=options) #gets youtube
driver = webdriver.Chrome(executable_path= exec_path, options=options) #gets chrome profile
print('webdriver getting youtube...')
driver.get("https://www.youtube.com/")
when I run the driver line that gets the chrome profile I get the error:
Traceback (most recent call last):
File "C:\Users\MyName\OneDrive\Documents\Python programming\Web automation\webAuto.py", line 24, in
driver = webdriver.Chrome(executable_path= exec_path, options=options) #gets chrome profile
File "C:\Users\MyName\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in init
self.service.start()
File "C:\Users\MyName\AppData\Local\Programs\Python\Python39\lib\site-packages\selenium\webdriver\common\service.py", line 104, in start
raise WebDriverException("Can not connect to the Service %s" % self.path)
selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service C:\Program Files (x86)\Google\Chrome\Application\chrome.exe


As per your question and your code trials if you want to open a Chrome with the customized Chrome Profile, Try using below code.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('C:\\Users\\MyName\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 1' )
driver = webdriver.Chrome(executable_path='C:\\Users\\MyName\\OneDrive\\Documents\\Python programming\\chromedriver.exe', chrome_options=options)
driver.get("https://www.youtube.com/")
Here you will find a detailed discussion on How to open a Chrome Profile through Python

Related

How to keep the chrome browser open with Selenium in Python?

I want to open google page with a python code in visual studio editor using the following code :
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
#browser = webdriver.Chrome(ChromeDriverManager().install())
Path = "C:\Program Files (x86)\chromedriver.exe"
browser = webdriver.Chrome(Path)
browser.get('https://www.google.com/')
My problem is the chrome window it opens for 2 or 3 secondes and it close. Did some one face this issue ?

The default framework of visual studio editor would close the Chrome browser, per say any browser client after the last line of your program gets executed.
Hence, once the line of code:
browser.get('https://www.google.com/')
gets executed, the Chrome browser is CLOSED. This is the expected behavior.
Incase you like to keep the Chrome browser OPEN even after the last line of your program gets executed, you can pass the experimental option detach as true as follows:
selenium4 compatible code
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_experimental_option("detach", True)
s = Service('C:\\BrowserDrivers\\chromedriver.exe')
driver = webdriver.Chrome(service=s, options=options)
driver.get('https://www.google.com/')

chromedriver opens a website then immediately crashes using selenium 4 in vs code

from selenium import webdriver
PATH= "C:\Program Files (x86)\chromedriver.exe"
driver= webdriver.Chrome(PATH)
driver.get("https://www.ieltsadvantage.com/writing-task-1/")
after running the code it opens a website but immediately crashes and window closes

With selenium 4 you need to set the "detach" option to True when starting chromedriver:
from selenium import webdriver
PATH= "C:\Program Files (x86)\chromedriver.exe"
opts = webdriver.ChromeOptions() # Create options
opts.add_experimental_option("detach", True) # add detach
driver = webdriver.Chrome(PATH, options=opts) # bind options in Chrome()
driver.get("https://www.ieltsadvantage.com/writing-task-1/")
However, executable_path is deprecated. Here's an updated code:
from selenium import webdriver
PATH = "C:\Program Files (x86)\chromedriver.exe"
chrome_executable = webdriver.chrome.service.Service(executable_path=PATH) # use services instead
opts = webdriver.ChromeOptions()
opts.add_experimental_option("detach", True)
driver = webdriver.Chrome(service=chrome_executable, options=opts) # bind service
driver.get("https://www.ieltsadvantage.com/writing-task-1/")

How can i open websites in default chrome profile using selenium?

I was able to open default chrome profile with selenium but now i am not able to open any website using .get() method, what should i do?
here is the code
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=/home/deepinder/.config/google-chrome/")
driver = webdriver.Chrome(executable_path="/usr/local/bin/chromedriver", chrome_options=options)
driver.get("https://mail.google.com/mail/u/0/#inbox")
driver .get is not working and this is the error message that comes in the terminal -
File "default_chrome_profile.py", line 5, in <module>
w = webdriver.Chrome(executable_path="/usr/local/bin/chromedriver", chrome_options=options)

Can't fetch URL in chrome browser

I wrote this code which is in selenium webdriver in python.
I want to open link in chrome browser, the browser is getting started but can't fetch url.
def setUp(self):
self.browser = webdriver.Chrome("/usr/bin/google-chrome")
browser = self.browser
browser.implicitly_wait(5)
browser.get("http://www.google.com")
This gives this error:
ERROR: test_start (__main__.Saletest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "sale_test.py", line 16, in setUp
self.browser = webdriver.Chrome("/usr/bin/google-chrome")
File "/usr/local/lib/python2.7/dist- packages/selenium/webdriver/chrome/webdriver.py", line 60, in __init__
self.service.start()
File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/chrome/service.py", line 88, in start
os.path.basename(self.path) + "'")
WebDriverException: Message: Can not connect to the 'google-chrome'

Basically Selenium find your binary, but can't connect to it.
Try this, this seems to correspond to your problem :)
WebDriverException: Message: 'Can not connect to the ChromeDriver'. Error in utils.is_connectable(self.port):

The first argument that webdriver.Chrome expects to be provided with is the executable_path - a path to the chromedriver, but you are passing a path to the Chrome itself.
The executable_path argument is optional. In case chromedriver is in PATH and selenium can automatically find it, you don't need to explicitly state where it is located.
If you want to provide a custom Chrome browser executable instead, define the Chrome binary location via ChromeOptions:
options = Options()
options.binary_location = "/path/to/google-chrome"
driver = webdriver.Chrome(chrome_options=options)
And, just for completeness, here is how would the setup look in case you have a custom chromedriver path and a custom Chrome browser binary path:
options = Options()
options.binary_location = "/path/to/google-chrome"
driver = webdriver.Chrome(executable_path="/path/to/chromedriver",
chrome_options=options)

Python Selenium binding with TOR browser

I researched on it but I get that solution:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
driver = webdriver.Firefox(profile)
driver.get('http://estoeslapollaconcebol.la')
It gives that error:
Can't load the profile. Profile Dir:
C:\Users\HPPAV1~1\AppData\Local\Temp\tmppcuwx3xd Firefox output:
None
When I try that solution.
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
profile=webdriver.FirefoxProfile('C:\\Users\\HP PAV 15\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default\\')
binary =FirefoxBinary('C:\\Users\\HP PAV 15\\Desktop\\Tor Browser\\Browser\\firefox')
#browser = binary.launch_browser(profile)
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9150)
browser=webdriver.Firefox( binary, profile)
browser.get("http://yahoo.com")
browser.save_screenshot("/Users/admin/Pictures/screenshot.png")
browser.close()
It gives me the following error:
Traceback (most recent call last): File
"C:/Python34/torfirstscript.py", line 10, in
browser=webdriver.Firefox( binary, profile) File "C:\Python34\lib\site-packages\selenium-2.43.0-py3.4.egg\selenium\webdriver\firefox\webdriver.py",
line 46, in init
self.NATIVE_EVENTS_ALLOWED and self.profile.native_events_enabled) AttributeError: 'FirefoxBinary' object has no attribute
'native_events_enabled'
By applying
browser=webdriver.Firefox( firefox_binary = binary, firefox_profile = profile)
I got this error:
Traceback (most recent call last):
File "C:\Python34\torfirstscript.py", line 9, in
browser=webdriver.Firefox( firefox_binary = binary, firefox_profile = >profile)
File "C:\Python34\lib\site-packages\selenium-2.43.0->py3.4.egg\selenium\webdriver\firefox\webdriver.py", line 59, in init
self.binary, timeout),
File "C:\Python34\lib\site-packages\selenium-2.43.0->py3.4.egg\selenium\webdriver\firefox\extension_connection.py", line 47, in >init
self.binary.launch_browser(self.profile)
File "C:\Python34\lib\site-packages\selenium-2.43.0->py3.4.egg\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "C:\Python34\lib\site-packages\selenium-2.43.0-py3.4.egg\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))
selenium.common.exceptions.WebDriverException: Message: "Can't load the profile. Profile Dir: >C:\Users\HPPAV1~1\AppData\Local\Temp\tmpig7zvx_0\webdriver-py-profilecopy Firefox output: None"
with that image as output.

A working example with Selenium and Tor on windows :
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Program Files (x86)\TorBrowser\Browser\firefox.exe")
profile = FirefoxProfile(r"C:\Program Files (x86)\TorBrowser\Browser\TorBrowser\Data\Browser\profile.default")
driver = webdriver.Firefox(profile, binary)
driver.get("http://stackoverflow.com")
driver.save_screenshot("screenshot.png")
driver.quit()

I tried something like this, and worked:
profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9150)
driver = webdriver.Firefox(profile)
Open the Tor browser while you are doing this

Another simple solution is:
Create a new profile in Firefox or Chrome,
configure your browser to use Tor proxy (Set a SOCKS 5 proxy to address 127.0.0.1 port 9150), and then load that profile when you use webdriver.

Code for latest TOR installation on Windows:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Users\<Windows User>\Desktop\Tor Browser\Browser\firefox.exe")
profile = FirefoxProfile(r"C:\Users\<Windows User>\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default")
driver = webdriver.Firefox(profile, binary)
driver.get("http://stackoverflow.com")

This is what worked for me, this doesn't use the tor browser but geckodriver
pip install selenium webdriver-manager
import asyncio
import os
import subprocess
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
profile_path = os.path.expandvars(
r"%USERPROFILE%\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default"
)
options = Options()
options.set_preference("profile", profile_path)
service = Service(
# os.path.expandvars(r"%USERPROFILE%\Desktop\Tor Browser\Browser\firefox.exe"),
executable_path=GeckoDriverManager().install()
)
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.socks", "127.0.0.1")
options.set_preference("network.proxy.socks_port", 9050)
options.set_preference("network.proxy.socks_remote_dns", False)
async def main():
async def cleanup():
driver.quit()
print(torexe.pid)
torexe.kill()
try:
# https://stackoverflow.com/a/62686067/8608146
torexe = subprocess.Popen(
os.path.expandvars(
r"%USERPROFILE%\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe"
)
)
driver = Firefox(service=service, options=options)
driver.get("https://check.torproject.org")
driver.save_screenshot("screenshot.png")
except Exception as e:
print(e, type(e))
finally:
await cleanup()
if __name__ == "__main__":
asyncio.run(main())

I solved my similar problem on Windows:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Users\<Windows User>\Desktop\Tor Browser\Browser\firefox.exe")
driver = webdriver.Firefox(firefox_binary=binary)
driver.profile.set_preference('network.proxy.type', 1)
driver.profile.set_preference('network.proxy.socks', '127.0.0.1')
driver.profile.set_preference('network.proxy.socks_port', 9051)
driver.get("http://stackoverflow.com")

As some of these methods do not work in the current Windows versions, returning a "tor failed to start" error would inform users that, in order to start the proxy, they will need tor already running before executing your script.

This is working as of 05-12-2020. You need to be running tor browser before running this script. This will run Tor in Chrome. Will do that only in incognito mode. If you remove that option it will connect through your isp.
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
tor_proxy = "127.0.0.1:9150"
chrome_options = Options()
'''chrome_options.add_argument("--test-type")'''
chrome_options.add_argument('--ignore-certificate-errors')
'''chrome_options.add_argument('--disable-extensions')'''
chrome_options.add_argument('disable-infobars')
'''chrome_options.add_argument("--incognito")'''
chrome_options.add_argument('--user-data=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data\\Default')
chrome_options.add_argument('--proxy-server=socks5://%s' % tor_proxy)
driver = webdriver.Chrome(executable_path='C:\\chromedriver.exe', options=chrome_options)
driver.get('https://www.google.com')
time.sleep(4)
driver.switch_to.frame(0)
driver.find_element_by_id("introAgreeButton").click()

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium import webdriver
caps = DesiredCapabilities.FIREFOX
caps['proxy'] = {
'proxyType': 'MANUAL',
'socksProxy': '127.0.0.1:9050',
'socksVersion': 5
}
driver = webdriver.Firefox(executable_path=r"C:\webdrivers\geckodriver.exe", capabilities=caps)
In my case this code is the only one that works.

Update selenium using:
pip install -U selenium
Then run your code, after starting TOR of course.
This error was acknowledged and repaired.

Categories

Resources