Develop Reference

using an if statment with selenium [duplicate]

First, I want to use some addons while selenium controlling my firefox.
So, i tried load default profile of firefox in selenium code.
My code:
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
profile_path = r'C:\Users\Administrator\AppData\Roaming\Mozilla\Firefox\Profiles\y1uqp5mi.default'
default_profile = FirefoxProfile(profile_path)
driver = webdriver.Firefox(service=service, options=options, firefox_profile=default_profile)
But, when i start the code, a DeprecationWarning happened: firefox_profile has been deprecated, please pass in an Options object
I search a lot and i don't think it's a difficult problem, but sadly i can't solve this problem finally, maybe my bad english encumber me... ...

Here is the documentation for this:
https://www.selenium.dev/documentation/webdriver/capabilities/driver_specific_capabilities/#setting-a-custom-profile
I tried this locally and it worked:
EDITED: I've changed the code, so there are no deprecation warnings
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Administrator\AppData\Roaming\Mozilla\Firefox\Profiles\y1uqp5mi.default'
options=Options()
options.set_preference('profile', profile_path)
service = Service(r'C:\WebDriver\bin\geckodriver.exe')
driver = Firefox(service=service, options=options)
driver.get("https://selenium.dev")
driver.quit()

This error message...
firefox_profile has been deprecated, please pass in an Options object
...implies that FirefoxProfile() have been Deprecated and with selenium4 to use a custom profile you have to use an instance of Options.
This DeprecationWarning was inline with the following CHANGELOGS:
Selenium 4 beta 1
Deprecate all but Options and Service arguments in driver instantiation. (#9125,#9128)
Selenium 4 beta 2
Deprecate using a Firefox profile in Options
Selenium 4 Beta 3
Only give deprecation warning if Profile is being used in options
All the configurations which was earlier set through profile.set_preference() now can be set through options.set_preference() as follows:
from selenium.webdriver import Firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Admin\AppData\Roaming\Mozilla\Firefox\Profiles\s8543x41.default-release'
options=Options()
options.set_preference('profile', profile_path)
options.set_preference('network.proxy.type', 1)
options.set_preference('network.proxy.socks', '127.0.0.1')
options.set_preference('network.proxy.socks_port', 9050)
options.set_preference('network.proxy.socks_remote_dns', False)
service = Service('C:\\BrowserDrivers\\geckodriver.exe')
driver = Firefox(service=service, options=options)
driver.get("https://www.google.com")
driver.quit()
tl; dr
Setting a custom profile

I tried this
from selenium.webdriver.firefox.options import Options
profile_path = r'C:\Users\Administrator\AppData\Roaming\Mozilla\Firefox\Profiles\y1uqp5mi.default'
options=Options()
options.set_preference('profile', profile_path)
driver = Firefox(options=options)

I tried using webdriver manager but i keep getting this error

I tried selenium automation with webdriver but I keep getting errors. Please help me fix the problem
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from time import sleep
browser = webdriver.Chrome("chromedriver.exe")
browser.get("https://python.org")
the Error I received:
File "C:\Users\LUCKY-PC\OneDrive\Desktop\Python\automation\selenium_test.py", line 2, in <module>
from webdriver_manager.chrome import ChromeDriverManager
File "C:\Users\LUCKY-PC\anaconda3\lib\site-packages\webdriver_manager\chrome.py", line 4, in <module>
from webdriver_manager import utils
File "C:\Users\LUCKY-PC\anaconda3\lib\site-packages\webdriver_manager\utils.py", line 8, in <module>
import requests
File "C:\Users\LUCKY-PC\anaconda3\lib\site-packages\requests\__init__.py", line 95, in <module>
from urllib3.contrib import pyopenssl
File "C:\Users\LUCKY-PC\anaconda3\lib\site-packages\urllib3\contrib\pyopenssl.py", line 109, in <module>
orig_util_SSLContext = util.ssl_.SSLContext
AttributeError: module 'urllib3.util' has no attribute 'ssl_'

Try with ChromeDriverManager().install():
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
browser = webdriver.Chrome(ChromeDriverManager().install())
browser.get("https://python.org")

It's better if you download the chrome driver and then provide the chrome driver path to code.
Go to your chrome browser and type chrome://settings/help. From here you can find your chrome version.
Then go to https://chromedriver.chromium.org/downloads to download a chrome driver that matches your chrome version.
Now give the path to of that driver in the code below.
import os
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
class Driver_Class(webdriver.Chrome):
def __init__(self, driver_path, teardown=False):
self.driver_path = driver_path
self.options = Options()
self.options.headless = False
self.driver = webdriver.Chrome(executable_path=self.driver_path,options=self.options)
self.options.add_argument('--ignore-certificate-errors')
self.options.add_argument('--ignore-ssl-errors')
self.teardown = teardown
os.environ['PATH'] += self.driver_path
self.driver.implicitly_wait(30)
self.driver.maximize_window()
def get_driver(self):
return self.driver
driverObj = Driver_Class("chrome_driver_path")
driver = driverObj.get_driver()
Now you can use this driver for the rest of your program.

Error with selenium for send_keys : str' object has no attribute 'send_keys'

I am going to scrape with selenium; I followed the instruction but I have an error regarding "send_key" when I am going to send my username and password :
runfile('C:/Users/thmag/untitled3.py', wdir='C:/Users/thmag')
C:\Users\thmag\untitled3.py:20: DeprecationWarning: use options instead of chrome_options
driver= webdriver.Chrome(driver_path, chrome_options = options)
Traceback (most recent call last):
File "C:\Users\thmag\untitled3.py", line 30, in <module>
user_ele.send_keys('MyEmail#gmail.com')
AttributeError: 'str' object has no attribute 'send_keys'
My code is as follow:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd
import time
driver_path = r"C:\\Program Files (x86)\\chromedriver.exe"
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver= webdriver.Chrome(driver_path, chrome_options = options)
url= "https://healthunlocked.com/"
driver.get(url)
loginpage= driver.find_element_by_id("sitebar-login-button")
loginpage.send_keys(Keys.ENTER)
user_ele = driver.find_element_by_xpath('//*[#id="email"]')
user_ele.send_keys('MyEmail#gmail.com')
user_ele.send_keys(Keys.ENTER)
pass_ele = driver.find_element_by_xpath('//*[#id="password"]')
pass_ele.send_keys('MyPass')
pass_ele.send_keys(Keys.ENTER)
time.sleep(10)
driver.quit()

Clear the placeholder, then send keys :
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import pandas as pd
import time
options = webdriver.ChromeOptions()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver= webdriver.Chrome(options = options)
url= "https://healthunlocked.com/"
driver.get(url)
# closing cookies thing
driver.find_element_by_id('ccc-notify-accept').click()
loginpage= driver.find_element_by_id("sitebar-login-button").click()
time.sleep(3)
user_ele = driver.find_element_by_id('email')
user_ele.clear()
user_ele.send_keys('MyEmail#gmail.com')
pass_ele = driver.find_element_by_xpath('//*[#id="password"]')
pass_ele.clear()
pass_ele.send_keys('MyPassword')
# submitting
driver.find_element_by_xpath('/html/body/div[3]/div/div[1]/div/div/section/div[1]/form/button').click()
time.sleep(10)
driver.quit()
if you just want to login, you can use requests to send a post request

Use options instead of chrome options and set user_ele.
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument('--ignore-certificate-errors')
options.add_argument('--ignore-ssl-errors')
driver= webdriver.Chrome(driver_path, options = options)
url= "https://healthunlocked.com/"
driver.get(url)
loginpage= driver.find_element_by_id("sitebar-login-button")
loginpage.send_keys(Keys.ENTER)
user_ele = driver.find_element_by_xpath('//*[#id="email"]')
user_ele.send_keys('MyEmail#gmail.com')
user_ele.send_keys(Keys.ENTER)
pass_ele = driver.find_element_by_xpath('//*[#id="password"]')
pass_ele.send_keys('MyPass')
pass_ele.send_keys(Keys.ENTER)

Unable to invoke firefox headless

I would like to know why i'm getting the following error message and how can i fix it:
Traceback (most recent call last):
File "teste.py", line 30, in <\module>
main()
File "teste.py", line 24, in main
driver = connectFirefox(defineOptions())
File "teste.py", line 18, in connectFirefox
driver = webdriver.Firefox(firefox_options=options)
File "/usr/lib/python3.6/site-packages/selenium/webdriver/firefox
/webdriver.py", line 125, in init
if options.binary is not None:
AttributeError: 'Options' object has no attribute 'binary'
My code:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.chrome.options import Options
def defineOptions():
options = Options()
options.add_argument("--headless")
return options
def connectChrome(options):
chromeDriverPath = "/usr/bin/chromedriver"
driver = webdriver.Chrome(chromeDriverPath, chrome_options=options)
print("Chrome Headless Browser Invoked")
return driver
def connectFirefox(options):
driver = webdriver.Firefox(firefox_options=options)
print("Firefox Headless Browser Invoked")
return driver
def main():
#driver = connectChrome(defineOptions())
driver = connectFirefox(defineOptions())
driver.get("https://www.archlinux.org/")
print("Headless Browser closing")
driver.quit()
#------------------------------------------------------------------------#
main()
What I'm trying to do is write a code where I can easily choose Chrome or Firefox headless.

I solved my problem. Here the code:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options as FirefoxOptions
from selenium.webdriver.chrome.options import Options as ChromeOptions
def connectChrome():
options = ChromeOptions()
options.add_argument("--headless")
chromeDriverPath = "/usr/bin/chromedriver"
driver = webdriver.Chrome(chromeDriverPath, chrome_options=options)
print("Chrome Headless Browser Invoked")
return driver
def connectFirefox():
options = FirefoxOptions()
options.add_argument("--headless")
driver = webdriver.Firefox(firefox_options=options)
print("Firefox Headless Browser Invoked")
return driver
def main():
#driver = connectChrome()
driver = connectFirefox()
driver.get("https://www.archlinux.org/")
print("Headless Browser closing")
driver.quit()
#------------------------------------------------------------------------#
main()

Python Selenium binding with TOR browser

I researched on it but I get that solution:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
driver = webdriver.Firefox(profile)
driver.get('http://estoeslapollaconcebol.la')
It gives that error:
Can't load the profile. Profile Dir:
C:\Users\HPPAV1~1\AppData\Local\Temp\tmppcuwx3xd Firefox output:
None
When I try that solution.
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
profile=webdriver.FirefoxProfile('C:\\Users\\HP PAV 15\\Desktop\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default\\')
binary =FirefoxBinary('C:\\Users\\HP PAV 15\\Desktop\\Tor Browser\\Browser\\firefox')
#browser = binary.launch_browser(profile)
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9150)
browser=webdriver.Firefox( binary, profile)
browser.get("http://yahoo.com")
browser.save_screenshot("/Users/admin/Pictures/screenshot.png")
browser.close()
It gives me the following error:
Traceback (most recent call last): File
"C:/Python34/torfirstscript.py", line 10, in
browser=webdriver.Firefox( binary, profile) File "C:\Python34\lib\site-packages\selenium-2.43.0-py3.4.egg\selenium\webdriver\firefox\webdriver.py",
line 46, in init
self.NATIVE_EVENTS_ALLOWED and self.profile.native_events_enabled) AttributeError: 'FirefoxBinary' object has no attribute
'native_events_enabled'
By applying
browser=webdriver.Firefox( firefox_binary = binary, firefox_profile = profile)
I got this error:
Traceback (most recent call last):
File "C:\Python34\torfirstscript.py", line 9, in
browser=webdriver.Firefox( firefox_binary = binary, firefox_profile = >profile)
File "C:\Python34\lib\site-packages\selenium-2.43.0->py3.4.egg\selenium\webdriver\firefox\webdriver.py", line 59, in init
self.binary, timeout),
File "C:\Python34\lib\site-packages\selenium-2.43.0->py3.4.egg\selenium\webdriver\firefox\extension_connection.py", line 47, in >init
self.binary.launch_browser(self.profile)
File "C:\Python34\lib\site-packages\selenium-2.43.0->py3.4.egg\selenium\webdriver\firefox\firefox_binary.py", line 64, in launch_browser
self._wait_until_connectable()
File "C:\Python34\lib\site-packages\selenium-2.43.0-py3.4.egg\selenium\webdriver\firefox\firefox_binary.py", line 108, in _wait_until_connectable
self.profile.path, self._get_firefox_output()))
selenium.common.exceptions.WebDriverException: Message: "Can't load the profile. Profile Dir: >C:\Users\HPPAV1~1\AppData\Local\Temp\tmpig7zvx_0\webdriver-py-profilecopy Firefox output: None"
with that image as output.

A working example with Selenium and Tor on windows :
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Program Files (x86)\TorBrowser\Browser\firefox.exe")
profile = FirefoxProfile(r"C:\Program Files (x86)\TorBrowser\Browser\TorBrowser\Data\Browser\profile.default")
driver = webdriver.Firefox(profile, binary)
driver.get("http://stackoverflow.com")
driver.save_screenshot("screenshot.png")
driver.quit()

I tried something like this, and worked:
profile = webdriver.FirefoxProfile()
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9150)
driver = webdriver.Firefox(profile)
Open the Tor browser while you are doing this

Another simple solution is:
Create a new profile in Firefox or Chrome,
configure your browser to use Tor proxy (Set a SOCKS 5 proxy to address 127.0.0.1 port 9150), and then load that profile when you use webdriver.

Code for latest TOR installation on Windows:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Users\<Windows User>\Desktop\Tor Browser\Browser\firefox.exe")
profile = FirefoxProfile(r"C:\Users\<Windows User>\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default")
driver = webdriver.Firefox(profile, binary)
driver.get("http://stackoverflow.com")

This is what worked for me, this doesn't use the tor browser but geckodriver
pip install selenium webdriver-manager
import asyncio
import os
import subprocess
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
profile_path = os.path.expandvars(
r"%USERPROFILE%\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default"
)
options = Options()
options.set_preference("profile", profile_path)
service = Service(
# os.path.expandvars(r"%USERPROFILE%\Desktop\Tor Browser\Browser\firefox.exe"),
executable_path=GeckoDriverManager().install()
)
options.set_preference("network.proxy.type", 1)
options.set_preference("network.proxy.socks", "127.0.0.1")
options.set_preference("network.proxy.socks_port", 9050)
options.set_preference("network.proxy.socks_remote_dns", False)
async def main():
async def cleanup():
driver.quit()
print(torexe.pid)
torexe.kill()
try:
# https://stackoverflow.com/a/62686067/8608146
torexe = subprocess.Popen(
os.path.expandvars(
r"%USERPROFILE%\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe"
)
)
driver = Firefox(service=service, options=options)
driver.get("https://check.torproject.org")
driver.save_screenshot("screenshot.png")
except Exception as e:
print(e, type(e))
finally:
await cleanup()
if __name__ == "__main__":
asyncio.run(main())

I solved my similar problem on Windows:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
binary = FirefoxBinary(r"C:\Users\<Windows User>\Desktop\Tor Browser\Browser\firefox.exe")
driver = webdriver.Firefox(firefox_binary=binary)
driver.profile.set_preference('network.proxy.type', 1)
driver.profile.set_preference('network.proxy.socks', '127.0.0.1')
driver.profile.set_preference('network.proxy.socks_port', 9051)
driver.get("http://stackoverflow.com")

As some of these methods do not work in the current Windows versions, returning a "tor failed to start" error would inform users that, in order to start the proxy, they will need tor already running before executing your script.

This is working as of 05-12-2020. You need to be running tor browser before running this script. This will run Tor in Chrome. Will do that only in incognito mode. If you remove that option it will connect through your isp.
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
tor_proxy = "127.0.0.1:9150"
chrome_options = Options()
'''chrome_options.add_argument("--test-type")'''
chrome_options.add_argument('--ignore-certificate-errors')
'''chrome_options.add_argument('--disable-extensions')'''
chrome_options.add_argument('disable-infobars')
'''chrome_options.add_argument("--incognito")'''
chrome_options.add_argument('--user-data=C:\\Users\\user\\AppData\\Local\\Google\\Chrome\\User Data\\Default')
chrome_options.add_argument('--proxy-server=socks5://%s' % tor_proxy)
driver = webdriver.Chrome(executable_path='C:\\chromedriver.exe', options=chrome_options)
driver.get('https://www.google.com')
time.sleep(4)
driver.switch_to.frame(0)
driver.find_element_by_id("introAgreeButton").click()

from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium import webdriver
caps = DesiredCapabilities.FIREFOX
caps['proxy'] = {
'proxyType': 'MANUAL',
'socksProxy': '127.0.0.1:9050',
'socksVersion': 5
}
driver = webdriver.Firefox(executable_path=r"C:\webdrivers\geckodriver.exe", capabilities=caps)
In my case this code is the only one that works.

Update selenium using:
pip install -U selenium
Then run your code, after starting TOR of course.
This error was acknowledged and repaired.