I am facing a problem automating the protection settings in IE using Selenium with python.
I found a solution to automate the settings in java but it is not working when i changed it to python .
I tried the following::
from selenium import webdriver
caps=webdriver.DesiredCapabilites.INTERNETEXPLORER
caps['INTRODUCE_FLAKINESS_BY_IGNORING_SECURITY_DOMAINS']=True
driver=webdriver.Ie(caps)
This gave an error with respect to the argument given .
and when I used driver = webdriver.Ie()
It says protection mode settings must be same for all zones.
Can anyone help me automate this thing using selenium in python.
According to documentation, in python-selenum, you should use setting called
ignoreProtectedModeSettings:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
caps = DesiredCapabilities.INTERNETEXPLORER
caps['ignoreProtectedModeSettings'] = True
driver = webdriver.Ie(capabilities=caps)
Desired capabilities doesn't work in some instances. Here is a method to change protection settings from the registry using winreg.
from winreg import *
def Enable_Protected_Mode():
# SECURITY ZONES ARE AS FOLLOWS:
# 0 is the Local Machine zone
# 1 is the Intranet zone
# 2 is the Trusted Sites zone
# 3 is the Internet zone
# 4 is the Restricted Sites zone
# CHANGING THE SUBKEY VALUE "2500" TO DWORD 0 ENABLES PROTECTED MODE FOR THAT ZONE.
# IN THE CODE BELOW THAT VALUE IS WITHIN THE "SetValueEx" FUNCTION AT THE END AFTER "REG_DWORD".
#os.system("taskkill /F /IM iexplore.exe")
try:
keyVal = r'Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\1'
key = OpenKey(HKEY_CURRENT_USER, keyVal, 0, KEY_ALL_ACCESS)
SetValueEx(key, "2500", 0, REG_DWORD, 0)
print("enabled protected mode")
except Exception:
print("failed to enable protected mode")
In case capabilities mode does not work, there is an alternative.
from selenium import webdriver
from selenium.webdriver.ie.options import Options
ie_options = Options()
ie_options.ignore_protected_mode_settings = True
driver = webdriver.Ie(options=ie_options)
driver.get('http://www.google.com')
Here is another variation on Dinesh's code that works to disable protected modes in the registry. It also closes the connection.
Simply put this code before your selenium automation code, and it will prepare the browser.
import winreg
def set_reg(REG_PATH, name, value):
try:
winreg.CreateKey(winreg.HKEY_CURRENT_USER, REG_PATH)
with winreg.OpenKey(winreg.HKEY_CURRENT_USER, REG_PATH, 0, winreg.KEY_WRITE) as registry_key:
winreg.SetValueEx(registry_key, name, 0, winreg.REG_DWORD, value)
winreg.CloseKey(registry_key)
return True
except WindowsError:
return False
for i in range(1,5): set_reg(r"Software\Microsoft\Windows\CurrentVersion\Internet Settings\Zones\\" + str(i),'2500', 3)
Adapted from the top answer in this thread by Mash and icc97:
python script to read and write a path to registry
Related
I am trying to run this script
import crawler
crawler.crawl(url="https://www.montratec.com",output_dir="crawling_test",method="rendered-all")
from this library:
https://github.com/SimFin/pdf-crawler
but I am getting this error:
Expected browser binary location, but unable to find binary in default location, no 'moz:firefoxOptions.binary' capability provided, and no binary flag set on the command line
I already have Firefox installed and I am using Windows.
If you have Firefox installed in a non-default location which is not in your system’s search path, you can specify a binary field on the moz:firefoxOptions capabilities object (documented in README), or use the --binary PATH flag passed to geckodriver when it starts.
Since Selenium is tagged, You can do the following changes to get rid of the above error :-
This is purely selenium solution, if you have a running instance of driver, you re-configure it using the FirefoxOptions like below :
options = webdriver.FirefoxOptions()
options.binary_location = r"C:\Program Files\Mozilla Firefox\firefox.exe"
driver = webdriver.Firefox(executable_path=r'\geckodriver.exe full path here', firefox_options=options)
driver.get("https://www.montratec.com1")
for crawler (Web scraping framework based on py3 asyncio & aiohttp libraries.)
Installation :
pip install crawler
Sample code :
import re
from itertools import islice
from crawler import Crawler, Request
RE_TITLE = re.compile(r'<title>([^<]+)</title>', re.S | re.I)
class TestCrawler(Crawler):
def task_generator(self):
for host in islice(open('var/domains.txt'), 100):
host = host.strip()
if host:
yield Request('http://%s/' % host, tag='page')
def handler_page(self, req, res):
print('Result of request to {}'.format(req.url))
try:
title = RE_TITLE.search(res.body).group(1)
except AttributeError:
title = 'N/A'
print('Title: {}'.format(title))
bot = TestCrawler(concurrency=10)
bot.run()
Official reference here
I am attempting to automate the push of a backup file from a (local) headless data server, to a (remote) secure backup server - which [for reasons] (currently) needs to be done via a web page.
I have written a Selenium/Python script which works well, but ignores ALL certificate errors - which, in this case, is NOT the desired behaviour.
Platform: Raspberry Pi 3B / Raspbian 10
Selenium: v3.141.0 (from Raspbian repo)
Geckdriver: v0.29 (built from github release source with rust v1.50)
...Installed with: cp /path/to/geckodriver-0.29.0/target/debug/geckodriver /usr/local/bin
Firefox: v78.7.0esr (from Raspbian repo)
Most people (in my searches) seem to have the exact /opposite/ problem - IE. they want to DISable the security mechanism ...But inverting the (literally True/False) logic in those many different solutions has NOT led me to a solution which allows me to ENable cert checking.
Here is some (abbreviated and annotated) example code, with one of my (many) attempts:
#!/bin/python3
from selenium import webdriver # selenium instance
from selenium.webdriver.firefox.options import Options # headless
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities # Reject bad SSL
# Set headless mode
opts = Options()
opts.headless = True
# Enforce certificate checking - [this fails]
caps = DesiredCapabilities.FIREFOX
print(caps) # {'browserName': 'firefox', 'marionette': True, 'acceptInsecureCerts': True}
caps['acceptInsecureCerts'] = False
print(caps) # {'browserName': 'firefox', 'marionette': True, 'acceptInsecureCerts': False}
driver = webdriver.Firefox(capabilities=caps, options=opts)
# LOG FILE ENTRY APPEARS in geckodriver.log:
# "Marionette WARN TLS certificate errors will be ignored for this session"
driver.get("https://wrong.host.badssl.com")
# <<perform upload here>>
driver.save_screenshot("test.png") # Page was NOT blocked! :(
driver.close()
My question is:
What modification(s) can be made to this code such that the <<upload>> will happen if (and only if) the SSL/TLS certificates are valid?
Extra kudos for an additional line of code to display a reason {"self-signed cert", "cert revoked", etc} if the <<upload>> is blocked.
from selenium import webdriver
capabilities = webdriver.DesiredCapabilities().FIREFOX
capabilities['acceptInsecureCerts'] = False
capabilities['marionette'] = True
driver = webdriver.Firefox(desired_capabilities=capabilities)
# LOG FILE ENTRY APPEARS in geckodriver.log:
# "Marionette WARN TLS certificate errors will be ignored for this session"
driver.get("https://wrong.host.badssl.com")
You should use desiredcapability , use the above code
Following solutions (actually it is only one) doesn't work to me :
How to get a name of default browser using python
How to get name of the default browser in windows using python?
Solution was:
from _winreg import HKEY_CURRENT_USER, OpenKey, QueryValue
# In Py3, this module is called winreg without the underscore
with OpenKey(HKEY_CURRENT_USER,
r"Software\Classes\http\shell\open\command") as key:
cmd = QueryValue(key, None)
But unfortunately, in Windows 10 Pro I don't have targeted registry value. I've tried to find alternative keys in Regedit, but no luck.
Please take a look, what my registry virtually contains:
The following works for me on Windows 10 pro:
from winreg import HKEY_CURRENT_USER, OpenKey, QueryValueEx
reg_path = r'Software\Microsoft\Windows\Shell\Associations\UrlAssociations\https\UserChoice'
with OpenKey(HKEY_CURRENT_USER, reg_path) as key:
print(QueryValueEx(key, 'ProgId'))
Result (first with Chrome set as default, then with IE):
$ python test.py
('ChromeHTML', 1)
$ python test.py
('IE.HTTPS', 1)
Please check for the key in windows 10
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\Shell\Associations\URLAssociations(http|https)\UserChoice
def get_windows_default_browser_launch():
""" On windows, return the default browser for 'https' urls
returns: example '"C:\Program Files\Mozilla Firefox\firefox.exe" -osint -url "%1"'
"""
import winreg
key = winreg.OpenKey(winreg.ConnectRegistry(None, winreg.HKEY_CURRENT_USER), r"Software\Microsoft\Windows\Shell\Associations\UrlAssociations\https\UserChoice")
prog_id, _ = winreg.QueryValueEx(key, "ProgId")
key = winreg.OpenKey(winreg.ConnectRegistry(None, winreg.HKEY_LOCAL_MACHINE), r"SOFTWARE\Classes\{}\shell\open\command".format(prog_id))
launch_string, _ = winreg.QueryValueEx(key, "") # read the default value
return launch_string
Windows 10 Python3 , may want to change the key for 'http' not https, but this is my code verbatim as my context is of a secured server. I wanted the browser binary name and path, which is just one more line.
I had try with python with webbrowser.open, but it only work on IE. How to let it open chrome or firefox. I don't want it to open on IE, i wants to be open on Chrome or Firefox. Due to i try many method, but none of them works.
import time
import webbrowser
webbrowser.open('www.google.com')
you need specify your webbrowser's name, detal see webbrowser.get
import webbrowser
webbrowser.open('www.google.com')
a = webbrowser.get('firefox')
a.open('www.google.com') # True
UPDATE
If you have chrome or firefox installed in your computer, do as following:
chrome_path =r'C:\Users\Administrator\AppData\Local\Google\Chrome\Application\chrome.exe' # change to your chrome.exe path
# webbrowser is just call subprocess.Popen, so make sure this work in your cmd firstly
# C:\Users\Administrator>C:\Users\Administrator\AppData\Local\Google\Chrome\Application\chrome.exe www.google.com
# there two way solve your problem
# you have change \ to / in windows
# this seems a bug in browser = shlex.split(browser) in windows
# ['C:UsersAdministratorAppDataLocalGoogleChromeApplicationchrome.exe', '%s']
a = webbrowser.get(r'C:/Users/Administrator/AppData/Local/Google/Chrome/Application/chrome.exe %s')
a.open('www.google.com') #True
# or by register
webbrowser.register('chrome', None,webbrowser.BackgroundBrowser(r'C:\Users\Administrator\AppData\Local\Google\Chrome\Application\chrome.exe'))
a = webbrowser.get('chrome')
a.open('www.google.com') #True
else you can try selenium, it provide much more functions and only need chromedriver.
I'm experimenting with creating a basic library extension for Robot Framework using Python, and I'm using PyCharm as the editor. For libraries imported directly code completion is working fine, but in this case I'm importing the Selenium2Library indirectly via a method:
def get_current_browser():
browser = BuiltIn().get_library_instance('Selenium2Library')._current_browser()
return browser
Which I call from other methods with something like
driver = get_current_browser()
This successfully grabs the webdriver browser instance from Robot Framework and lets me do as I please, but I don't get code hints when I go to edit a 'driver' variable. Is there way I can get hints in this scenario?
Here's the code in full:
from robot.libraries.BuiltIn import BuiltIn
from Selenium2Library.keywords.keywordgroup import KeywordGroup
import logging
def get_current_browser():
browser = BuiltIn().get_library_instance('Selenium2Library')._current_browser()
return browser
class MyLibrary(KeywordGroup):
def get_title_via_python(self):
driver = get_current_browser()
title = driver.title
logging.warn("checking title %s" % title)
return title
Try adding a docstring to your function to help PyCharm.
from selenium.webdriver import Remote # Remote imported only for code completion
def get_current_browser():
"""
:rtype: Remote
"""
browser = BuiltIn().get_library_instance('Selenium2Library')._current_browser()
return browser
More at http://www.jetbrains.com/pycharm/webhelp/type-hinting-in-pycharm.html