I got a website which is just compatible with Internet Explorer.
We activated the Edge Internet Explorer Mode Option, but im unable to handle the website with Selenium. Is there any way to use IE-Mode with Edge in Selenium?
You need to download the recommended version of IE Driver Server from this link then refer to the code below to use Edge IE mode in Selenium in Python:
from selenium import webdriver
ieOptions = webdriver.IeOptions()
ieOptions.add_additional_option("ie.edgechromium", True)
ieOptions.add_additional_option("ie.edgepath",'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe')
driver = webdriver.Ie(executable_path='E:\webdriver\IEDriverServer.exe', options=ieOptions)
driver.maximize_window()
driver.get('https://www.google.com/')
Note: Change the paths in the code to your owns.
Result:
at the moment there is no Edge browser IE Mode option for Python
but there is an option in C#
if you are familiar with C# you can follow steps below
Download the latest version of IEDriverServer from the Selenium site.
Create a C# console project using Visual Studio.
Install Selenium.WebDriver 3.141.0 NuGet package from Nuget package manager.
Add the code below to the project and modify the paths.
static void Main(string[] args)
{
var dir = "{FULL_PATH_TO_IEDRIVERSERVER}";
var driver = "IEDriverServer.exe";
if (!Directory.Exists(dir) || !File.Exists(Path.Combine(dir, driver)))
{
Console.WriteLine("Failed to find {0} in {1} folder.", dir, driver);
return;
}
var ieService = InternetExplorerDriverService.CreateDefaultService(dir, driver);
var ieOptions = new InternetExplorerOptions{};
ieOptions.AddAdditionalCapability("ie.edgechromium", true);
ieOptions.AddAdditionalCapability("ie.edgepath", "{FULL_PATH_TO_MSEDGE.EXE}");
var webdriver = new InternetExplorerDriver(ieService, ieOptions, TimeSpan.FromSeconds(30));
webdriver.Url = "http://Your_Site_URL_here...";
}
Related
I currently have a working python script leveraging Selenium driver for Edge browser. I need this script to run as job but due to security setting SQL Agent and Windows agent cant open the browser to scrape webpage. I would need script to run in silent mode without opening the edge browser window.
This article seems to be what I need but it is chrome.
https://stackoverflow.com/questions/62137334/disable-console-output-of-webdriver-using-selenium-in-python
I dont post script because the connections and data in script have connections to a private company intranet site.
Use:
var options = new EdgeOptions();
options.AddArguments("--headless");
You can also hide the driver console window:
var options = new EdgeOptions();
options.AddArguments("--headless");
options.AddArgument("--no-default-browser-check");
var chromeDriverService = EdgeDriverService.CreateDefaultService();
chromeDriverService.HideCommandPromptWindow = true;
var driver = new EdgeDriver(chromeDriverService, options);
To save PDF by using CHrome Browser does not cause any issues (I'm using these options):
options.add_experimental_option('prefs',{
'credentials_enable_service': False,
'plugins':{
'always_open_pdf_externally': True
},
'profile': {
'password_manager_enabled': False,
},
'download': {
'prompt_for_download': False,
'directory_upgrade': True,
'default_directory': ''
}
})
BUT .... How to save PDF by using webdriver.Ie() Internet Explorer Driver with Python + Selenium?
P.S. AFAIK Internet explorer can not be executed by using headless mode, but if someone will not the way to do it, will be amazing !!!
You can't use Selenium to deal with the download prompt in IE because that's an OS-level prompt. Selenium WebDriver has no capability to automate OS-level prompt window. You need to use some 3rd party tools to help you to download file in IE using Selenium.
Here I use Wget to bypass the download prompt and download file in IE. You can refer to this article about how to use Wget.
About using headless mode in IE in Selenium, you can also use a 3rd party tool called headless_ie_selenium. You can download this tool and use headless_ie_selenium.exe instead of IEDriverServer.exe to automate IE.
The sample code to download a pdf file is like below, please note to change the paths in the code to your owns:
from selenium import webdriver
import time
import os
url = "https://file-examples.com/index.php/sample-documents-download/sample-pdf-download/"
driver = webdriver.Ie('D:\\headless-selenium-for-win-v1-4\\headless_ie_selenium.exe')
driver.get(url)
time.sleep(3)
link = driver.find_elements_by_class_name("download-button")[0]
hrefurl = link.get_attribute("href")
os.system('cmd /c C:\\Wget\\wget.exe -P D:\\Download --no-check-certificate ' + hrefurl)
print("*******************")
I would like to capture network traffic by using Selenium Webdriver on Python. Therefore, I must use a proxy (like BrowserMobProxy)
When I use webdriver.Chrome:
from browsermobproxy import Server
server = Server("~/browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
co = webdriver.ChromeOptions()
co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port))
driver = webdriver.Chrome(executable_path = "~/chromedriver", chrome_options=co)
proxy.new_har
driver.get(url)
proxy.har # returns a HAR
for ent in proxy.har['log']['entries']:
print ent['request']['url']
the webpage is loaded properly and all requests are available and accessible in the HAR file.
But when I use webdriver.Firefox:
# The same as above
# ...
from selenium import webdriver
profile = webdriver.FirefoxProfile()
driver = webdriver.Firefox(firefox_profile=profile, proxy = proxy.selenium_proxy())
proxy.new_har
driver.get(url)
proxy.har # returns a HAR
for ent in proxy.har['log']['entries']:
print ent['request']['url']
The webpage cannot be loaded properly and the number of requests in the HAR file is smaller than the number of requests that should be.
Do you have any idea what the problem of proxy settings in the second code? How should I fix it to use webdriver.Firefox properly for my purpose?
Just stumbled across this project https://github.com/derekargueta/selenium-profiler. Spits out all network data for a URL. Shouldn't be hard to hack and integrate into whatever tests you're running.
Original source: https://www.openhub.net/p/selenium-profiler
For me, following code component works just fine.
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
I am sharing my solution, this would not write any logs to any file
but you can collect all sort of messages such as Errors,
Warnings, Logs, Info, Debug , CSS, XHR as well as Requests(traffic)
1. We are going to create Firefox profile so that we can enable option of
"Persist Logs" on Firefox (you can try it to enable on your default browser and see
if it launches with "Persist Logs" without creating firefox profile )
2. we need to modify the Firefox initialize code
where this line will do magic : options.AddArgument("--jsconsole");
so complete Selenium Firefox code would be, this will open Browser Console
everytime you execute your automation :
else if (browser.Equals(Constant.Firefox))
{
var profileManager = new FirefoxProfileManager();
FirefoxProfile profile = profileManager.GetProfile("ConsoleLogs");
FirefoxDriverService service = FirefoxDriverService.CreateDefaultService(DrivePath);
service.FirefoxBinaryPath = DrivePath;
profile.SetPreference("security.sandbox.content.level", 5);
profile.SetPreference("dom.webnotifications.enabled", false);
profile.AcceptUntrustedCertificates = true;
FirefoxOptions options = new FirefoxOptions();
options.AddArgument("--jsconsole");
options.AcceptInsecureCertificates = true;
options.Profile = profile;
options.SetPreference("browser.popups.showPopupBlocker", false);
driver = new FirefoxDriver(service.FirefoxBinaryPath, options, TimeSpan.FromSeconds(100));
driver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(10);
}
3. Now you can write your logic since you have traffic/ logging window open so don't
go to next execution if test fails. That way Browser Console will keep your errors
messages and help you to troubleshoot further
Browser : Firefox v 61
How can you launch Browser Console for firefox:
1. open firefox (and give any URL )
2. Press Ctrl+Shift+J (or Cmd+Shift+J on a Mac)
Link : https://developer.mozilla.org/en-US/docs/Tools/Browser_Console
tl/dr: What am I doing wrong?
I'm trying to run selenium tests locally and be compatible with Browserstack platform. I use this code to connect locally:
wd = webdriver.Remote('http://xxxxxxxx#hub.browserstack.com:80/wd/hub', {'browser':'firefox'})
wd.get('http://google.com')
wd.get_screenshot_as_file('/tmp/googl.png')
wd.close()
I see a good screenshot in /tmp/.
Now I try to do the same with local Selenium:
$ java -jar /usr/share/java/selenium-server-standalone-2.44.0.jar &
The server starts nominally. I try creating a session with Firefox (30.0), it works correctly. (Default browser is Opera.)
Then I try to run Python code:
wd = webdriver.Remote('http://localhost:4444/wd/hub', {'browser':'firefox'})
wd.get('http://google.com')
wd.get_screenshot_as_file('/tmp/googl2.png')
wd.close()
Selenium opens Opera instead of Firefox.
I see this in Python console:
Message: <html>
<head>
<title>Error 500 org/json/JSONObject</title>
</head>
<body>
<h2>HTTP ERROR: 500</h2><pre>org/json/JSONObject</pre>
<p>RequestURI=/wd/hub/session</p>
<p><i><small>Powered by Jetty://</small></i></p>
Why does it open Opera instead of Firefox?
The problem is in this line:
wd = webdriver.Remote('http://localhost:4444/wd/hub', {'browser':'firefox'})
Changing browser to browserName will fix it. Use
wd = webdriver.Remote('http://localhost:4444/wd/hub', {'browserName':'firefox'})
instead.
Another solution (very close to the accepted answer) is to use predefined DesiredCapabilities constants:
from selenium import webdriver
capabilities = webdriver.DesiredCapabilities.FIREFOX.copy()
wd = webdriver.Remote('http://localhost:4444/wd/hub', desired_capabilities=capabilities)
In this case capabilities is a dict already containing browserName property set to firefox.
I've been grappling with using PhantomJS/Selenium/python-selenium to download a file to the filesystem.
I'm able to easily navigate through the DOM and click, hover etc. Downloading a file is, however, proving to be quite troublesome. I've tried a headless approach with Firefox and pyvirtualdisplay but that wasn't working well either and was unbelievably slow. I know That CasperJS allows for file downloads. Does anyone know how to integrate CasperJS with Python or how to utilize PhantomJS to download files. Much appreciated.
Despite this question is quite old, downloading files through PhantomJS is still a problem. But we can use PhantomJS to get download link and fetch all needed cookies such as csrf tokens and so on. And then we can use requests to download it actually:
import requests
from selenium import webdriver
driver = webdriver.PhantomJS()
driver.get('page_with_download_link')
download_link = driver.find_element_by_id('download_link')
session = requests.Session()
cookies = driver.get_cookies()
for cookie in cookies:
session.cookies.set(cookie['name'], cookie['value'])
response = session.get(download_link)
And now in response.content actual file content should appear. We can next write it with open or do whatever we want.
PhantomJS doesn't currently support file downloads. Relevant issues with workarounds:
File download
How to handle file save dialog box using Selenium webdriver and PhantomJS?
As far as I understand, you have at least 3 options:
switch to casperjs (and you should leave python here)
try with headless on xvfb
switch to normal non-headless browsers
Here are also some links that might help too:
Selenium Headless Automated Testing in Ubuntu
XWindows for Headless Selenium (with further links inside)
How to run browsers(chrome, IE and firefox) in headless mode?
Tutorial: How to use Headless Firefox for Scraping in Linux
My use case required a form submission to retrieve the file. I was able to accomplish this using the driver's execute_async_script() function.
js = '''
var callback = arguments[0];
var theForm = document.forms['theFormId'];
data = new FormData();
data.append('eventTarget', "''' + target + '''"); // this is the id of the file clicked
data.append('otherFormField', theForm.otherFormField.value);
var xhr = new XMLHttpRequest();
xhr.open('POST', theForm.action, true);
'''
for cookie in driver.get_cookies():
js += ' xhr.setRequestHeader("' + cookie['name'] + '", "' + cookie['value'] + '"); '
js += '''
xhr.onload = function () {
callback(this.responseText);
};
xhr.send(data);
'''
driver.set_script_timeout(30)
file = driver.execute_async_script(js)
Is not posible in that way. You can use other alternatives to download files like wget o curl.
Use firefox to find the right request and selenium to get the values for that and finally use out of to the box to download the file
curlCall=" curl 'http://www_sitex_org/descarga.jsf' -H '...allCurlRequest....' > file.xml"
subprocess.call(curlCall, shell=True)