With python I am trying to use the credentials provider to provide credentials when connecting to a website for automated GUI testing with selenium. I found the following page which explains how to do this, possibly for JAVA:
#Override
protected WebClient newWebClient() {
WebClient client = super.newWebClient();
DefaultCredentialsProvider provider = new DefaultCredentialsProvider();
provider.addCredentials("username","password");
client.setCredentialsProvider(provider);
return client;
}
I am trying to pythonize is, but I run into problems, and I do not find the appropriate class name fr the DefaultCredentialsProvider:
from selenium import webdriver as original_webdriver
class webdriver(original_webdriver):
def newWebClient(self):
client = super().newWebClient()
provider = DefaultCredentialsProvider()
provider.addCredentials("username","password")
client.setCredentialsProvider(provider)
return client
The error when running this script is:
File "C:/Users/adi0341/PycharmProjects/SeleniumTest/tester.py", line 12, in <module>
class webdriver(original_webdriver):
TypeError: module.__init__() takes at most 2 arguments (3 given)
How to fix it? Or how to do something similar as explained in that link? Maybe there is an altogether different approach to provide authentication in order to open a web page for selenium automated GUI-testing?
P.S: The authentication will be an essential part of the testing itself. Logging in as different users and check access rights...
Step 1
For this requirement use keyring
import keyring
keyring.set_password("https://my.sharepoint.come", "username", "password")
After this the credentials will be stored under credentials manager for automatic login, you can run this control /name Microsoft.CredentialManager command in a command prompt to get it:
Newly added credentials will appear under "Generic Credentials"
Further more even before you write the code you can test this manually.
Step 2
Once you are through with this, you need to set the preference of
Firefox to hold your url under
network.automatic-ntlm-auth.trusted-uris:
from selenium import webdriver
url = 'http://my.sharepoint.com'
fp = webdriver.FirefoxProfile()
fp.set_preference('network.automatic-ntlm-auth.trusted-uris',url)
driver = webdriver.Firefox(firefox_profile=fp)
driver.get(url)
Can you point which line is 12?
I am unsure about your line with super, as in python inheritance of constructors looks like in this example.
DefaultCredentialsProvider ship with Java HtmlUnit, not webdriver. So you cannot find it under webdriver (whether in Java webdriver nor Python webdriver nor etc webdriver)
check this reference : How do I handle authentication with the HtmlUnitDriver using Selenium WebDriver?
Can you check whether there is similar python counter part for DefaultCredentialsProvider. Otherwise, the answer is : there is no DefaultCredentialsProvider for you in python.
Perhaps you should look for other authentication solution, such as this:
Authentication with selenium (Python)
Related
I am trying to use Python to automate a number of work processes that primarily take place in an online content management system, and to access the CMS, I need to be logged into my work profile on Chrome. While I could sign in once Chromedriver is open, I would have to approve the login on my authenticator each time, somewhat defeating the purpose of automation.
I am using the following code for opening Chromedriver and attempting to load my regular Chrome profile. However, while the rest of the code works as intended, it does not load my profile and I am unsure why.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.add_argument("--start-maximized")
options.add_argument("--user-data-dir=C:\\Users\\Saul\\AppData\\Local\\Google\\Chrome\\User Data\\Default")
driver = webdriver.Chrome("C:\\Users\\Saul\\Downloads\\chromedriver_win32\\chromedriver.exe", options=options)
driver.get("https://myaccount.google.com/")
while(True):
pass
I do get the following error while running the code, but I don't believe this is affecting my issue.
C:\Users\Saul\PycharmProjects\OnlineAutomation\main.py:12: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome("C:\\Users\\Saul\\Downloads\\chromedriver_win32\\chromedriver.exe", options=options)
Your question is not related to the error you mentioned.
For the error - 'DeprecationWarning: executable_path has been deprecated, please pass in a Service object', you have to use Service:
from selenium.webdriver.chrome.service import Service
driver = webdriver.Chrome(service=Service("C:\\Users\\Saul\\Downloads\\chromedriver_win32\\chromedriver.exe"), options=options)
For the Chrome profile issue, you have to mention the profile path like below:
options.add_argument("--user-data-dir=C:\\Users\\Saul\\AppData\\Local\\Google\\Chrome\\User Data")
options.add_argument("--profile-directory=Default")
If your website has MFA enabled, you might be able to use SeleniumBase (a Python framework), which has methods for handling MFA Login:
📘📝 An example test with the BaseCase class. Runs with pytest or nosetests. (Learn more)
from seleniumbase import BaseCase
class TestMFALogin(BaseCase):
def test_mfa_login(self):
self.open("https://seleniumbase.io/realworld/login")
self.type("#username", "demo_user")
self.type("#password", "secret_pass")
self.enter_mfa_code("#totpcode", "GAXG2MTEOR3DMMDG") # 6-digit
self.assert_exact_text("Welcome!", "h1")
self.assert_element("img#image1")
self.click('a:contains("This Page")')
self.save_screenshot_to_logs()
📗📝 An example test with the sb pytest fixture. Runs with pytest.
def test_mfa_login(sb):
sb.open("https://seleniumbase.io/realworld/login")
sb.type("#username", "demo_user")
sb.type("#password", "secret_pass")
sb.enter_mfa_code("#totpcode", "GAXG2MTEOR3DMMDG") # 6-digit
sb.assert_exact_text("Welcome!", "h1")
sb.assert_element("img#image1")
sb.click('a:contains("This Page")')
sb.save_screenshot_to_logs()
📙📝 An example test with the SB Context Manager. Runs with pure python.
from seleniumbase import SB
with SB() as sb: # By default, browser="chrome" if not set.
sb.open("https://seleniumbase.github.io/realworld/login")
sb.type("#username", "demo_user")
sb.type("#password", "secret_pass")
sb.enter_mfa_code("#totpcode", "GAXG2MTEOR3DMMDG") # 6-digit
sb.assert_text("Welcome!", "h1")
sb.highlight("img#image1") # A fancier assert_element() call
sb.click('a:contains("This Page")') # Use :contains() on any tag
sb.click_link("Sign out") # Link must be "a" tag. Not "button".
sb.assert_element('a:contains("Sign in")')
sb.assert_exact_text("You have been signed out!", "#top_message")
There's a SeleniumBase pytest command-line option, --user-data-dir=DIR, which you can add to set a user data directory to use. Eg:
pytest --user-data-dir=DIR
So I am trying to login programatically (python) to https://www.datacamp.com/users/sign_in using my email & password.
I have tried 2 methods of login. One using requests library & another using selenium (code below). Both time facing [403] issue.
Could someone please help me login programatically to it ?
Thank you !
Using Requests library.
import requests; r = requests.get("https://www.datacamp.com/users/sign_in"); r (which gives <response [403]>)
Using Selenium webdriver.
driver = webdriver.Chrome(executable_path=driver_path, options=option)
driver.get("https://www.datacamp.com/users/sign_in")
driver.find_element_by_id("user_email") # there is supposed to be form element with id=user_email for inputting email
Implicit wait at least should have worked, like this:
from selenium import webdriver
driver = webdriver.Chrome(executable_path='/snap/bin/chromium.chromedriver')
driver.implicitly_wait(10)
url = "https://www.datacamp.com/users/sign_in"
driver.get(url)
driver.find_element_by_id("user_email").send_keys("test#dsfdfs.com")
driver.find_element_by_css_selector("#new_user>button[type=button]").click()
BUT
The real issue is the the site uses anti-scraping software.
If you open Console and go to request itself you'll see:
It means that the site blocks your connection even before you try to login.
Here is similar question with different solutions: Can a website detect when you are using Selenium with chromedriver?
Not all answers will work for you, try different approaches suggested.
With Firefox you'll have the same issue (I've already checked).
You have to add a wait after driver.get("https://www.datacamp.com/users/sign_in") before driver.find_element_by_id("user_email") to let the page loaded.
Try something like WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.ID, 'user_email')))
I want to use selenium to automatically log in a website(https://www.cypress.com/) and download some materials.
I successfully open the website using selenium. But when I use selenium to click the "Log in" button. It shows this:
Access Denied
Here is my code:
from time import sleep
from selenium import webdriver
class Cypress():
def extractData(self):
browser = webdriver.Chrome(executable_path=r"C:chromedriver.exe")
browser.get("https://www.cypress.com/")
sleep(5)
element = browser.find_element_by_link_text("Log in")
sleep(1)
element.click()
pass
if __name__ == "__main__":
a = Cypress()
a.extractData()
pass
Can anyone give me some idea?
The website is protected using Akamai CDN, services, or whatever is loaded there.
I took a quick glance and it seems like the Akamai service worker is up, but I don't see any sensor data protection, selenium is simply detected as webdriver (and plenty other things) and flagged, try to login using requests, or ask the website owner to give you API access for your project.
Akamai cookies are up, so surely the protection is too, the 301 you got is the bot protection stopping you from automating something on a protected endpoint.
Goal:
I want to run a Selenium Python script through BrowserMob-Proxy, which will capture and output a HAR file capture.
Problem:
I have a functional (very basic) Python script (shown below). When it is altered to utilize BrowserMob-Proxy to capture HAR however, it fails. Below I provide two different scripts that both fail, but for differing reasons (details provided after code snippets).
BrowserMob-Proxy Explanation:
As mentioned before, I am using both 0.6.0 AND 2.0-beta-8. The reasoning for this is that A) LightBody (lead designer of BMP) recently indicated that his most current release (2.0-beta-9) is not functional and advises users to use 2.0-beta-8 instead and B) from what I can tell from reading various site/stackoverflow information is that 0.6.0 (acquired through PIP) is used to make calls to the Client.py/Server.py, whereas 2.0-beta-8 is used to initiate the Server. To be honest, this confuses me. When importing BMP's Server however, it requires a batch (.bat) file to initiate the server, which is not provided in 0.6.0, but is with 2.0-beta-8...if anyone can shed some light on this area of confusion (I suspect it is the root of my problems described below), then I'd be most appreciative.
Software Specs:
Operating System: Windows 7 (64x) -- running in VirtualBox
Browser: FireFox (32.0.2)
Script Language: Python (2.7.8)
Automated Web Browser: Selenium (2.43.0) -- installed via PIP
BrowserMob-Proxy: 0.6.0 AND 2.0-beta-8 -- see explanation below
Selenium Script (this script works):
"""This script utilizes Selenium to obtain the Google homepage"""
from selenium import webdriver
driver = webdriver.Firefox() # Opens FireFox browser.
driver.get('https://google.com/') # Gets google.com and loads page in browser.
driver.quit() # Closes Firefox browser
This script succeeds in running and does not produce any errors. It is provided for illustrative purposes to indicate it works before adding BMP logic.
Script ALPHA with BMP (does not work):
"""Using the same functional Selenium script, produce ALPHA_HAR.har output"""
from browsermobproxy import Server
server = Server('C:\Users\Matt\Desktop\\browsermob-proxy-2.0-beta-8\\bin\\browsermob-proxy')
server.start()
proxy = server.create_proxy()
from selenium import webdriver
driver = webdriver.Firefox() # Opens FireFox browser.
proxy.new_har("ALPHA_HAR") # Creates a new HAR
driver.get("https://www.google.com/") # Gets google.com and loads page in browser.
proxy.har # Returns a HAR JSON blob
server.stop()
This code will succeed in running the script and will not produce any errors. However, when searching the entirety of my hard drive, I never succeed in locating ALPHA_HAR.har.
Script BETA with BMP (does not work):
"""Using the same functional Selenium script, produce BETA_HAR.har output"""
from browsermobproxy import Server
server = Server("C:\Users\Matt\Desktop\\browsermob-proxy-2.0-beta-8\\bin\\browsermob-proxy")
server.start()
proxy = server.create_proxy()
from selenium import webdriver
profile = webdriver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("BETA_HAR") # Creates a new HAR
driver.get("https://www.google.com/") # Gets google.com and loads page in browser.
proxy.har # Returns a HAR JSON blob
server.stop()
This code was taken from http://browsermob-proxy-py.readthedocs.org/en/latest/. When running the above code, FireFox will attempt to get google.com, but will never succeed in loading the page. Eventually it will time out without producing any errors. And BETA_HAR.har can't be found anywhere on my hard drive. I have also noticed that, when trying to use this browser to visit any other site, it will similarly fail to load (I suspect this is due to the proxy not being configured properly).
Try this:
from browsermobproxy import Server
from selenium import webdriver
import json
server = Server("path/to/browsermob-proxy")
server.start()
proxy = server.create_proxy()
profile = webdriver.FirefoxProfile()
profile.set_proxy(self.proxy.selenium_proxy())
driver = webdriver.Firefox(firefox_profile=profile)
proxy.new_har("http://stackoverflow.com", options={'captureHeaders': True})
driver.get("http://stackoverflow.com")
result = json.dumps(proxy.har, ensure_ascii=False)
print result
proxy.stop()
driver.quit()
I use phantomJS, here is an example of how to use it with python:
import browsermobproxy as mob
import json
from selenium import webdriver
BROWSERMOB_PROXY_PATH = '/usr/share/browsermob/bin/browsermob-proxy'
url = 'http://google.com'
s = mob.Server(BROWSERMOB_PROXY_PATH)
s.start()
proxy = s.create_proxy()
proxy_address = "--proxy=127.0.0.1:%s" % proxy.port
service_args = [ proxy_address, '--ignore-ssl-errors=yes', ] #so that i can do https connections
driver = webdriver.PhantomJS(service_args=service_args)
driver.set_window_size(1400, 1050)
proxy.new_har(url)
driver.get(url)
har_data = json.dumps(proxy.har, indent=4)
screenshot = driver.get_screenshot_as_png()
imgname = "google.png"
harname = "google.har"
save_img = open(imgname, 'a')
save_img.write(screenshot)
save_img.close()
save_har = open(harname, 'a')
save_har.write(har_data)
save_har.close()
driver.quit()
s.stop()
What worked for me was to downgrade java version to java11. I used jenv to install and manage multiple java versions.
When you do:
proxy.har
You need to parse that response, proxy.har is a JSON object, so if you need to generate a file, you need to do this:
myFile = open('BETA_HAR.har','w')
myFile.write( str(proxy.har) )
myFile.close()
Then you will find your .har
Finding your HAR file
Inherently, the HAR object generated by the proxy is just that: an object in memory. The reason you can't find it on your hard drive is because it's not being saved there unless you write it there yourself. This is a pretty simple operation, as the HAR is just JSON.
with open("harfile", "w") as harfile:
harfile.write(json.dumps(proxy.har))
Why does ALPHA not work?
When you start dumping your HAR file, you'll find that your HAR file is empty with the ALPHA script. This is because you are not adding the proxy to the settings for Firefox, meaning that it will just connect directly bypassing your proxy.
What about BETA?
This code is written correctly as far as connecting to the proxy, although personally I prefer adding the proxy to the capabilities and passing those through. The code for that is:
cap = webdriver.DesiredCapabilities.FIREFOX.copy()
proxy.add_to_capabilities(cap)
driver = webdriver.Firefox(capabilities=cap)
I would guess that your issue lies with the proxy itself. Check the bmp.log and/or server.log files in the location of the python script and see what it is saying if something is going wrong.
Another alternative is that selenium is reporting back that the webpage has loaded before it actually has finished getting all of the elements, and as such your proxy is shutting down too early. Try making the script wait a bit longer before shutting down the proxy, or running it interactively through the interpreter.
I am trying to write a selenium testing case using Python bindings. I am very new to this, so I am really at loss.
My problem: The test is to check basic functionality for wordpress blogs, so logging in is required. I am using unittest, and Im trying to set up user authentication in setUp(), so every time before new test case runs, each cases is logged in as admin.
I have done some researches and tried things without luck. Here is what I have so far.
def setUp(self):
prof_dir = "C:/Python27/python_tests/ff_profile"
firefox_profile = webdriver.FirefoxProfile(prof_dir)
firefox_profile.native_events_enables = True
self.driver = webdriver.Firefox(firefox_profile)
self.base_url = "https://blogaddress.com"
self.verificationErrors = []
self.driver.get(self.base_url + "/wp-login.php")
I have saved credentials in profile, so I would expect when webdriver reaches the login page is already filled with credentials. However, it does not do that, and I cannot find a way to set up credentials in setUp(). (I tried quick and dirty way of loggin a user in by sending keys directly but this is just not good)
Is there any good ideas on setting credentials in setUp()?
Thanks for your help.