I am working with python and selenium to set up a webscraper. I used the ChromeOptions module to open the chrome browser with a specific chrome user I have created. User name is: run_scraper_run. After creating the user a desktop connection has been created. I checked the desktop connection for the profile directory and copied the respective directory path. When I run the script it opens a browser but it seems like it opens another instance of google chrome. There is neither my default account selectable nor the one I created for the scraper. It seems like a separate environment if I can say that in that case. Does anyone have an idea what could have gone wrong or how I can get the created user account to be used?
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
options = webdriver.ChromeOptions()
options.add_argument(r'--user-data-dir=C:\Users\test\AppData\Local\Google\Chrome\User Data\run_scraper_run')
options.add_argument('----profile-directory="run_scraper_run"')
ser = Service(r'C:\[PATH OF CHROMEDRIVER]')
driver = webdriver.Chrome(options = options, service = ser)
This is my actual environment:
But this opens instead despite the same path as in the desktop icon properties
I believe you have the wrong user data directory path.
The user data directory path that you provide is actually the Profile path.
The following change should fix the issue:
options.add_argument(r'--user-data-dir=C:\Users\test\AppData\Local\Google\Chrome\User Data')
options.add_argument('----profile-directory="run_scraper_run"')
P.S. Verify that the Profile Directory name is correct. You can do this by checking that by accessing chrome://version in the chrome browser having the desired profile.
Related
I'm trying to automate some tasks with Selenium and I need to interact with certain extensions, so I created a profile that would keep this extensions.
Since Chrome's new update, my script stopped working because it wasn't opening the right profile anymore.
My code:
def open_browser():
optionsinput = webdriver.ChromeOptions()
optionsinput.add_argument("user-data-dir=/Users/User1/Library/Application\ Support/Google/Chrome/Profile 13")
# Open the browser
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options = optionsinput)
driver.implicitly_wait(10) # seconds
return driver
When I try to open the extension, I find that the extension is not installed in the profile and I get this:
and further investigation by opening chrome://version/ shows that it's adding /Default to the user path:
Profile Path /Users/User1/Library/Application\ Support/Google/Chrome/Profile 13/Default
I'm out of ideas on how to fix it... I've created new profiles, tried to create a default folder within the profile that would hold the extension... nothing.
i need to launch with custom profile user data that sets already. I need to setup for gmail login. So first i launch manually by typing path and login. what i want is when i use selenium to launch that profile. it logon already. but, now its like a seperate profile when launch with selenium whereas custom-profile name is same
C:\Users\Pandora\AppData\Local\Google\Chrome\Application\chrome.exe
--profile-directory="custom-profile"
from selenium import webdriver
options = webdriver.ChromeOptions()
options.add_argument("user-data-dir=C:\\Users\\Pandora\\AppData\\Local\\Google\\Chrome\\User Data\\custom-profile")
driver = webdriver.Chrome(executable_path=r"d:/automation/chromedriver.exe", options=options)
driver.get("https://mail.google.com")
Shouldn't the argument be similar to below?
"--user-data-dir==C:\\Users\\Pandora\\AppData\\Local\\Google\\Chrome\\User Data\\custom-profile"
The user-data-dir argument should conclude at User Data and the desired profile is set using the profile-directory argument.
The default profile is profile 1.
Like so,
options.add_argument("user-data-dir=C:/Users/<username>/AppData/Local/Google/Chrome/User
Data")
options.add_argument("profile-directory=Profile 5")
Also note that Selenium won't run in a custom profile unless all other instances of chrome are closed
install_addon is a method on the Firefox webdriver, but I'm running tests remotely. How can I install an add-on to a driver like the following:
driver = webdriver.Remote(
command_executor=SELENIUM_URL,
desired_capabilities=webdriver.DesiredCapabilities.FIREFOX
)
? It looks like creating a webdriver.FirefoxProfile() and using add_extension() should work, but it's currently broken.
The use case is testing the effect of Firefox add-on manifest.json's all_frames property.
Аfter a long search and visits on this topic from google i found solution
Install addon in you Firefox profile from browser, save it and set him into webdriver options.
Firefox profile creating
Open you firefox browser
Go to about:profiles
Create a New Profile
Launch browser with created profile
Install addons what you need
Export profile
Open root directory of your firefox profile (see in in about:profiles in Root Directory section)
Copy folder into you project
Python
from selenium.webdriver import FirefoxOptions, FirefoxProfile, Remote
# init options object
options = FirefoxOptions()
# init profile object with path to you profile
profile = FirefoxProfile('{path_to_your_profile_folder}')
# set profile into options
options.profile = profile
# init you Remote browser with created options (commented arguments for example)
driver = Remote(
options=options,
# command_executor= ...,
# desired_capabilities=...
)
It`s works for me. The disadvantage of this solution is the presence of a folder with a profile that needs to be stored somewhere
Why am I doing this:
I need to automate a website that requires client-side SSL certificates. I understand this to be an option which cannot be specified using fp.set_preference(). I am not in control of the server I am connecting to thus I cannot change the security setup.
What have I tried
I have created a separate Firefox profile which has the required 'client-side password protected SSL certificates' set up, select one certificate automaticaly and some manual proxy settings (SOCKS 5). After much googling I have set my code as follows:
from selenium import webdriver
url = 'https://www.paininneck.co.uk'
fp = webdriver.FirefoxProfile(r"""C:\Users\
<user>\AppData\Local\Mozilla\Firefox\Profiles\<Firefox>""")
driver = webdriver.Firefox(fp)
driver.get(url)
The Problem:
The browser does open, however, it is still using the default profile. None of the settings I have changed in the other profile has copied across. The profile specified in my code is still working with selecting it through the Firefox UI.
I am hoping I missed something simple and all this time googling has not been in vain! I am reluctant to change to default settings, however after tweaking the default profile to see if the settings would copy over it is apparent that they don't and Selenium is making a clean copy each time.
Kind regards
Rich
Versions:
Python==3.6.1,
Selenium==3.4.3,
Firefox==53
gecko driver==v0.16.1
OS==Windows(Its for work dont judge me!)
Using Selenium 3.4.x, Python 3.6.1 along with geckodriver v0.16.1 & Mozilla Firefox 53.0, you can use the existing Firefox profile through the following steps:
Locate the Firefox Profile directory on your windows box. For e.g. my Firefox Profile "debanjan" was located at C:\\Users\\AtechM_03\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles by the name w8iy627a.debanjan.
Next, you have to specify the absolute path of the Firefox Profile directory when you initiate the webdriver.
Here is the working code which opens an existing Firefox Profile 'debanjan' on my Windows machine:
It is to be noted that the current Selenium-Python binding is unstable with geckodriver and looks to be Architecture specific. You can find the github discussion and merge here. So you may additionally need to pass the absolute path of the firefox binary while initializing the webdriver
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
profile = webdriver.FirefoxProfile('C:\\Users\\AtechM_03\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\w8iy627a.debanjan')
binary = FirefoxBinary('C:\\Program Files\\Mozilla Firefox\\firefox.exe')
driver = webdriver.Firefox(firefox_profile=profile, firefox_binary=binary, executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
url = 'https://www.paininneck.co.uk'
driver.get(url)
I am learning Selenium (using my local machine) and need to use it to access my accounts (FB, gmail, etc) and would like to use it without having to input my user id and password each time.
Is there a way to either:
1) Have whatever instructions are in my script apply to my existing chrome session
2) Somehow have the new window that loads remember my user name/password (this seems like using an existing chrome session, but im not sure how)
What I have so far :
(I am unsure if my profile should contain my saved credentials for websites like google and FB, but i don't see why not)
chrome_options = Options()
chrome_options.add_argument("--user-data-dir=/Users/username/Library/Application Support/Google/Chrome/Default")
driver = webdriver.Chrome(chrome_options=chrome_options)
This opens up a new window with some of my credentials, but crashes thereafter with error.
selenium.common.exceptions.WebDriverException: Message: unknown error: Chrome failed to start: exited abnormally
(Driver info: chromedriver=2.14.313457 (3d645c400edf2e2c500566c9aa096063e707c9cf),platform=Mac OS X 10.9.5 x86_64)
You can do it like this
browser = webdriver.Firefox()
browser.get("https:one_url.com")
time.sleep(1)
# Login to this one_url.com
username = browser.find_element_by_xpath("find username field")
password = browser.find_element_by_class_name("find password field")
username.send_keys("user")
password.send_keys("password")
login_attempt = browser.find_element_by_xpath("find submit button")
login_attempt.click()
time.sleep(5)
# enter into another web without login
browser.get("https://one_url/profile.com")
time.sleep(5)
# one more
browser.get("https://one_url/profile.com")
time.sleep(5)
In Java, we can do it by using ChromeOptions and Chrome Profile. In chrome navigate to chrome://version/ It will display profile path and Executable path.
As per my working on this, \Local\Google\Chrome\User Data\Profile 3 is displaying when navigating to chrome://version/ in standard chrome. In this profile, I navigated to StackOverflow and saved credentials. So used below code
Map<String, Object> prefs = new HashMap<String, Object>();
prefs.put("binary", "C:\\Program Files (x86)\\Google\\Chrome\\Application\\chrome.exe");
System.setProperty("webdriver.chrome.driver", "E:\\selenium_setups\\poi-3.12\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
options.setExperimentalOption("prefs", prefs);
options.addArguments("user-data-dir=C:\\Users\\murali\\AppData\\Local\\Google\\Chrome\\User Data\\Profile 3");
DesiredCapabilities capabilities = DesiredCapabilities.chrome();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
WebDriver driver = new ChromeDriver(capabilities);
//WebDriver driver = new ChromeDriver(options);
driver.get("http://stackoverflow.com/");
As per my understanding, I excepted the stackoverflow.com page displayed as logged in. But for the first time, I am not logged in. so cross-checked with chrome://version/ in chrome opened by driver, profile path is displayed as
\Local\Google\Chrome\User Data\Profile 3\Default. Then logged manually in that profile itself, which is opened by webdriver and executed gain by closing it.
Finally, the page is displayed as logged in. So it may be in java. I hope it will help you to try in python.
Thank You
if you using Mac, you should change to --user-data-dir path.
You can change your code;
chrome_options.add_argument("--user-data-dir=/Users/username/Library/Application Support/Google/Chrome")
Remove the "Default/" from the end of your path.