When I try to use Cookies file (sqlite fomatted) on Linux (Ubuntu) and Windows I fall into troubles with decryption of 'encrypted_value'. Is there any chance to make Cookies file compatible with two systems?
Basically selenium driver use Cookies file for various purpouses, and everything works on Linux. Sometimes my action is needed, so I want to have this Cookies file on my desktop which works on Windows, but when I download it directly and copy-paste it to my profile directory, my chromedriver logs error:
[4708:4884:0604/082853.607:ERROR:os_crypt_win.cc(61)] Failed to decrypt: The parameter is incorrect. (0x57)
I assume that there is some problem with 'encrypted_value' column decryption but I am unable to work-around this issue.
I use selenium for python this is snippet where I create options for my webdriver:
def create_options_for_webdriver(session_directory):
print('Creating options for webdriver!')
options = Options()
options.add_argument("user-data-dir=my_userdir")
options.add_argument("user-agent=my_useragent")
options.add_argument('--disable-background-networking ')
options.add_argument('--disable-client-side-phishing-detection')
options.add_argument('--disable-default-apps')
options.add_argument('--disable-hang-monitor')
options.add_argument('--disable-popup-blocking')
options.add_argument('--disable-prompt-on-repost')
options.add_argument('--disable-sync')
options.add_argument('--disable-web-resources')
options.add_argument('--enable-automation')
options.add_argument('--enable-blink-features=ShadowDOMV0')
options.add_argument('--force-fieldtrials=SiteIsolationExtensions/Control')
options.add_argument('--ignore-certificate-errors')
options.add_argument('--no-first-run')
options.add_argument('--password-store=basic')
options.add_argument('--use-mock-keychain')
return options
before option creating I create minimal directory structure which looks like my_userdir/Default/, and I download Cookies file to Default folder.
... when I download it directly and copy-paste it to my profile directory, my chromedriver logs error:
[... ERROR:os_crypt_win.cc(61)] Failed to decrypt: The parameter is incorrect. (0x57)
It looks like it is not possible. Or maybe not possible they way you are trying to do it. It takes extra effort.
The question Decrypting Chrome's cookies on windows has a link to os_crypt_win.cc. os_crypt_win.cc uses DPAPI, which is the old WinCrypt gear. DPAPI ties encryption to a user's Windows login. DPAPI also places a MAC on the encrypted data. I believe the MAC is the reason for the message you are seeing: "The parameter is incorrect". DPAPI sees the MAC over the encrypted data is wrong, and it gives you the generic error message.
So if you truly want to use the Linux cookie on Windows, you will need to decrypt it using the Linux spec, and then re-encrypt it using the Windows spec.
If you are going to pursue it, then you may want to visit this BlackHat talk: Reversing dpapi and stealing windows secrets offline. It will allow you to encrypt the user's data on Linux for Windows.
Coming back with update!
It appears that, as #jww mentioned process is more complicated, but just a little :)
In order to make Cookies fully compatible with any OS some special treatment must be applied.
In my case I used pickle library to create compatible file. To achieve it, things needs to be done as follow:
from selenium.webdriver import Chrome
import pickle
driver = Chrome()
####here you do some job which generate cookies like FB login or whatever
input("Press any key to close session") #you cant simply close browser, in order to make it work browser have to be closed in console so the rest of script will execute
pickle.dump(driver.get_cookies(), open('cookies.pkl',"wb"))
driver.quit()
print("Session closed!")
This will create cookies.pkl file which can be accessed under any OS like that:
import pickle
from selenium import Chrome
driver = Chrome()
for cookie in pickle.load(open("cookies.pkl"."rb")):
driver.add_cookie(cookie)
### anything you want to execute
As I mentioned this kind of cookies file will work under any OS, sadly it forced me to use selenium, but better this than nothing :)
Related
I want to access Session Buddy using Python.
In my case "access" means to get all currently opened URLs from Chrome.
Session Buddy allows you to save all opened URLs into a .csv file.
To do so you need to "setup" a few things (simplified: press buttons) and then all URLs are downloaded to Chromes /Downloads directory.
I would like to fully automate this process though. This means python needs to access Session Buddy, initiate the download and then save the file to the directory you want it to.
I can't use requests or something though since an extension won't work using an URL. This is what the extension calls: chrome-extension://edacconmaakjimmfgnblocblbcdcpbko/main.html
In general, I don't necessarily want to use Session Buddy to get all the URLs, it just seems to be the easiest way..
So, in summary, I just want to ask: How can I automatically use Python to fetch all currently opened URLs in my Chrome Browser (using Session Buddy)?
I'm thankful for any kind of help.
You can use selenium webdriver and load .crx(extension) file to automate
chrome_options = Options()
chrome_options.add_extension('path_to_extension')
driver = webdriver.Chrome(executable_path=executable_path,chrome_options=chrome_options)
driver.get("http://stackoverflow.com")
driver.quit()
I would like to write a script (in python) which scans the machine.(assumption is system has Linux) and retrieve the list of browser's installed. Looking for suggestions to implement it. I am using selenium to open links
browser = webdriver.Firefox()
Here we have to mention Firefox to open link in Firefox browser. What if user don't have Firefox in machine (chrome installed)?
I had already searched but haven't got any result.
P.S: If system has windows/Mac OS
A better approach: use try/except block
try:
browser = webdriver.Firefox()
browser.get('url')
except (IOException, Exception):
pass
It will help in cases where driver is not able to find the browser or there is some problem while launching.
You can scan all .desktop files in /usr/share/applications looking for WebBrowser in Categories field.
I wrote a code in which multiple URL is opening in Mozilla browser. But all URL have required login details. To avoid login code in my script, I used 'Saved Password' tactics. Before running to script I opened all URL and fill credential and marked as saved password. Assuming when I will run script then It would not ask any login. Unfortunately this is not working. When I am running script then login is required at that time too.
Please suggest where I am missing.
Every time a selenium-powered browser is fired up - it starts with a complete clean session by default. What you save and configure in your actual browser manually would not automagically be applied there.
What you need to do is to let your driver know that you want to use an existing profile via FirefoxProfile. Here is a great HOWTO.
In short: locate the existing profile directory and point your FirefoxProfile instance to it:
profile = webdriver.FirefoxProfile('/path/to/profile/directory')
driver = webdriver.Firefox(profile)
I've got a crunchy scraping problem in my hands with a lot of javascript that creates session-dependent cookies and I am trying to bypass this question using selenium. I'm using the python driver (python-selenium, version 2.2.0-1 on debian). Without the Remote Driver obtained from selenium-server-standalone-2.39.0.jar the browser was starting but not working (it reported some profile issues). Using the Remote Driver everything is fine except that set_preference is not working (I need to define a browser_profile in order to be able to automatically save some files):
profile = webdriver.FirefoxProfile()
profile.set_preference('browser.download.folderList', 2) # custom location
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', '/tmp')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'application/pdf')
#browser = webdriver.Firefox()
browser = webdriver.Remote(
command_executor='http://127.0.0.1:4444/wd/hub',
desired_capabilities=DesiredCapabilities.FIREFOX, browser_profile=profile)
In this way not only the dialog with the save/open choice is opened but examining about:config I do not find the confs to have been set.
On this debian I've got Iceweasel 24.2.0 but I've also tried on a ubuntu with plain firefox with no luck.
Any idea about what could be my problem?
I realize this is a bit late, but I found what I believe is the actual answer to this question.
Turns out that
profile.set_preference("browser.download.manager.showWhenStarting",False)
attempts to set a preference that FF doesn't recognize. If you look in about:config at a normal profile, that setting isn't there. At least not as of FF 35. Maybe it was sometime in the past, but no more.
Anyhow, that is apparently enough to make the revised profile invalid, causing FF to reject the whole thing (or possibly causes the Selenium library to crash when passing the profile to FF) at:
browser = webdriver.Remote(..., browser_profile=profile)
Related issue:
https://code.google.com/p/selenium/issues/detail?id=7017
I found that removing that call to set_preference setting allowed the modified profile to take effect.
My tentative conclusion is that the modified profile must have only modifications that FF understands.
Could you just make the settings you need and permanently save the profile? You could then load this profile without the need for set_preference.
edit:
Also, see Python - Remote Webdriver with Extension installed in it
Your solution for this case here
This cause of the profile has not be adopted. We need to notify the firefox to update the new profile.
I am using a hmtl file as a help document for my program, and would really like to be able to open the file at a specific point. i assumed i would be able to do this using the built in webbrowser module by specifying a url with a bookmark.
this is my html file name: help.html
i assumed that i would be able to use: help.html#top
this is the code i am using to open the file, this works fine:
webbrowser.open("Files\help.html")
and this is the code i have been trying to use to open at a specific point which ie9 apparently cant display (not sure why it is trying to load in ie9 as chrome is my default browser, and the working one above loads in chrome):
webbrowser.open("Files\help.html#2.1.0")
any ideas guys?
webbrowser.open() calls the browser from the command line. So you might try to do that yourself first. If that doesn't work, it's likely that your browser just doesn't support that for local files or something.
With Ubuntu+Firefox for example, webbrowser.open() does what you ask. (but - as Dave Webb said in his answer - you do have to provide a file: url, not just a filename).
(not on Windows at the moment, so haven't checked there)
As for why it doesn't load chrome but ie9: (you can look in the webbrowser.py code yourself if you want) I think it does try to use your default webbrowser, by doing os.startfile(url). What happens when you doubleclick your help.html file, of when you just type help.html (adjust path as needed) at the command line? It should do the same.
EDIT:
It seems that it doesn't always use the command line. On Windows, when trying to use the default browser, it uses os.startfile() which in turn uses the win32 ShellExecute api. ShellExecute can be used to perform certain actions on a file, folder or URL, like "open", "edit" or "print" with its default application. In this case, ShellExecute is asked to "open" the URL.
It seems however, that ShellExecute ignores the fragment identifier (the part after #) when opening file: urls. Strangely enough, this is not the case with http:urls. Presumably, a file: url is just converted to a plain filename first.
There seems to be little you can do about this except:
write something that "does the right thing" yourself (and register it as a browser controller for the webbrowsermodule, and use webbrowser.get() to get your controller, see docs)
as many applications do: configure the browser you want to use (or make it possible for your users to do so). The easiest way would be to set the BROWSER environment variable (see webbrowser module docs)
serve the file via a localhost http server, and open the http url, which would then be something like "http://. "http://localhost:8000/help.html#2.1.0". (The SimpleHttpServer module might come in handy)
Or, the easiest way: as you seem to be on windows: just try to open internet explorer specifically:
try:
browser = webbrowser.get('c:\\Program Files\\Internet Explorer\\IEXPLORE.EXE')
except Webbrowser.Error:
browser = webbrowser.get()
browser.open(url)
(This will fall back to using the default, so your code would still work on other platforms)
I think webbrowser is expecting a URL, so have you tried something like:
webbrowser.open("file://c:/path/to/files/html.html#2.1.0")