I want to automatically download a .ics file (a calendar file) from a website using selenium with python. The goal is to disable the pop-up window that firefox open when you download a file. To do this, I use the following code :
#I set my preferences
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)#not using default folder for downloading
profile.set_preference("browser.download.manager.showWhenStarting", False)#dont show downloading process
profile.set_preference("browser.download.dir", 'C:/Users/UserName/Documents/rpi/some folder')#set the directory for download
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', 'text/calendar')#tell it to automaticaly download a file
#using the profile to access firefox
browser = webdriver.Firefox(executable_path='geckodriver',firefox_profile=profile)
I first thought that it was a MIME type problem in the line browser.helperApps.neverAsk.saveToDisk, but after changing the MIME type it's still not working.
From here I have no idea of what is going wrong because my MIME type seems right according to all internet resources I have been able to find.
Perhaps it is a problem of settings or something I haven't noticed...
Anyway thank you for reading this, ask me if you need a bit more code.
Hello from what I understand, downloading the file prompts a alert box asking if you want to download it, have you tried,
browser.switchtoalert.accept?
Sorry i write in VBA, but you should get the idea,
Thanks
So I did find a solution by avoiding the problem : i created a new firefox profile > launched firefox with this profile (all of this is done in about:profiles) > went to the web site from wich I wanted to download the file > downloaded the file and checked the "always do this with this type of file" box > launch my programm with this custom profile using the line :
profile= webdriver.FirefoxProfile("C:/Users/user/AppData/Roaming/Mozilla/Firefox/Profiles/bqpa3bzv.nameofprofile")
This seems to work fine and might be an efficient way of doing this.
Well done for figuring it out, I know the
.switchtoalert.accept works in chromedriver
Let me know if you want any help with anything 👍👍
Related
I have multiple links scraped, and I want a code that creates a shortcut for each link and saves on the desktop. So whenever we click that link, It opens that particular URL on chrome. Can we do that using python ?
Take a look at the answer to this question. It solves both aspects of your problem:
finding the correct desktop folder using the winshell package
creating the link-file (*.url) by writing a text file with the following content
[InternetShortcut]
URL=https://stackoverflow.com/
Note that on my machine it asked whether to open the *.url file with my default internet browser or some other program when I double-clicked it.
I am currently trying to save a webpage as it appears on the website in html format. Approach I am using is prerssing Ctrl + S using autoit. On pressing that save as dialog box appears where I am asked to enter the name of the file to save. This is working fine. However, I want to save the file by pressing Ctrl + S instead of bring the dialog box in front. I read somewhere by using "set_preference" we can do that. CAn someone suggest how to set a preference. Below is the code I am using for Chrome broswer:
driver=Webdriver.Chrome()
driver.get('http://www.yahoo.com/')
autoit.send("{CTRL down}")
autoit.send("{CTRL down}")
autoit.send("{CTRL up}")
autoit.send("C:\\Users\\karanjuneja\\Downloads\\kj\\ABCD.mhtml")
autoit.send("{ENTER}")
Currently I am using the aboved code, however I want that on pressing Ctrl + S it saves the file in the desired location.
Thanks
Karan
Selenium isn't the designed for this, you could either:
Use getHtmlSource and parse the resulting HTML for references to external files, which you can then download and store outside of Selenium.
Use something other than Selenium to download and store an offline version of a website - I'm sure there are plenty of tools that could do this if you do a search. For example WGet can perform a recursive download (http://en.wikipedia.org/wiki/Wget#Recursive_download)
Is there any reason you want to use Selenium? Is this part of your testing strategy or are you just wanting to find a tool that will create an offline copy of a page?
I am trying to use Selenium in Python to download a file from a website. In order to do that, I have read that I need to change the settings in my Firefox Profile to avoid opening the download dialogue window. I provided sample code below. This code works absolutely great at home, but it does not function properly with my work PC. I am suspecting that somehow Python can not change the settings of the firefox profile, even though the code below does not throw an error but rather works fine and in the end opens the download dialogue window.
from selenium import webdriver
import os
profile = webdriver.FirefoxProfile("C:\\Users\\Ric\\Documents\\Python Scripts\\FirefoxProfileCopies\\ric.copy")
profile.set_preference('browser.download.folderList', 2)
profile.set_preference('browser.download.manager.showWhenStarting', False)
profile.set_preference('browser.download.dir', os.getcwd())
profile.set_preference('browser.helperApps.neverAsk.saveToDisk',('application/vnd.ms-excel'))
browser = webdriver.Firefox(profile)
browser.get("http://www.sample-videos.com/download-sample-xls.php")
elem1 = browser.find_element_by_css_selector(".push-form > table:nth-child(2) > tbody:nth-child(2) > tr:nth-child(4) > td:nth-child(4) > a:nth-child(1)")
elem1.click()
This code works perfectly with my Firefox and its profile at home, but not with my computer at work. Does anybody know why this might be? Thank you in advance.
EDIT
I tried to add all the MIMEtypes from the Microsoft webpage, but still, the download manager window opens. When stopping the code to execute before opening the download link and trying to look at the settings for the used firefox profile with about:configthe following values are displayed:
So, after a lot of trying, I figured to look at the firefox settings in Firefox again, since it worked with an empty profile. I managed to resolve my issue and finally have the download window disappear by going to firefox, settings and changing the settings for applications:
Then, when opening this menu, search for excel and change the values from "asking every time" to "save file/download file". Sorry if these entries in the list differ from the actual ones in firefox but my Firefox is in German. After doing this, my issue was resolved. I hope it resolves somebody else :) and thanks to anderson.
I'm trying to use the Selenium module in python to generate a text list from one website, save it in a directory, and browse to that text list on another site to submit it.
I'm working on the script in two parts- 1. Get metadata and 2. Order data. I've successfully completed the script in part 1, except for the very last thing: Choosing to save the metadata file that was just generated. I left it alone to work on part 2, hoping I would stumble upon the answer, but I'm just reaching to same problem when the pop-up to choose file comes along.
In the documentation, I'm told that Selenium WebDriver has built-in support for handling popup dialog boxes and that after triggering a dialog box, if I call alert = driver.switch_to_alert() then I can "accept, dismiss, read its contents, or even type into a prompt."
However, it's not working. When I try alert.text('some text') or alert.send_keys(Keys.TAB), I keep getting the error NoAlertPresentException: Message: No alert is present and after adding the command to wait, I get the error TimeoutException: Message:
Are the popups I'm getting (screenshots attached) not recognized by Selenium? If so, how do I interact with them? It seems like using this to save and/or upload files is something that many people have to do, but I cannot find anything on Google. Specifically, I would like to choose 'Save File' then 'OK' for the first image and for the second I would like to browse to the file (i.e. enter the path into the file name field) and click 'Open.' I don't want to just change my Firefox settings to automatically save because this will eventually be run in a different environment, and that won't help solve my second problem.
Thanks!
EDIT:
I'm testing my script on windows but it will eventually be implemented on a linux cloud server. I thought I was going to have to switch to PhantomJS webdriver (which was probably going to make my problem worse) to do headless browsing but I found a way to keep firefox. I guess all this means is that I can't use AutoIT to fix my problem.
The popups you see are not regular popups that can be interacted with using switch_to. These popups are system dialogs and cannot be automated using selenium.
Usually people avoid having these dialogs shown in the first place by tweaking browser preferences, e.g.:
downloading file using selenium
Access to file download dialog in Firefox
How to download a file using Selenium's WebDriver?
For uploading, usually you can find the appropriate input element and send keys to it with a path to the file:
How to upload file ( picture ) with selenium, python
How to upload files into file inputs? (python-selenium docs)
Let me know if your case cannot be solved by using the answers in the links I've attached.
As for your first, "download file automatically" problem, you just need to set a correct content-type:
profile.set_preference('browser.helperApps.neverAsk.saveToDisk', "application/xml,text/xml")
Second problem fix (upload part):
driver.find_element_by_name("input_product_list").send_keys(textpath)
driver.find_element_by_name('include_sr').click()
driver.find_element_by_id('submit').click()
Extremely simple implementation using AutoIT.
Below scripts and steps can hely you to click on 'Save>OK' on this window's pop-UP
Step 1: Download AutoIT package/tool here AutoIt You may select ZIP format (extract it)
Step 2: Open any text editor (say notepad) and copy below code and save with extension .au3
(e.g file.au3)
WinWait("[TITLE:Opening ; CLASS:MozillaDialogClass]","", 10)
If WinExists("[TITLE:Opening ; CLASS:MozillaDialogClass]") Then
WinActivate("[TITLE:Opening ; CLASS:MozillaDialogClass]")
Send("{DOWN}")
Sleep(20)
Send("{TAB}")
Sleep(20)
Send("{TAB}")
Sleep(20)
Send("{ENTER}")
EndIf
Step 3: From extracted zip (Step 1) look for folder named: Aut2Exe and open it
Step 4: Click Aut2exe_x64.exe if your OS is 64 bit otherwise click Aut2exe.exe
Step 5: Browse/Locate file created in Step2. (file saved as extension .a3)
AND Choose Destination (.exe/.a3x) and select .exe option (say file.exe)
AND then Click convert
Step 6: include this file.exe in your project folder and use it as per your requirement using below code (as it is in Eclipse):
driver.dwonload().click(); // it can be something else as per your flow
Runtime.getRuntime().exec("C:/*path_to_your_EXE_file(selected in step 6))*/file.exe");
I am web scraping with selenium and whenever i try to download i file the firefox download/save as file pops up however, even If i apply profile.set_preference('browser.helperApps.neverAsk.saveToDisk', "application/csv"), it still doesnt work, I have tried everyt .csv related MIME but doesn't work, is it possible to either click save as radio button and then click ok on the dialog or disable it entirely.
you should do two things, first set these three preferences as follows (this is in Java but I guess you manage to translate that to python :-):
profile.setPreference("browser.download.dir", "c:/yourDownloadDir");
profile.setPreference("browser.download.folderList", 2);
profile.setPreference("browser.helperApps.neverAsk.saveToDisk", "application/csv, text/csv");
secondly, you should make sure the download file has the desired mime type. To do that, you can use the web developer tools and inspect the download.
EDIT:
To find out the MIME type open Chrome, press Ctrl+Shift+I (Cmd+Alt+I on Mac OS) change to the 'Network' tab and click your download link. You should see something like this:
Just an additional answer that might help someone, as comments to the accepted answer put me on the right track (thanks!). Another MIME type of CSV you might be dealing with is application/x-csv - that was my case and once I looked it up in the Network tab of the browser, I became a happier man :)
In C#
FirefoxOptions options = new FirefoxOptions();
options.SetPreference("browser.download.folderList", 2);
options.SetPreference("browser.download.manager.showWhenStarting", false);
options.SetPreference("browser.download.dir", "c:\\temp");
options.SetPreference("browser.helperApps.neverAsk.saveToDisk", "text/csv");