I have a python-selenium script running on a debian 11 server without any Desktop - thats why i use pyvirtualdisplay. The script has to login to a website and then go to a page on this site that, on open, instantly downloads a xlsx file (no dialogue, direct download)
#!/usr/bin/env python
from pyvirtualdisplay import Display
from selenium import webdriver
display = Display(visible=0, size=(800, 600))
display.start()
browser = webdriver.Firefox()
profile = webdriver.FirefoxProfile()
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", '/updatescript')
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/x-gzip")
browser.get("https://example.com/login")
browser.find_element_by_class_name('name').send_keys("mymail#mysite.com")
browser.find_element_by_class_name('passwd').send_keys("password")
browser.find_element_by_class_name('btn-primary').click()
browser.get("https://example.com/download")
Note this isnt site specific but works that way on all similiar sites (only .part files are downloaded).
The second problem is that my download folder is ignored, the script always downloads in /tmp/mozilla_root0
What is the reason? Dialogue cant be as the downloads are always instantly
Related
I have read these QA :
Downloading file to specified location with Selenium and python
Access to file download dialog in Firefox
and this blog : Working with file download in Python Selenium WebDriver
I then executed this code to specify the right path for the download:
from selenium import webdriver
my_path = r'C\some_dir\where_I\want_to\store\the_downloads'
profile = webdriver.FirefoxProfile()
profile.set_preference("brower.download.foldeList", 2)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.download.dir", my_path)
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "text/csv/pdf")
driver = webdriver.Firefox(executable_path=driver_path, firefox_profile=profile)
My issue is that the documents are not saved in my_path but in the default download directory of firefox.
EDIT 1:
I am unable to change the browser.download.folderList parameter.
Here is my command:
profile.set_preference('brower.download.folderList', '2')
I also tried each of these:
profile.set_preference('brower.download.folderList', 2)
profile.set_preference("brower.download.folderList", '2')
profile.set_preference("brower.download.folderList", 2)
When I open the about:config page on the opened Firefow window, brower.download.folderList is set to 1.
Sorry for the mistake.
I wrote brower instead of browser...
With selenium I try to download something (in order to verify its content), using the following code as a proof-of-concept:
from selenium import webdriver
profile = webdriver.FirefoxProfile()
#Set Location to store files after downloading.
profile.set_preference("browser.download.dir", "/tmp")
profile.set_preference("browser.download.folderList", 2)
#Set Preference to not show file download confirmation dialogue using MIME types Of different file extension types.
#profile.set_preference("browser.helperApps.neverAsk.saveToDisk",
# "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet;")
profile.set_preference("browser.download.manager.showWhenStarting", False )
profile.set_preference("pdfjs.disabled", True )
profile.set_preference("browser.helperApps.neverAsk.saveToDisk","application/zip")
profile.set_preference("plugin.disable_full_page_plugin_for_types", "application/zip")
browser = webdriver.Firefox(profile)
browser.implicitly_wait(10)
browser.get('https://www.thinkbroadband.com/download')
time.sleep(15)
elem = browser.find_element_by_xpath('//a[#href="http://ipv4.download.thinkbroadband.com/5MB.zip"]')
elem.click()
time.sleep(15)
However, nothing 'happens' (i.e. the download is not performed), and also no error message is shown. When I click on that download link manually, the test-file is being downloaded into /tmp.
Is there anything I am missing?
The issue could be because the click in this case needs to go through a child element
elem = browser.find_element_by_xpath('//a[#href="http://ipv4.download.thinkbroadband.com/5MB.zip"]/img')
elem.click()
But otherwise when you click on a link browser checks stuff in background for that link, and the site seems to have a problem, when I open the target link in browser I get an empty response
I currently have a script that will log on to my company's wiki, visit a page, and select a download to pdf option available on the page. However, when this option is chosen, this dialogue box
pops up asking me to tell Firefox what to do with it. I just need selenium to interact and hit the "ok" button.
I'm not sure how to inspect this window for elements, and am need of direction. Any documentation helps.
from splinter import Browser
browser = Browser()
browser.visit('https://company.wiki.com')
browser.find_by_id('login-link').click()
browser.fill('os_username', 'user')
browser.fill('os_password', 'pass')
browser.find_by_name('login').click()
browser.visit('https://pageoncompany.wiki.com')
browser.find_by_xpath('//*[#id="navigation"]/ul/li[4]').click()
browser.find_by_id('action-export-pdf-link').click()
I was able to set the preferences through the web browser, then call my profile:
browser = Browser('firefox', profile=r'C:\Users\craab\AppData\Roaming\Mozilla\Firefox\Profiles\0lot9hun.default')
You can set preferences in order to prevent coming of download popup ad download it to pre-defined folder.
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.download.folderList", 2) # custom folder as set by repo
fp.set_preference("browser.download.manager.showWhenStarting", False)
fp.set_preference("browser.download.dir", <download_folder_path>)
fp.set_preference("browser.helperApps.neverAsk.saveToDisk", content_type)
# Enable auto download, Avoid popup during downloads
fp.set_preference("browser.download.panel.shown", False)
fp.set_preference("browser.helperApps.neverAsk.openFile", content_type)
driver = webdriver.Firefox(fp)
I´m really new to programming with Python.
I want to write a code that opens a URL in different Breakpoints (1280px / 768px / 320px) and then take screenshots from the site.
So far this is my code:
from selenium import webdriver
from pyvirtualdisplay import Display
import time
# Set Breakpoint for different View
Breakpoints = [1280, 768, 320]
"""
Open invisible Website in Breakpoint 1280
"""
display = Display(visible=0, size=(1280, 800))
display.start()
"""
Open the Headless Chrome Webdriver
"""
driver = webdriver.Chrome('/usr/local/bin/chromedriver')
driver.set_page_load_timeout(10)
driver.get('https://example.com')
driver.maximize_window()
"""
Take Screenshots and Save as File
"""
driver.get_screenshot_as_file('Screenshot_BrPt_1280.png')
time.sleep(1o)
driver.get_screenshot_as_file('Screenshot_BrPt_768.png')
time.sleep(1o)
driver.get_screenshot_as_file('Screenshot_BrPt_320.png')
My question is now:
How can i rewrite this code that he loops through the different Breakpoints, open the site in the different Breakpoints an take a screenshot of every breakpoint?
I am performing some file uploading tests. I found that my test code hangs at element.send_keys(file) if I am using PhantomJS, however the same code does not hang if I am using Firefox.
element = self.browser.find_element_by_xpath("//input[#type='file']")
element.send_keys(file)
Is there any workarounds to make PhantomJS upload files properly? Currently I am using Windows 7, Python 3.4.1, selenium 2.42.1, PhantomJS 1.9.7.
browser = webdriver.PhantomJS()
browser.set_window_size(1200,800)
Without setting the window size, the browser remains in mobile size causing errors. Try a implicit wait too.
Should use PhantomJS.uploadFile(). However, didn't find python selenium API.
var webPage = require('webpage');
var page = webPage.create();
page.uploadFile('input[name=image]', '/path/to/some/photo.jpg');
Oddly enough i couldn't get anything to run in my ubuntu shell but it would run via iPython from Jupyter notebook on the exact same server.
I had to add a virtual display into the code to make it run from the shell as a .py script...
if it helps anyone facing the similiar problem here is the lines of code i added to my script and the send keys start to work without an issue.
from pyvirtualdisplay import Display
# Set screen resolution to 1366 x 768 like most 15" laptops. This is needed
#to run in the shell. Seems fine in iPython
display = Display(visible=0, size=(1366, 768))
display.start()
I use this approach when I can not simply change the value of the file input tag.
We will run it console, so we should use PyVirtualDisplay with window manager (I am using dwm, you can try fluxbox it is easy to install but need more RAM then dwm) and Xephyr for debug.
To run window manager we will use EasyProcess
So we will call File Upload Dialog, by clicking on element or button, then we simulate send keys with pynput and all these running in our console.
import time
from easyprocess import EasyProcess
from pynput.keyboard import Key, Controller
from pyvirtualdisplay import Display
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
class ImageUploadAutomation:
chrome = 'path/to/chrome'
chrome_options = Options()
# chrome_options.add_argument('--headless')
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--disable-notifications")
chrome_options.add_argument("--disable-dev-shm-usage")
chrome_options.add_argument("--window-size=1280,900")
chrome_options.add_argument("--start-maximized")
chrome_options.add_argument("user-data-dir=selenium")
chrome_options.add_experimental_option(
"excludeSwitches", ["disable-popup-blocking"]
)
chrome_options.add_argument(
"--disable-blink-features=AutomationControlled"
)
driver = None
def upload(self, photo):
# Change visible to 1 if you want to use Xephyr debug
with Display(visible=0, size=(1280, 900)) as display:
with EasyProcess(["dwm"]) as process:
keyboard = Controller()
url = "https://www.exmaple.com"
self.driver = webdriver.Chrome(
executable_path=self.chrome,
chrome_options=self.chrome_options
)
self.driver.get(url)
test = self.driver.find_element_by_xpath(
"//div[#aria-label='Add Photos']"
)
time.sleep(1)
test.click()
time.sleep(1)
for key in photo.path:
keyboard.press(key)
keyboard.release(key)
# keyboard.press(Key.enter)
with keyboard.pressed(Key.enter):
pass