Working on web scraping since approximately 2 hours, I am using python selenium webdriver in order to extract a plain text table.
The webpage I am working on displays a dynamic table with the first 10 entries. Next to the table there is a print button that, once clicked, opens a new window with all the table entries in plain text (that's what I try to get).
The problem is that there is also the mac print popup window that also opens, waiting for pressing the cancel button before I can actually interact with the plain text table window.
When running the commands below, the last command print_button.click() keeps running in my terminal until I press the cancel button in the printer popup window.
I saw a couple of potential solutions to programmatically press the cancel print button (using ActionChains commands for instance), but I would first need to stop my print_button.click() running.
How could I make the print_button.click() command to stop once the printer window opens?
Or maybe prevent firefox to open any system print windows?
Pseudo-code:
import pandas as pd
from selenium import webdriver
driver = webdriver.Firefox()
driver.implicitly_wait(10)
driver.get('my_URL')
print_button = driver.find_element_by_xpath("/html/body/div/my_element/span")
# this is the command that keeps running until I click the cancel print button
print_button.click()
Related
I'm opening Firefox browser through selenium(3.141.0) on python 3.9, and it will always start minimized, even though in code I passed function .maximise_window() after opening the browser. When its started this way, it wont execute code to maximise and it wont execute code to switch tabs, and it wont even do it if im trying to switch tabs manually with my mouse.
If I immediately click on the initiated firefox browser on my taskbar when the program is started, it will function normally, but it will open my browsers home tab, and execute my code in new tab. Thats why you may see in code a part that closes that first tab. When i dont do it and when it starts "faulty", the home tab wont be opened.
Im also using the lenght of tabs as an indicator if the browser initiated faulty, where I tried to put it into loop and make it restart till it opens with 2 tabs, but it just wont unless I manually click it.
The only solution so far I can think of is kinda "hacky"...using pyautogui to scan my taskbar after initializing browser and clicking it fast, but I dont really like the idea.
The code goes through my company data warehouse site and manipulates it to download data.
Update
Other hacky solution I found is starting 2 browsers. First won't work, but second will. Meaning that browser works normally IF there is another Firefox browser open at the time.
Snippets of code:
from selenium import webdriver
import time
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
class DWH_browser:
def __init__(self):
self.browser = webdriver.Firefox()
self.browser.maximize_window()
self.browser.get("www.letskeepthecompanysiteasecret.org")
self.len_windows = len(self.browser.window_handles)
print(len(self.browser.window_handles))#this next part is used to close the extra tab when browser
#started normally and opens an extra tab
if len(self.browser.window_handles) == 2:
self.browser.switch_to.window(self.browser.window_handles[0])
self.browser.close()
self.browser.switch_to.window(self.browser.window_handles[0])
self.a = ActionChains(self.browser)
time.sleep(5)
DWH = DWH_browser()#and i initiate it in the code "normally"
#This was the other code I tried using to initate the browser and restart till its in 2 tabs, but not working
# issue_lenght = 1
# while issue_lenght == 1:
# DWH = DWH_browser()
# issue_lenght = DWH.len_windows
# if DWH.len_windows == 1:
# DWH.browser.quit()
# print("RESTARTING BROWSER")
Summarizing your issues:
Always start minimized.
Passed function maximise_window() after opening the browser.
Won't execute code to switch tabs.
To get rid of all these issues you need to ensure that:
Selenium is upgraded to current levels Version 4.4.0.
GeckoDriver is updated to current GeckoDriver v0.31.0 level.
Firefox Browser is updated to current firefox=103.0.2.
Additionally, you won't be needing self.browser.maximize_window() as firefox by default opens in a maximized mode.
Finally, trying to switch tabs manually with my mouse is a big no as the program execution may get interupted.
I wrote a scraper in python which uses selenium, chrome and an added extension. The scraper works locally with the display visible but not when I try to run it in headless mode.
On the scraped website there is a button which automatically opens a new window (I cannot access it via URL in order to open it in a new tab).
click_button() # the button opens a new window with a confirmation button
driver.switch_to.window(driver.window_handles[1]) # switching to the confirmation window
click_confirm() # when confirm is clicked the window closes automatically
driver.switch_to.window(driver.window_handles[0]) # switching back to original window
Chrome doesn't work in headless when it has extensions installed so I'm forced to use pyvirtualdisplay.
The problem is that the Display of pyvirtualdisplay opens only one browser window, therefore the confirmation window never appears. How can I allow pyvirtualdisplay to open new windows and alternate between them?
Turned out that pyvirtualdisplay did open a window, however its size was 1px by 1px. Apparently setting up the webdriver window size the following way doesn't work properly with pyvirtualdisplay:
options = Options()
options.add_argument('--window-size="1980,1080"')
driver = webdriver.Chrome(executable_path=chrome_driver_path, options=options)
Setting it up like bellow solved my problem and the window was of a normal size:
driver = webdriver.Chrome(executable_path=chrome_driver_path)
driver.set_window_size(1980,1080)
I am having trouble to login using selenium windows Internet Explorer. I am attaching the popup dialog
I need to use IE. I have tried
driver.get('https://username:password#xyz/Secure/Local/console/mc_index.html') and it does not work in IE. So I need to handle this popup dialog box.
with above code chrome works but I have to use IE (version ~ 11). So please avoid any other browser question.
Here is the code I am using
driver.get("https://xyz/Secure/Local/console/mc_index.html")
driver.find_element_by_id('moreInfoContainer').click()
driver.implicitly_wait(5)
parent_h = driver.current_window_handle
# click on the link that opens a new window
print(" before Popup")
driver.find_element_by_id("overridelink").click()
print("After popup")
It only prints up-to following 2 line:
Starting Opening the Automation
before Popup
After popup dialog it does not execute next code. So regardless what I do after the line "driver.find_element_by_id("overridelink").click()" selenium does nothing. It does not even print a simple "hello" after that line.
After the popup dialog it does not return the control to selenium.
It prints following line after i hit the "cancel" button.
After popup
The issue here is selenium never get the control back after popup dialog shows.
I'm quite new in Python, and I'm creating a robot to export some reports from a system that is in Silverlight. Therefore, I can't use elements of the page to click/select/insert dates, etc.
Then, my solution was using pyautogui to move the mouse, click on the right buttons and save the files...
The process is:
-Select dates of the report
-Click on the "team" I need the report for
-Click on preview button (a new window opens here)
-Maximise this window which contains the report
-Click on 'Export' button (drop-down)
-Select 'Export to CSV'
And then, via code, get last downloaded file and move it into the folder where I save the reports.
The bottleneck on this is: sometimes, due to an unknown reason, 'Export' button is 20/'30 pixels aside. It means that the robot clicks in nowhere, and the code stops since there is no "last downloaded file" to handle.
I already noticed this second window (where the report opens) is not in Silverlight. It's simple HTML. I know I could use some methods in selenium to select the Export/Export to CSV option (similarly as I've done to login into this system). But my problem is: how do I refer to this new window?
The code I'm using refers to the main page using selenium.wedriver but I couldn't find a way to refer to the new window (that is closed after I downloaded the CSV. Then I click again in "Preview" and another window opens, and the process repeat hundreds of times)...
wsite='https://WEBSITE TO EXTRACT'
driver = webdriver.Ie('C:\\MyFolder\\IEDriverServer.exe')
driver.get(wsite) #opens the site
driver.find_element_by_id('userNameInput').send_keys('USERNAME')
driver.find_element_by_id ('passwordInput').send_keys('PASSWORD')
driver.find_element_by_id('submitButton').click()
Then, there are a lots of lines of code (where the mouse clicks where it's needed...
pyautogui.moveTo(1846,1000,intervaltoclick)#Preview button
pyautogui.click()
time.sleep(3)
hwnd = win32gui.GetForegroundWindow()
win32gui.ShowWindow(hwnd, win32con.SW_MAXIMIZE)
if i<1:
time.sleep(10+t)
else:
time.sleep(5+(t/5))
pyautogui.moveTo(566,52,intervaltoclick)#Export
pyautogui.click()
pyautogui.moveTo(603,205,intervaltoclick)#Save as CSV
pyautogui.click()
time.sleep(3)
pyautogui.moveTo(1270,1025,intervaltoclick)#close window
pyautogui.click()
Would someone have any idea on how can I do that?
You can try to keep the selenium object of IE browser that you used for login. You can keep it hidden if you don't want to show on a screen. You can than try to loop through the IE windows and switch to desired window.
Set<String> allWindowHandles = browser.getWindowHandles();
for(String handle : allWindowHandles)
{
System.out.println("Window handle - > " + handle);
browser.switchTo().window(handle);
System.out.println(browser.getTitle());
}
Here is a helpful link which shows the examples to switch the window.
Selenium WebDriver Switch Window Commands
The webpage I need to go to has a javascript popup message with an OK button that appears before the page can finish loading.
There's ways to get rid of the message by either clicking the button, pressing enter, pressing the x to close, or pressing ALT+F4. But all attempts to either click or press keys fails and just remains on the page with the popup.
I must be missing something.
Everything is current and installed, even the registry additions for the IE server executable.
Thanks
If this Java popup is generated by IE you could be able to handle it using Alert(driver).accept()
I have run into this issue a couple of times where the popup is not part of IE at all and this does not work. there are a couple different methods you can try to get around this:
try switching your browser to phantomJS (this is a headless browser that works with selenium and runs in the background so you will not be able to see what selenium does anymore but it often will eliminate popups and is good if you don't actually need to click the popup)
try using the mouse or keyboard packages to hit that button.