I'm using Python Selenium to execute a few daily tasks overnight. Unfortunately, one of the websites I'm logging into is strangely unreliable. Therefore, I call that task repeatedly until it works. Each time I start over, I have to use a new Chrome window. As a result, there are sometimes several erroneous Chrome windows open in the morning.
I would to automatically close all these windows all the tasks are complete. However, I don't want to close all instances of Chrome because I often keep my normal Chrome windows open overnight. Is there a way to close just the instances of Chrome started by selenium even if they are not still attached to a running script?
I'm not sure how to start!
Related
Ever since I started using Python Selenium and windows task manager to automate a python script to scrape the web using Google chrome I am having this issue where even though I haven't opened Chrome it still shows in the task manager and uses 60-70% CPU. This is slowing my laptop significantly and I can't seem to solve it. If I end task Chrome then it fixes but randomly after some time chrome again pops up in task manager slowing my laptop. If anyone has a solution to this it would be a great help if you can help me.
I have created a python program which uses selenium and chromedriver. I cannot successfully run this script (or any others using selenium) from the TaskScheduler in any way. However, it runs perfectly fine and does all tasks I need when I run it from Spyder. It also runs perfectly while logged in when I call it via the command line.
What does the program do when working as intended:
Launches a chrome browser.
Automates clicks and page requests.
Downloads a file.
-does stuff with files that is irrelevant to this post-
What does the program do when called from the TaskScheduler:
Starts chrome but it does not appear (no visible browser, but task manager recognizes the chromedriver and chrome being kicked off and running persistently after the script is called)
All of my clicks are on elements by full xpath so I thought maybe the invisible browser wouldn't break it, but it does indeed fail, never fetching the file download.
Possibly relevant information:
My chromedriver is not on path, but is set via
driver = webdriver.Chrome(r'F:\chromedriver.exe') and this works absolutely fine when run by Spyder or command line.
Task Scheduler input
Action: Start a program
Program/Script: C:\ProgramData\Anaconda3\python.exe
Add arguments (optional): "C:\Users\[My_redacted_name]\.spyder-py3\[Client's_redacted_name]\[redacted_task].py"
What I know: The working directory as suggested in Python script not running in task scheduler does not fix anything.
Running from the command line
C:\ProgramData\Anaconda3\python.exe C:\Users\[My_redacted_name]\.spyder-py3\[Client's_redacted_name]\[redacted_task].py
yields the exact results as intended
No other programs I have made have had this kind of issue, and I have dozens of programs running via TaskScheduler with similar functionality to all the components OTHER than selenium / chromedriver.
I actually have two scripts using selenium that both encounter the same issue when running from command line. Their tasks are more or less the same, so solving ONE should solve the other, but it should be noted that the issue is not unique to a single script, but instead unique to scripts using selenium and running from task scheduler
I also see Selenium - Using Windows Task Scheduler vs. command line and am attempting to see if the single response with 0 votes could help, but I'm not sure if the issue is truly the same given it was for IE and on java.
My initial workaround for running the script on a set schedule was utilizing datetime variables and while True loops, then switched to apscheduler for convenience. Switching back to the workaround again ran the script flawlessly.
Reference code:
I have a python selenium script that uses chromedriver in headless mode to test links on a website. I'm running this script on MacOS High Sierra.
If I have Chrome open prior to running the script, I can continue to use the browser normally. Headless mode appears to isolate me from the testing. However, if Chrome is closed first, I cannot launch the browser while the selenium script is running.
Looking at Activity monitor in the first scenario, it appears to me that chromedriver creates a new process for Chrome to run. So I (perhaps incorrectly) assumed I should be able to launch the browser manually while the script is running.
What am I missing?
How do I launch Chrome while selenium is running on Mac?
Ah-HA!
This post I stumbled upon made it possible to open Chrome while selenium is running (although I don't understand why it works one way but not the other).
For most applications, launching with open -naF "App Name" or open -nF
/path/to/application.app should be sufficient (see Safari, Forklift,
Office apps, etc.).
-n opens a new instance
-a tells open to look in the list of registered apps (no need to specify the full path, but can go wrong if you have multiple versions
of an app. I sometimes see issues on my system with differentiating
between macOS apps and apps installed by Parallels OSes… I prefer to
specify the path directly.)
-F opens a fresh instance (ignores an app save state left over from previous instances)
I'm running a selenium test using Firefox-17, and it will randomly "freeze" - the window is visible but completely useless. The mouse cursor that usually shows up when you hover over a link is active over the entire firefox window, but cannot actually interact/click on the page or firefox's menus. This only happens on the Debian machine, and only with selenium. I use Firefox-28 for daily browsing, and I've never experienced any issues like this.
The code runs fine for several minutes, but then it always randomly freezes in the middle of requesting a new page. The process must then be force killed.
Things I've tried:
Using firefox-28 - still freezes at random
Running the same code on my Windows machine - this runs for hours with no problem
Hypothesis:
I'm running the tests with python's multiprocessing. (For debugging purposes I've only been using one master queue that feeds to a single driver instance.)
Could this freezing problem be related to the forking mechanism used by Linux for multiprocessing?
Maybe somehow related to the http://shallowsky.com/blog/linux/firefox-freeze-and-dbus.html - although I have no problems accessing the bbc podcast referenced in that link
I have other code that runs firefox with javascript disabled, and it hasn't had any issues on this Debian machine. Could this be something to do with Linux's javascript engine?
I finally figured out that this only occurs with javascript enabled in firefox. Fortunately, chromium doesn't have this javascript problem on the same debian system.
I have a python script that uses selenium RC; specifically webdriver.
I would love to have the script run as a post-commit hook, ideally through IE, Safari (if possible), Chrome & Firefox.
But I'm not sure what I'd need to do, since every time I run it on my local machine, a browser pops up.
I've heard of "saucelabs". Is it the best solution?
Yes, you need to have X server running because Selenium requires a running browser and browsers need some kind of X server in order to run.
Fortunately there are many types of X servers and one of them is Xvfb aka X virtual framebuffer that does all the operations in memory and thus requires no screen.
Wikipedia has also some nice examples.
Also there are lots of questions here on SO for keywords headless, xvfb and selenium. Choose wisely ;)
[Edit]: Here's a nice example.