I have a Python program that uses APScheduler and Selenium to automate webscraping. Basically it scrapes a particular website once every hour, and then schedules certain more detailed scrapes to happen at intervals.
The issue is that while I want to start the scraper, and then be free to use my computer for other work, Selenium will then automatically focus on the opened chrome tabs whenever a new scrape is started by APScheduler. Because of this, I am trying to find a way to have the new chrome windows open - but I don't have to focus on them. I've already tried headless and phantomJS, but the site is dynamically generated so these don't really work.
My current solution is to open the new window, minimise it, and then immediately shift back to the old window. To do this I want to perform a keyboard shortcut using ActionChains. I currently have this test code:
driver = webdriver.Chrome(ChromeDriverManager().install())
ActionChains(driver).key_down(Keys.CONTROL)
ActionChains(driver).send_keys(Keys.RIGHT)
ActionChains(driver).key_up(Keys.CONTROL)
Currently this code does nothing however. Is there anyway to fix this? I am using Jupyter Notebook and I am on a mac for reference.
NOTE: this code is being run apart from the main program, I am just using it to test if the script will work.
Try this out. You never .perform() it.
ActionChains(driver).key_down(Keys.CONTROL).send_keys(Keys.RIGHT).key_up(Keys.CONTROL).perform()
Related
I have been using Selenium for a couple of years now. I learned a ton, thanks to all the amazing people here at SO. Now, I just encountered a new challenge/opportunity. It seems like Selenium always wants to open a new window, which is a new instance, as far as I can tell, and this new instance doesn't know that I am already logged into the system that I want to be logged into. Normally I would run some generic code like this.
import time
from selenium.webdriver import ActionChains
from selenium import webdriver
driver = webdriver.Chrome('C:\\Users\\chromedriver.exe')
driver.get ('https://corp-login_place')
driver.find_element_by_id('USERID').send_keys('myid')
driver.find_element_by_id ('PASSWORD').send_keys('mypw')
list = [145, 270, 207]
for i in list:
driver.find_element_by_id('InputKeys_LOCATION').send_keys(i)
driver.find_element_by_class('PSPUSHBUTTON').click()
button = driver.find_element_by_class_name("Button")
button.click()
time.sleep(5)
So, I think that automation script should work, but when I run the code, it always opens a new browser window, and asks me to login. At this point my login credential don't work. Very weird. Anyway, I'm already logged in, and everything works fine if I just use the mouse and keyboard. It seems like the open browser window, which is out of focus, works fine, but Selenium don't focus on ths open window, and it opens a new window, which doesn't allow me to login.
For some reason, the code doesn't work when I hit F9, but if I run the process manually, using the mouse and keyboard, everything works totally fine. I feel like
Any insight or illumination, as to what is happening here, would be greatly appreciated. Thanks to all!
What I got is that you have a manually opened browser window (which was not opened with Selenium) and you want to control this window with Selenium?
Apparently this is not supported, but there is some code here that maybe can help you.
Selenium always uses a new instance of the browser no matter how many other browser windows are open in your system. If your problem is related to logging into the website you want to test, then please share the website url.
Or else if you are talking about attaching a session of browser window already opened in your system then this article (http://tarunlalwani.com/post/reusing-existing-browser-session-selenium/) might help, albeit the whole purpose of automating is compromised.
ok; I wrote a code and use python selenium and then with pyqt5 made it graphical.
but when I push the button start webdriver opened and I can't access to my windows's code any more, It got frozen , but I should input some valuable or click on some button as the code running.
what should I do ?
I've faced the same problem before. My solution was to use the time.sleep(duration) when the web driver opened the Window. For your specific situation, I think that you can use javascript on the page to check the user input is complete or not. Once the input fields are filled then trigger the process behind.
What I did was using a Process, so it would seperate the GUI-Loop and everything what is being done by selenium in a Process. By doing so you'll be able to use your GUI parallel to running Selenium in the Background by your process.
When I try to run any kind of code using winium, it will open the app, but then won't execute any of the code afterwards. It's not as if it throws up an error, it just hangs there and won't move on.
I Am using Python 3.7 on a Windows 10 PC.
I have tried the two 'magic' examples that are listed on the github wiki page for Winium, but even that doesn't work. I am able to use selenium to do automated web testing, so I don't think the selenium module is the issue. I have tried importing the time module and making it sleep for 10 seconds in between lines but this has no effect on the outcome.
from selenium import webdriver
driver = webdriver.Remote(
command_executor='http://localhost:9999',
desired_capabilities={
"debugConnectToRunningApp": 'false',
"app": r"C:/windows/system32/calc.exe"
})
# THIS IS WHERE IT SEEMS TO PAUSE INDEFINITELY
window = driver.find_element_by_class_name('CalcFrame')
view_menu_item = window.find_element_by_id('MenuBar').find_element_by_name('View')
view_menu_item.click()
view_menu_item.find_element_by_name('Scientific').click()
view_menu_item.click()
view_menu_item.find_element_by_name('History').click()
window.find_element_by_id('132').click()
window.find_element_by_id('93').click()
window.find_element_by_id('134').click()
window.find_element_by_id('97').click()
window.find_element_by_id('138').click()
window.find_element_by_id('121').click()
driver.close()
I would expect it to press the corresponding buttons, but it doesn't seem to do anything except open the calculator app.
I think this example is written for an older version of calculator. In Windows 10, the "Scientific" button is under the Menu button.
You'll have to find the menu button, click it, and then look for the element "Scientific" in the list.
Also, the numeric values for your arithmatic case are not correct. Pick up a UI inspector tool (inspect.exe, uispy, etc...) to make sure you are targeting the elements correctly.
I am using webbroswer.open() in a loop to download multiple files at given intervals.
The issue I am having is that whenever the browser window opens, it becomes the primary window and thereby interrupts and disrupts my ability to use the computer. Downloading multiple files means this can last some time. The broswer continuously flashing open is obviously jarring.
Is there any way to instruct webbrowser to open the browser minimised by default or otherwise avoid this issue in some other ingenious way?
Much appreciated!
If you are open to using other modules I would urge you to look into selenium. This allows you to do many things, and one of them is to launch in headless mode (so as not to disturb you as it loads pages). The documentation is at:
https://selenium-python.readthedocs.io/
And you would be interested in the headless option
You would be advised though to make sure your script works without this enabled before you enable it though.
Sample code:
import selenium
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
my_options = Options()
my_options.headless = True # set to False for debugging!!
browser = webdriver.Chrome(options=my_options)
browser.get('http://www.google.com')
print('Done.')
You will need to download the proper drivers (just follow the instructions on the link I posted) for whatever browser you'd like. I picked Chrome, but they have Edge, Firefox, and Safari browsers as well!
I'm using Python with Selenium 2.44. When the test fails, I can't just uncomment all the code before the failure when debugging it, because the driver will not be declared for the browser. Therefore, whenever I try fixing something, I always have to open a new browser in the test case. This is rather... slow since I have to login, which adds an additional 30 seconds (not devastating, but annoying). I want to know if there's a way for me to just continue a session, or do something that allows me to start the test midway through (so if I have the webpage open already, I can just immediately start clicking things rather than opening a new browser). Is this possible?
For example, if I had something along the lines of:
driver = webdriver.Firefox()
driver.get("google.com")
driver.find_element_by_xpath("//input[#id='gbqfq']").send_keys("cats" + Keys.RETURN)
This should open Firefox, go to google, and search for cats. Pretend like there's a ton of stuff you have to do before you can actually make it to the google page, though. Now if it were to fail on the search for cats, the only way I would be able to test to see if I fixed the code would be to rerun the test (webdriver.Firefox() would open a new browser). Rather than that, assuming I'd still have google open, I'd like the selenium test to just start off on the previous browser and google page (therefore saying the first step in the code would be the send_keys("cats")). Is this possible?
I think that this was a similar question, but it didn't get checked off as answered: How to resume browser session or use existing browser window with Selenium-Python?
This one also seems similar, only pertaining to Java: How do I rerun Selenium 2.0 (webdriver) tests on the same browser?
Thanks.
Look into pdb: https://docs.python.org/2/library/pdb.html
Placing this in your code will stop the progression of the test as is until you tell it to continue in your shell.
Using your code snippit:
from pdb import set_trace
driver = webdriver.Firefox()
driver.get("google.com")
set_trace()
driver.find_element_by_xpath("//input[#id='gbqfq']").send_keys("cats" + Keys.RETURN)
will stop your execution after getting the url, allow you to tinker, and then continue from where the test left off.
Alternatively, while debugging, you can just remove the driver.quit() statement, wherever it happens to be, which will keep the browser open wherever your assertion failed. But if you're using a framework like Django with the LiveTestServer Client, you won't have access to browse the site further. pdb will allow you to keep the test server active.