Opening element(url) in new tabs in Python Selenium

Opening element(url) in new tabs in Python Selenium - python

Objective:
Need to open a webpage, find urls by class and then open each url in a new tab in chrome
What I tried:
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path="C:\\Users\xxx\Downloads\chromedriver_win32\chromedriver.exe")
driver.maximize_window()
driver.get("https://xxxxxxxx.com")
elements = driver.find_elements_by_class_name("post-comments")
for e in elements:
e.click()
driver.quit()
current output:
It's just opening the first url in the same parent window then stop there.. focus is not going to base url so it is not able to open second and other urls in new tab.
Please let me know your suggestions.
Update1: when I search for relative xpath, I came to know that I need to use regexp so that I can find all links containing "respond" in it.
For that I tried to use
driver.find_element_by_css_selector("a[href*='respond']]")
but it errored there is no find_element.
Update 2:
After some trials, I am able to achieve to the most output that I require with below code. But below code is working on first page only, when below code completes for first page, then there will be 'NextPage' button , I need to click on that to open that page, that will become new main page and I need to run the below piece of code again on that page. How to tell below piece of code once elements on this page done, click on 'NextPage' button and run below logic for that page. Please suggest
from selenium import webdriver
import time
from selenium.webdriver import ActionChains
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path="C:\\Users\xxx\Downloads\chromedriver_win32\chromedriver.exe")
driver.maximize_window()
driver.get("https://xxxxxxxx.com")
current_windows_handle = driver.current_window_handle
elements = driver.find_elements_by_css_selector('h2.post-title.entry-title>a')
for ele in (elements):
url = [ele.get_attribute("href")]
for x in url:
time.sleep(2)
driver.execute_script("window.open('');")
handles = driver.window_handles
driver.switch_to.window(handles[1])
driver.get(x)
driver.find_element_by_xpath("//span[#class='mb-text'][contains(.,'Posts')]").click()
time.sleep(2)
driver.close()
driver.switch_to.window(driver.window_handles[1])
time.sleep(5)
actualurl = driver.current_url
with open(r'c:\test\new.txt', 'a') as text_file:
text_file.write(actualurl + '\n')
driver.close()
time.sleep(5)
driver.switch_to.window(current_windows_handle)
driver.close()
driver.quit()

The error is valid, cause
driver.find_element_by_css_selector("a[href*='respond']]")
this is syntax error, but since it is wrapped inside "", compiler won't show it.
try this instead :
elements = driver.find_elements_by_css_selector("a[href*='respond']")
and then print the length like this :-
print(len(elements))
I am assuming that it will have some elements in it. and you can iterate it as well. Pleas see below, it should help you figure out how to simulate those actions.
current_windows_handle = driver.current_window_handle
elements = driver.find_elements_by_css_selector("a[href*='respond']")
for ele in elements:
time.sleep(2)
driver.execute_script("window.open('');") # open a new tab.
handles = driver.window_handles
driver.switch_to.window(handles[1])
driver.get(ele.get_attribute('href'))
time.sleep(2)
# Now it would have loaded a new url in new tab.
# do whatever you wanna do.
# Now close the new tab
driver.close()
time.sleep(2)
driver.switch_to.window(current_windows_handle)
updated 1 :
current_windows_handle = driver.current_window_handle
elements = driver.find_elements_by_css_selector("a[href*='respond']")
j = 0
for ele in range(len(elements)):
time.sleep(2)
elements = driver.find_elements_by_css_selector("a[href*='respond']")
driver.execute_script("window.open('');") # open a new tab.
handles = driver.window_handles
driver.switch_to.window(handles[1])
driver.get(elements[j].get_attribute('href'))
time.sleep(2)
j = j + 1
# Now it would have loaded a new url in new tab.
# do whatever you wanna do.
# Now close the new tab
driver.close()
time.sleep(2)
driver.switch_to.window(current_windows_handle)

Related

find_elements not finding all elements

I'm trying to find all the elements on a page with the specific selector but as it seems, not all the elements are found.
Code I'm using:
from selenium import webdriver
from selenium.webdriver.common.by import By
PATH = "C:\Program Files (x86)\chromedriver.exe"#path of chrome driver
driver = webdriver.Chrome(PATH)#accesses the chrome driver
web = driver.get("https://www.eduqas.co.uk/qualifications/computer-science-as-a-level/#tab_pastpapers")#website
driver.maximize_window()
driver.implicitly_wait(3)
driver.execute_script("window.scrollTo(0, 540)")
driver.implicitly_wait(3)
elements = driver.find_elements(By.CSS_SELECTOR, ".css-13punl2")
driver.implicitly_wait(3)
for x in elements:
x.click()
print(len(elements))
When I print the length of the array "elements" it returns 1, when there are multiple elements on the web page with the selector ".css-13punl2". As seen here image of web page code
link to the website: https://www.eduqas.co.uk/qualifications/computer-science-as-a-level/#tab_pastpapers
For some reason, when I inspect the web page there will sometimes be 6 elements with selector ".css-13punl2" and sometimes there will be 7, but I'm not too sure.

is the selector stable?
im not much familiar with selenium in python, but from what i know, in runtime there are some element attributes that change...
my advice:
put a sleep for 30 seconds, open console (F12) in the opened driver, and write the following command:
$$(".css-13punl2")
if it gives you only 1 element, than you found the problem
or that even it gave you 6 elements, but most of them are invisible.
could you also provide a screenshot of the web itself? or even the link to it
EDITED ANSWER:
try this selector:
#pastpapers_content button

You are trying to wait for the element with driver.implicitly_wait(3). This method is created for activating implicit wait. It can be turned On once and then you don't need to turn it again. When it's turned On Selenium will try to find the element you want for 3 (in this case) seconds instead of one attempt immediately after page loading. In your case it finds one element quickly and doesn't wait for all the elements to appear on the page.
You need to give the page some time to load all the stuff. For that you can use a sleep method for example. It pauses the script execution for the amount of seconds you've set.
Also, the elements on this page disappear from the view so you need to scroll each time you click an element. Additionally you need to close this 'Cookie' prompt as it intercepts the clicks.
And I guess you need the elements with Year, so it's better to skip clicking the 'GCSE' which is also found by this selector.
So, the code will look like that:
from selenium import webdriver
from selenium.webdriver.common.by import By
from time import sleep
PATH = "C:\Program Files (x86)\chromedriver.exe" # path of chrome driver
driver = webdriver.Chrome(PATH) # accesses the chrome driver
driver.get("https://www.eduqas.co.uk/qualifications/computer-science-as-a-level/#tab_pastpapers") # website
driver.maximize_window()
driver.implicitly_wait(3)
driver.execute_script("window.scrollTo(0, 540)")
sleep(3) # Giving time to fully load the content
elements = driver.find_elements(By.CSS_SELECTOR, ".css-13punl2")
driver.find_element(By.ID, 'accept-cookies').click() # Closes the cookies prompt
for x in elements:
if x.text == 'GCSE':
continue
x.click()
driver.execute_script("window.scrollTo(0, document.body.scrollHeight);") # this scrolls the page to the bottom
sleep(1) # This sleep is necessary to give time to finish scrolling
print(len(elements))

Don't use implicitly_wait() more than once, try the below code, it will click on each year:
driver.get("https://www.eduqas.co.uk/qualifications/computer-science-as-a-level/#tab_pastpapers")
wait = WebDriverWait(driver, 10)
time.sleep(1)
# to click on Accept Cookies button
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "#accept-cookies"))).click()
# waiting for list of years to appear and scrolling to it
content = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, "#pastpapers_content")))
driver.execute_script("arguments[0].scrollIntoView(true)", content)
# get all the elements
elements = wait.until(EC.presence_of_all_elements_located((By.XPATH, ".//*[#id='pastpapers_content']//button[#type='button']")))
print("Total elements: ", len(elements))
for i in range(len(elements)):
ele = driver.find_element(By.XPATH, "(.//*[#id='pastpapers_content']//button[#type='button'])[" + str(i + 1) + "]")
time.sleep(1)
ele.location_once_scrolled_into_view
time.sleep(1)
ele.click()

Why cant i iterate through a list in selenium?

I am trying to scrape the headers of wikipedia pages as an exercise, and i want to be able to distinguish between headers with "h2" and "h3" tags.
Therefore i wrote this code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys #For being able to input key presses
import time #Useful for if your browser is faster than your code
PATH = r"C:\Users\Alireza\Desktop\chromedriver\chromedriver.exe" #Location of the chromedriver
driver = webdriver.Chrome(PATH)
driver.get("https://de.wikipedia.org/wiki/Alpha-Beta-Suche") #Open website in Chrome
print(driver.title) #Print title of the website to console
h1Header = driver.find_element_by_tag_name("h1") #Find the first heading in the article
h2HeaderTexts = driver.find_elements_by_tag_name("h2") #List of all other major headers in the article
h3HeaderTexts = driver.find_elements_by_tag_name("h3") #List of all minor headers in the article
for items in h2HeaderTexts:
scor = items.find_element_by_class_name("mw-headline")
driver.quit()
However, this does not work and the program does not terminate.
Anybody have a solution for this?
The problem here lies in the for loop! Apparently i can not scrape any elements by class name (or anything else) from the elements in h2HeaderTexts, although this should be possible.

You can filter in xpath iteself :
PATH = r"C:\Users\Alireza\Desktop\chromedriver\chromedriver.exe" #Location of the chromedriver
driver = webdriver.Chrome(PATH)
driver.maximize_window()
driver.implicitly_wait(30)
driver.get("https://de.wikipedia.org/wiki/Alpha-Beta-Suche") #Open website in Chrome
print(driver.title)
for item in driver.find_elements(By.XPATH, "//h2/span[#class='mw-headline']"):
print(item.text)
this should give you, h2 heading with class mw-headline elements.
output :
Informelle Beschreibung
Der Algorithmus
Implementierung
Optimierungen
Vergleich von Minimax und AlphaBeta
Geschichte
Literatur
Weblinks
Fußnoten
Process finished with exit code 0
Update 1 :
The reason why your loop is still running and program does not terminate, is cause if you look the page HTML source, and the first h2 tag, that h2 tag does not have a child span with mw-headline, so selenium is trying to locate the element which is not there in HTML DOM. also you are using find_elements which return a list of web elements if found, if not return an empty list, is the reason you do not see exception as well.

You should wait until elements appearing on the page before accessing them.
Also, there are several elements with h1 tag name too there.
To search for elements inside element you should use xpath starting with a dot. Otherwise this will search for the first match on the entire page.
The first h2 element on that page has no element with class name mw-headline inside it. So, you should handle this issue too.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC #For being able to input key presses
import time #Useful for if your browser is faster than your code
PATH = r"C:\Users\Alireza\Desktop\chromedriver\chromedriver.exe" #Location of the chromedriver
driver = webdriver.Chrome(PATH)
wait = WebDriverWait(driver, 20)
driver.get("https://de.wikipedia.org/wiki/Alpha-Beta-Suche") #Open website in Chrome
print(driver.title) #Print title of the website to console
wait.until(EC.visibility_of_element_located((By.XPATH, "//h1")))
h1Headers = driver.find_elements_by_tag_name("h1") #Find the first heading in the article
h2HeaderTexts = driver.find_elements_by_tag_name("h2") #List of all other major headers in the article
h3HeaderTexts = driver.find_elements_by_tag_name("h3") #List of all minor headers in the article
for items in h2HeaderTexts:
scor = items.find_elements_by_xpath(".//span[#class='mw-headline']")
if scor:
#do what you need with scor[0] element
driver.quit()

You're version does not finish executing because selenium will drop the process if it could not locate an element.
Devs do not like to use try/catch but i personnaly have not found a better way to work around. If you do :
for items in h2HeaderTexts:
try:
scor = items.find_element_by_class_name('mw-headline').text
print(scor)
except:
print('nothing found')
You will notice that it will execute till the end and you have a result.

Selenium webdriver closes unexpectedly with terminal [duplicate]

So I am trying to open websites on new tabs inside my WebDriver. I want to do this, because opening a new WebDriver for each website takes about 3.5secs using PhantomJS, I want more speed...
I'm using a multiprocess python script, and I want to get some elements from each page, so the workflow is like this:
Open Browser
Loop throught my array
For element in array -> Open website in new tab -> do my business -> close it
But I can't find any way to achieve this.
Here's the code I'm using. It takes forever between websites, I need it to be fast... Other tools are allowed, but I don't know too many tools for scrapping website content that loads with JavaScript (divs created when some event is triggered on load etc) That's why I need Selenium... BeautifulSoup can't be used for some of my pages.
#!/usr/bin/env python
import multiprocessing, time, pika, json, traceback, logging, sys, os, itertools, urllib, urllib2, cStringIO, mysql.connector, shutil, hashlib, socket, urllib2, re
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from PIL import Image
from os import listdir
from os.path import isfile, join
from bs4 import BeautifulSoup
from pprint import pprint
def getPhantomData(parameters):
try:
# We create WebDriver
browser = webdriver.Firefox()
# Navigate to URL
browser.get(parameters['target_url'])
# Find all links by Selector
links = browser.find_elements_by_css_selector(parameters['selector'])
result = []
for link in links:
# Extract link attribute and append to our list
result.append(link.get_attribute(parameters['attribute']))
browser.close()
browser.quit()
return json.dumps({'data': result})
except Exception, err:
browser.close()
browser.quit()
print err
def callback(ch, method, properties, body):
parameters = json.loads(body)
message = getPhantomData(parameters)
if message['data']:
ch.basic_ack(delivery_tag=method.delivery_tag)
else:
ch.basic_reject(delivery_tag=method.delivery_tag, requeue=True)
def consume():
credentials = pika.PlainCredentials('invitado', 'invitado')
rabbit = pika.ConnectionParameters('localhost',5672,'/',credentials)
connection = pika.BlockingConnection(rabbit)
channel = connection.channel()
# Conectamos al canal
channel.queue_declare(queue='com.stuff.images', durable=True)
channel.basic_consume(callback,queue='com.stuff.images')
print ' [*] Waiting for messages. To exit press CTRL^C'
try:
channel.start_consuming()
except KeyboardInterrupt:
pass
workers = 5
pool = multiprocessing.Pool(processes=workers)
for i in xrange(0, workers):
pool.apply_async(consume)
try:
while True:
continue
except KeyboardInterrupt:
print ' [*] Exiting...'
pool.terminate()
pool.join()

Editor's note: This answer no longer works for new Selenium versions. Refer to this comment.
You can achieve the opening/closing of a tab by the combination of keys COMMAND + T or COMMAND + W (OSX). On other OSs you can use CONTROL + T / CONTROL + W.
In selenium you can emulate such behavior.
You will need to create one webdriver and as many tabs as the tests you need.
Here it is the code.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.google.com/")
#open tab
driver.find_element_by_tag_name('body').send_keys(Keys.COMMAND + 't')
# You can use (Keys.CONTROL + 't') on other OSs
# Load a page
driver.get('http://stackoverflow.com/')
# Make the tests...
# close the tab
# (Keys.CONTROL + 'w') on other OSs.
driver.find_element_by_tag_name('body').send_keys(Keys.COMMAND + 'w')
driver.close()

browser.execute_script('''window.open("http://bings.com","_blank");''')
Where browser is the webDriver

This is a common code adapted from another examples:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
driver.get("http://www.google.com/")
#open tab
# ... take the code from the options below
# Load a page
driver.get('http://bings.com')
# Make the tests...
# close the tab
driver.quit()
the possible ways were:
Sending <CTRL> + <T> to one element
#open tab
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')
Sending <CTRL> + <T> via Action chains
ActionChains(driver).key_down(Keys.CONTROL).send_keys('t').key_up(Keys.CONTROL).perform()
Execute a javascript snippet
driver.execute_script('''window.open("http://bings.com","_blank");''')
In order to achieve this you need to ensure that the preferences browser.link.open_newwindow and browser.link.open_newwindow.restriction are properly set. The default values in the last versions are ok, otherwise you supposedly need:
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.link.open_newwindow", 3)
fp.set_preference("browser.link.open_newwindow.restriction", 2)
driver = webdriver.Firefox(browser_profile=fp)
the problem is that those preferences preset to other values and are frozen at least selenium 3.4.0. When you use the profile to set them with the java binding there comes an exception and with the python binding the new values are ignored.
In Java there is a way to set those preferences without specifying a profile object when talking to geckodriver, but it seem to be not implemented yet in the python binding:
FirefoxOptions options = new FirefoxOptions().setProfile(fp);
options.addPreference("browser.link.open_newwindow", 3);
options.addPreference("browser.link.open_newwindow.restriction", 2);
FirefoxDriver driver = new FirefoxDriver(options);
The third option did stop working for python in selenium 3.4.0.
The first two options also did seem to stop working in selenium 3.4.0. They do depend on sending CTRL key event to an element. At first glance it seem that is a problem of the CTRL key, but it is failing because of the new multiprocess feature of Firefox. It might be that this new architecture impose new ways of doing that, or maybe is a temporary implementation problem. Anyway we can disable it via:
fp = webdriver.FirefoxProfile()
fp.set_preference("browser.tabs.remote.autostart", False)
fp.set_preference("browser.tabs.remote.autostart.1", False)
fp.set_preference("browser.tabs.remote.autostart.2", False)
driver = webdriver.Firefox(browser_profile=fp)
... and then you can use successfully the first way.

OS: Win 10,
Python 3.8.1
selenium==3.141.0
from selenium import webdriver
import time
driver = webdriver.Firefox(executable_path=r'TO\Your\Path\geckodriver.exe')
driver.get('https://www.google.com/')
# Open a new window
driver.execute_script("window.open('');")
# Switch to the new window
driver.switch_to.window(driver.window_handles[1])
driver.get("http://stackoverflow.com")
time.sleep(3)
# Open a new window
driver.execute_script("window.open('');")
# Switch to the new window
driver.switch_to.window(driver.window_handles[2])
driver.get("https://www.reddit.com/")
time.sleep(3)
# close the active tab
driver.close()
time.sleep(3)
# Switch back to the first tab
driver.switch_to.window(driver.window_handles[0])
driver.get("https://bing.com")
time.sleep(3)
# Close the only tab, will also close the browser.
driver.close()
Reference: Need Help Opening A New Tab in Selenium

The other solutions do not work for chrome driver v83.
Instead, it works as follows, suppose there is only 1 opening tab:
driver.execute_script("window.open('');")
driver.switch_to.window(driver.window_handles[1])
driver.get("https://www.example.com")
If there are already more than 1 opening tabs, you should first get the index of the last newly-created tab and switch to the tab before calling the url (Credit to tylerl) :
driver.execute_script("window.open('');")
driver.switch_to.window(len(driver.window_handles)-1)
driver.get("https://www.example.com")

In a discussion, Simon clearly mentioned that:
While the datatype used for storing the list of handles may be ordered by insertion, the order in which the WebDriver implementation iterates over the window handles to insert them has no requirement to be stable. The ordering is arbitrary.
Using Selenium v3.x opening a website in a New Tab through Python is much easier now. We have to induce an WebDriverWait for number_of_windows_to_be(2) and then collect the window handles every time we open a new tab/window and finally iterate through the window handles and switchTo().window(newly_opened) as required. Here is a solution where you can open http://www.google.co.in in the initial TAB and https://www.yahoo.com in the adjacent TAB:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
options.add_argument("start-maximized")
options.add_argument('disable-infobars')
driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\Utility\BrowserDrivers\chromedriver.exe')
driver.get("http://www.google.co.in")
print("Initial Page Title is : %s" %driver.title)
windows_before = driver.current_window_handle
print("First Window Handle is : %s" %windows_before)
driver.execute_script("window.open('https://www.yahoo.com')")
WebDriverWait(driver, 10).until(EC.number_of_windows_to_be(2))
windows_after = driver.window_handles
new_window = [x for x in windows_after if x != windows_before][0]
driver.switch_to.window(new_window)
print("Page Title after Tab Switching is : %s" %driver.title)
print("Second Window Handle is : %s" %new_window)
Console Output:
Initial Page Title is : Google
First Window Handle is : CDwindow-B2B3DE3A222B3DA5237840FA574AF780
Page Title after Tab Switching is : Yahoo
Second Window Handle is : CDwindow-D7DA7666A0008ED91991C623105A2EC4
Browser Snapshot:
Outro
You can find the java based discussion in Best way to keep track and iterate through tabs and windows using WindowHandles using Selenium

Try this it will work:
# Open a new Tab
driver.execute_script("window.open('');")
# Switch to the new window and open URL B
driver.switch_to.window(driver.window_handles[1])
driver.get(tab_url)

After struggling for so long the below method worked for me:
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + Keys.TAB)
windows = driver.window_handles
time.sleep(3)
driver.switch_to.window(windows[1])

from selenium import webdriver
import time
driver = webdriver.Firefox()
driver.get('https://www.google.com')
driver.execute_script("window.open('');")
time.sleep(5)
driver.switch_to.window(driver.window_handles[1])
driver.get("https://facebook.com")
time.sleep(5)
driver.close()
time.sleep(5)
driver.switch_to.window(driver.window_handles[0])
driver.get("https://www.yahoo.com")
time.sleep(5)
#driver.close()
https://www.edureka.co/community/52772/close-active-current-without-closing-browser-selenium-python

just for future reference, the simple way could be done as this:
driver.switch_to.new_window()
t=driver.window_handles[-1]# Get the handle of new tab
driver.switch_to.window(t)
driver.get(target_url) # Now the target url is opened in new tab

The 4.0.0 version of Selenium supports the following operations:
to open a new tab try:
driver.switch_to.new_window()
to switch to a specific tab (note that the tabID starts from 0):
driver.switch_to.window(driver.window_handles[tabID])

Strangely, so many answers, and all of them are using surrogates like JS and keyboard shortcuts instead of just using a selenium feature:
def newTab(driver, url="about:blank"):
wnd = driver.execute(selenium.webdriver.common.action_chains.Command.NEW_WINDOW)
handle = wnd["value"]["handle"]
driver.switch_to.window(handle)
driver.get(url) # changes the handle
return driver.current_window_handle

I'd stick to ActionChains for this.
Here's a function which opens a new tab and switches to that tab:
import time
from selenium.webdriver.common.action_chains import ActionChains
def open_in_new_tab(driver, element, switch_to_new_tab=True):
base_handle = driver.current_window_handle
# Do some actions
ActionChains(driver) \
.move_to_element(element) \
.key_down(Keys.COMMAND) \
.click() \
.key_up(Keys.COMMAND) \
.perform()
# Should you switch to the new tab?
if switch_to_new_tab:
new_handle = [x for x in driver.window_handles if x!=base_handle]
assert len new_handle == 1 # assume you are only opening one tab at a time
# Switch to the new window
driver.switch_to.window(new_handle[0])
# I like to wait after switching to a new tab for the content to load
# Do that either with time.sleep() or with WebDriverWait until a basic
# element of the page appears (such as "body") -- reference for this is
# provided below
time.sleep(0.5)
# NOTE: if you choose to switch to the window/tab, be sure to close
# the newly opened window/tab after using it and that you switch back
# to the original "base_handle" --> otherwise, you'll experience many
# errors and a painful debugging experience...
Here's how you would apply that function:
# Remember your starting handle
base_handle = driver.current_window_handle
# Say we have a list of elements and each is a link:
links = driver.find_elements_by_css_selector('a[href]')
# Loop through the links and open each one in a new tab
for link in links:
open_in_new_tab(driver, link, True)
# Do something on this new page
print(driver.current_url)
# Once you're finished, close this tab and switch back to the original one
driver.close()
driver.switch_to.window(base_handle)
# You're ready to continue to the next item in your loop
Here's how you could wait until the page is loaded.

you can use this to open a new tab
driver.execute_script("window.open('http://google.com', 'new_window')")

This worked for me:-
link = "https://www.google.com/"
driver.execute_script('''window.open("about:blank");''') # Opening a blank new tab
driver.switch_to.window(driver.window_handles[1]) # Switching to newly opend tab
driver.get(link)

just enough use this to open new window(for example):
driver.find_element_by_link_text("Images").send_keys(Keys.CONTROL + Keys.RETURN)

I tried for a very long time to duplicate tabs in Chrome running using action_keys and send_keys on body. The only thing that worked for me was an answer here. This is what my duplicate tabs def ended up looking like, probably not the best but it works fine for me.
def duplicate_tabs(number, chromewebdriver):
#Once on the page we want to open a bunch of tabs
url = chromewebdriver.current_url
for i in range(number):
print('opened tab: '+str(i))
chromewebdriver.execute_script("window.open('"+url+"', 'new_window"+str(i)+"')")
It basically runs some java from inside of python, it's incredibly useful. Hope this helps somebody.
Note: I am using Ubuntu, it shouldn't make a difference but if it doesn't work for you this could be the reason.

Opening the new empty tab within same window in chrome browser is not possible up to my knowledge but you can open the new tab with web-link.
So far I surfed net and I got good working content on this question.
Please try to follow the steps without missing.
import selenium.webdriver as webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get('https://www.google.com?q=python#q=python')
first_link = driver.find_element_by_class_name('l')
# Use: Keys.CONTROL + Keys.SHIFT + Keys.RETURN to open tab on top of the stack
first_link.send_keys(Keys.CONTROL + Keys.RETURN)
# Switch tab to the new tab, which we will assume is the next one on the right
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + Keys.TAB)
driver.quit()
I think this is better solution so far.
Credits: https://gist.github.com/lrhache/7686903

tabs = {}
def new_tab():
global browser
hpos = browser.window_handles.index(browser.current_window_handle)
browser.execute_script("window.open('');")
browser.switch_to.window(browser.window_handles[hpos + 1])
return(browser.current_window_handle)
def switch_tab(name):
global tabs
global browser
if not name in tabs.keys():
tabs[name] = {'window_handle': new_tab(), 'url': url+name}
browser.get(tabs[name]['url'])
else:
browser.switch_to.window(tabs[name]['window_handle'])

As already mentioned several times, the following approaches are NOT working anymore:
driver.find_element_by_tag_name('body').send_keys(Keys.CONTROL + 't')
ActionChains(driver).key_down(Keys.CONTROL).send_keys('t').key_up(Keys.CONTROL).perform()
Moreover, driver.execute_script("window.open('');") is working but is limited by the popup blocker. I process hundreds of tabs in parallel (web scraping using scrapy). However, the popup blocker became active after opening 20 new tabs using JavaScript's window.open('') and, thus, has broke my crawler.
As work around I declared a tab as "master" which has opended the following helper.html:
<!DOCTYPE html>
<html><body>
<a id="open_new_window" href="about:blank" target="_blank">open a new window</a>
</body></html>
Now, my (simplified) crawler can open as many tabs as necessary by purposely clicking the link which is not considered by the popup blogger at all:
# master
master_handle = driver.current_window_handle
helper = os.path.join(os.path.dirname(os.path.abspath(__file__)), "helper.html")
driver.get(helper)
# open new tabs
for _ in range(100):
window_handle = driver.window_handles # current state
driver.switch_to_window(master_handle)
driver.find_element_by_id("open_new_window").click()
window_handle = set(driver.window_handles).difference(window_handle).pop()
print("new window handle:", window_handle)
Closing these windows via JavaScript's window.close() is no problem.

#Change the method of finding the element if needed
self.find_element_by_xpath(element).send_keys(Keys.CONTROL + Keys.ENTER)
This will find the element and open it in a new tab. self is just the name used for the webdriver object.

Is there a way to click a list of multiple buttons, one by one, in selenium?

I am a student working on a scraping project with selenium on python. I am trying to scrape data from multiple profiles, that are all formatted the same. There is a directory website with buttons that lead to all of the profiles, but the problem that I am having is being able to click every button one at a time because they are all formatted the same. My goal is to be able to open the 1st link in a new tab, scrape the data from that profile, then close the first profile's tab and move on too the second. I hope to make this process repeatable. Here is what I have so far:
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
import time
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path="MY PATH TO MY CHROME DRIVER")
driver.implicitly_wait(5)
driver.get("http://directory.bcsp.org")
buttons = driver.find_elements_by_link_text('View Profile')
Please let me know if you have any solutions to my problem. Thank you :)

"body" for to go down on the page
"profile_count" for get link
".get_attribute('href')" for to scrape the link from the "View Profile" button
"temp" the information in the old window does not switch to the new window so we need temp
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
driver = webdriver.Edge()
driver.get("https://directory.bcsp.org/")
count = int(input("Count : "))
body = driver.find_element_by_xpath("//body") #
profile_count = driver.find_elements_by_xpath("//div[#align='right']/a")
while len(profile_count) < count: # Get links up to "count"
body.send_keys(Keys.END)
sleep(1)
profile_count = driver.find_elements_by_xpath("//div[#align='right']/a")
for link in profile_count: # Calling up links
temp = link.get_attribute('href') # temp for
driver.execute_script("window.open('');") # open new tab
driver.switch_to.window(driver.window_handles[1]) # focus new tab
driver.get(temp)
# you can do
# what you want
# to do here
driver.close()
driver.switch_to.window(driver.window_handles[0])
driver.close()

Python - Selenium going through array

I would like to do automated script. I define an array in start of the program.
Later program is opening browser and search in google some specific word (for example apple), next program is clicking in first of string from array and close browser. Later is doing the same but it will click in second of word from array.
My code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome("C:/Users/Daniel/Desktop/chromedriver.exe")
driver.implicitly_wait(30)
driver.maximize_window()
hasla = ["ispot","myapple"]
for slogan in hasla:
driver.get("http://www.google.com")
search_field = driver.find_element_by_id("lst-ib")
search_field.clear()
search_field.send_keys("apple")
search_field.submit()
name = driver.find_element_by_link_text(slogan)
name.click()
driver.quit()
driver.implicitly_wait(10)
When Im starting this program from console in windows.
Program is opening browser, looking for apple click in ispot and clsoe browser but its not opening new browser and it not doing the same for next string in array. Any solution ?
In console i have this:
screen

You're exiting the browser in your for loop, so the second iteration can't do anything because there's not a browser open. You could try opening a new tab and closing the old one if you need to start fresh each time. Try this:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome("C:/Users/Daniel/Desktop/chromedriver.exe")
driver.implicitly_wait(30)
driver.maximize_window()
hasla = ["ispot","myapple"]
for slogan in hasla:
driver.get("http://www.google.com")
search_field = driver.find_element_by_id("lst-ib")
search_field.clear()
search_field.send_keys("apple")
search_field.submit()
name = driver.find_element_by_link_text(slogan)
name.click()
# Save the current tab id
old_handle = driver.current_window_handle
# Execute JavaScript to open a new tab and save its id
driver.execute_script("window.open('');")
new_handle = driver.window_handles[-1]
# Switch to the old tab and close it
driver.switch_to.window(old_handle)
driver.close()
# Switch focus to the new tab
driver.switch_to.window(new_handle)
If you are closing the tab you won't be able to see the results. You might want to keep it open and just go to the new tab. In which case, just remove driver.close().
Alternatively, if you really want to completely close out the browser each time and reopen it, you just need to include the first three lines in your for-loop.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
hasla = ["ispot","myapple"]
for slogan in hasla:
driver = webdriver.Chrome("C:/Users/Daniel/Desktop/chromedriver.exe")
driver.implicitly_wait(30)
driver.maximize_window()
driver.get("http://www.google.com")
search_field = driver.find_element_by_id("lst-ib")
search_field.clear()
search_field.send_keys("apple")
search_field.submit()
name = driver.find_element_by_link_text(slogan)
name.click()
driver.quit()
To answer your second question:
First, import NoSuchElementException:
from selenium.common.exceptions import NoSuchElementException
Then replace your try/except with this:
try:
name = driver.find_element_by_link_text(slogan)
name.click()
except NoSuchElementException:
print('No such element')
driver.quit()
It's still going to close the browser and go to the next iteration if the element is found or not.

Develop Reference

Python is a programming language that lets you work quickly and integrate systems more effectively.

Opening element(url) in new tabs in Python Selenium - python

Related

find_elements not finding all elements

Why cant i iterate through a list in selenium?

Selenium webdriver closes unexpectedly with terminal [duplicate]

Is there a way to click a list of multiple buttons, one by one, in selenium?

Python - Selenium going through array

Categories

Resources