How do I use Python to simply find a link containing text/id/etc and then click that element?
My imports:
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.keys import Keys
Code to go to my URL:
browser = webdriver.Firefox() # Get local session of firefox
browser.get(myURL) # Load page
#right here I want to look for element containing "Click Me" in the text
#and then click it
Look at method list of WebDriver class - http://selenium.googlecode.com/svn/trunk/docs/api/py/webdriver_remote/selenium.webdriver.remote.webdriver.html
For example, to find element by its ID and click it, the code will look like this
element = browser.find_element_by_id("your_element_id")
element.click()
Hi there are a couple of ways to find a link (or any element):
element = driver.find_element_by_id("element-id")
element = driver.find_element_by_name("element-name")
element = driver.find_element_by_xpath("//input[#id='element-id']")
element = driver.find_element_by_link_text("link-text")
element = driver.find_element_by_class_name("class-name")
I think the best option for you is find_element_by_link_text since it's a link.
Once you saved the element in a variable, you call the click function: element.click() or element.send_keys(Keys.RETURN)
Take a look to the selenium-python documentation, there are a couple of examples there.
You just use this code
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
driver = webdriver.Firefox()
driver.get("https://play.spotify.com/")# here change your link
wait=WebDriverWait(driver,250)
# it will wait for 250 seconds an element to come into view, you can change the #value
submit=wait.until(EC.presence_of_element_located((By.LINK_TEXT, 'Click Me')))
submit.click()
here wait is best fit for javascript enabled website let suppose if you load the page in webdriver but due to javascript some element loads after some second then it best suite to use wait element there.
for link containing text
browser = webdriver.firefox()
browser.find_element_by_link_text(link_text).click()
You can also do it by xpath:
Browser.find_element_by_xpath("//a[#href='you_link']").click()
Finding Element by Link Text
driver.find_element_by_link_text("")
Finding Element by Name
driver.find_element_by_name("")
Finding Element By ID
driver.find_element_by_id("")
Try SST - it is a simple yet very good test framework for python.
Install it first: http://testutils.org/sst/index.html
Then:
Imports:
from sst.actions import *
First define a variable - element - that you're after
element = assert_element(text="some words and more words")
I used this: http://testutils.org/sst/actions.html#assert-element
Then click that element:
click_element('element')
And that's it.
First start one instance of browser. Then you can use any of the following methods to get element or link. After locating the element you can use element.click() or element.send_keys(Keys.RETURN) or any other object of selenium webdriver
browser = webdriver.Firefox()
Selenium provides the following methods to locate elements in a page:
To find individual elements. These methods will return individual element.
browser.find_element_by_id(id)
browser.find_element_by_name(name)
browser.find_element_by_xpath(xpath)
browser.find_element_by_link_text(link_text)
browser.find_element_by_partial_link_text(partial_link_text)
browser.find_element_by_tag_name(tag_name)
browser.find_element_by_class_name(class_name)
browser.find_element_by_css_selector(css_selector)
To find multiple elements (these methods will return a list). Later you can iterate through the list or with elementlist[n] you can locate individual element from list.
browser.find_elements_by_name(name)
browser.find_elements_by_xpath(xpath)
browser.find_elements_by_link_text(link_text)
browser.find_elements_by_partial_link_text(partial_link_text)
browser.find_elements_by_tag_name(tag_name)
browser.find_elements_by_class_name(class_name)
browser.find_elements_by_css_selector(css_selector)
browser.find_element_by_xpath("")
browser.find_element_by_id("")
browser.find_element_by_name("")
browser.find_element_by_class_name("")
Inside the ("") you have to put the respective xpath, id, name, class_name, etc...
Related
I am trying to create an automation program. I want to click on the "Accept Cookies" shadowbox on the given website.
Here's how I have tried to achieve this:
driver = webdriver.Chrome('chromedriver')
driver.get(r'https://www.studydrive.net/')
script = '''return document.querySelector('#usercentrics-root').shadowRoot.querySelector('button[aria-label="Accept All"]')'''
accept_all_btn = driver.execute_script(script)
accept_all_btn.click()
Here's the error that I get after following this approach:
AttributeError: 'NoneType' object has no attribute 'click'
I don't know, what I am doing wrong here. Any help is appreciated. Thank you in advance.
wait=WebDriverWait(driver, 60)
driver.get("https://www.studydrive.net/")
elem = wait.until(EC.presence_of_element_located((By.ID,"usercentrics-root")))
script = '''return document.querySelector('#usercentrics-root').shadowRoot.querySelector('button[data-testid="uc-accept-all-button"]')'''
accept_all_btn = driver.execute_script(script)
accept_all_btn.click()
Simply wait for the element and then proceed to click the accept all button. No aria-label was specified so I used another attribute.
Imports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
The element Accept all button is within a #shadow-root (open)
Solution
Tto click() on the desired element you need to use shadowRoot.querySelector()
You can use the following Locator Strategy:
driver = webdriver.Chrome(service=s, options=options)
driver.get("https://www.studydrive.net/")
time.sleep(5)
accept_all = driver.execute_script('''return document.querySelector("#usercentrics-root").shadowRoot.querySelector("button[data-testid='uc-accept-all-button']")''')
accept_all.click()
PS: The cookie popup surfaces on the screen after significant amount of time, so you may have to induce some waits
References
You can find a couple of relevant detailed discussions in:
How to locate the First name field within shadow-root (open) within the website https://www.virustotal.com using Selenium and Python
How to get past a cookie agreement page using Python and Selenium?
Unable to locate the Sign In element within #shadow-root (open) using Selenium and Python
I am having trouble accessing a input element from this specific webpage. http://prod.symx.com/MTECorp/config.asp?cmd=edit&CID=428D77C8A7ED4DA190E6170116F3A71B
if the webpage has timed out just go ahead and click on this clink below
https://www.mtecorp.com/click-find/
and click on the hyperlink "RL_reactors" to take you to the page.
On this page, I am currently trying to access the search bar/ input element of the webpage to type in a part number that the company sells. This is for school projects and collecting data from different companies for pricing and etc. I am using pycharm(python) and selenium to write this script. Currently, this is the snippet of my code at the moment
# web scraping for MTE product cost list
# reading excel files on the drive
import time
from openpyxl import workbook, load_workbook
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
.........................
..more code
..........................
#part that is getting stuck on
if((selection >= 1) and (selection <= 7)):
print("valid selection going to page...")
if(selection == 1):
target=driver.find_element(By.XPATH,"/html/body/main/article/div/div/div/table/tbody/tr[1]/td[1]/a")
driver.execute_script("arguments[0].click();", target)
element = WebDriverWait(driver,100).until(EC.element_to_be_clickable((By.CSS_SELECTOR,".plxsty_pid"))).send_keys("test")
print("passed clickabel element agruement\n")
currently, my code does go to the RL_reactors page as shown below but however when I'm using CSS selector by class name it doesn't recognize the class type I'm trying to get. Now of course many would say why not use XPath and etc. The reason I cant use XPath and etc is that the element id changes for every iteration of the script. So for example the 1st run of the program id name would be "hr8" when for the other script the program name could be "dsfsih". For my observation, the only part of the element that stays constant is the value and the class name. I have tried using XPath, id, ccselector, and such but to no result. Any suggestions
thanks!
Because you are using javascript to click the link on your website, selenium doesn't change the tab (hence it cannot locate the class you are searching for). You can explicitly tell selenium to change the tab window.
url = "https://www.mtecorp.com/click-find/"
driver.get(url)
target=driver.find_element(By.XPATH,"/html/body/main/article/div/div/div/table/tbody/tr[1]/td[1]/a")
driver.execute_script("arguments[0].click();", target)
driver.switch_to.window(driver.window_handles[1])
element = WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,".plxsty_pid"))).clear()
element = WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,".plxsty_pid"))).send_keys('test')
Alternatively, instead of clicking the link, you can grab the href and open it in a new instance of selenium by calling driver.get() again.
url = "https://www.mtecorp.com/click-find/"
driver.get(url)
target_link=driver.find_element(By.XPATH,"/html/body/main/article/div/div/div/table/tbody/tr[1]/td[1]/a").get_attribute('href')
driver.get(target_link)
element = WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,".plxsty_pid"))).clear()
element = WebDriverWait(driver,20).until(EC.element_to_be_clickable((By.CSS_SELECTOR,".plxsty_pid"))).send_keys("test")
Here is a screenshot of the html structure:
I want to go through each item of league-list(every league-item) and look for the value aria-expanded.
Here is my code:
_1bet = 'https://1bet.com/ca/sports/tennis?time_range=all'
driver.get(_1bet) # enter the website
league1 = driver.find_elements_by_class_name('league-list')[0] # first league-item
league1.find_element_by_xpath(".//div[#class='league-title.d-flex.justify-content-between.align-items-center.collapsible']")
Selenium can't find any element and I don't understand why.
for reference I was inspired by another post :
Parsing nested elements using selenium not working - python
First of all you have to add a wait to make a page loaded before accessing the elements there.
Then you can get all those elements into the list and then iterate over the list getting each element attribute.
I'm not sure the list is initially expanded as it is presented on your picture.
Also not sure all those league-title elements are matching the selector you provided.
If not - we can correct the code accordingly
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 20)
_1bet = 'https://1bet.com/ca/sports/tennis?time_range=all'
driver.get(_1bet) # enter the website
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "div.league-list")))
league1 = driver.find_elements_by_class_name('league-list')[0] # first league-item
titles = league1.find_elements_by_xpath(".//div[#class='league-title.d-flex.justify-content-between.align-items-center.collapsible']")
for title in titles:
print(title.get_attribute("aria-expanded"))
Hoping you can help. I'm relatively new to Python and Selenium. I'm trying to pull together a simple script that will automate news searching on various websites. The primary focus was football and to go and get me the latest Manchester United news from a couple of places and save the list of link titles and URLs for me. I could then look through the links myself and choose anything I wanted to review.
In trying the the independent newspaper (https://www.independent.co.uk/) I seem to have come up against a problem with element not interactable when using the following approaches:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('chromedriver')
driver.get('https://www.independent.co.uk')
time.sleep(3)
#accept the cookies/privacy bit
OK = driver.find_element_by_id('qcCmpButtons')
OK.click()
#wait a few seconds, just in case
time.sleep(5)
search_toggle = driver.find_element_by_class_name('icon-search.dropdown-toggle')
search_toggle.click()
This throws the selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable error
I've also tried with XPATH
search_toggle = driver.find_element_by_xpath('//*[#id="quick-search-toggle"]')
and I also tried ID.
I did a lot of reading on here and then also tried using WebDriverWait and execute_script methods:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//*[#id="quick-search-toggle"]')))
driver.execute_script("arguments[0].click();", element)
This didn't seem to error but the search box never appeared, i.e. the appropriate click didn't happen.
Any help you could give would be fantastic.
Thanks,
Pete
Your locator is //*[#id="quick-search-toggle"], there are 2 on the page. The first is invisible and the second is visible. By default selenium refers to the first element, sadly the element you mean is the second one, so you need another unique locator. Try this:
search_toggle = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="row secondary"]//a[#id="quick-search-toggle"]')))
search_toggle.click()
First you need to open search box, then send search keys:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import os
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome(executable_path=os.path.abspath(os.getcwd()) + "/chromedriver", options=chrome_options)
link = 'https://www.independent.co.uk'
browser.get(link)
# accept privacy
button = browser.find_element_by_xpath('//*[#id="qcCmpButtons"]/button').click()
# open search box
li = browser.find_element_by_xpath('//*[#id="masthead"]/div[3]/nav[2]/ul/li[1]')
search_tab = li.find_element_by_tag_name('a').click()
# send keys to search box
search = browser.find_element_by_xpath('//*[#id="gsc-i-id1"]')
search.send_keys("python")
search.send_keys(Keys.RETURN)
Can you try with below steps
search_toggle = driver.find_element_by_xpath('//*[#class="row secondary"]/nav[2]/ul/li[1]/a')
search_toggle.click()
I have been tasked with writing a parser to click a button on a website and I am having issues to click only one of the buttons. The following code works on every button except one.
Here's the html:
http://pastebin.com/6dLF5ru8
here's the source html:
http://pastebin.com/XhsedGLb
python code:
driver = webdriver.Firefox()
...
el = driver.find_element_by_id("-spel-nba")
actions.move_to_element(el)
actions.sleep(.1)
actions.click()
actions.perform()
I am getting this error.
ElementNotVisibleException: Message: Element is not currently visible and so may not be interacted with
as per Saifur I just tried waits with the same element not visible exception:
wait = WebDriverWait(driver, 10)
wait.until(EC.presence_of_element_located((By.XPATH, "//input[contains(#id,'spsel')][#value='nba']"))).click()
If you look at the page source, you'll understand that almost all of theSELECT, DIV elements are faked and created from JavaScript, that is why webdriver cannot SEE them.
There's a workaround though, by using ActionChains to open your developer console, and inject an artificial CLICK on the desired element, which in fact, is the Label triggering the NBA data loading... here's a working example:
from selenium import webdriver
from selenium.webdriver.common import action_chains, keys
import time
driver = webdriver.Firefox()
driver.get('Your URL here...')
assert 'NBA' in driver.page_source
action = action_chains.ActionChains(driver)
# open up the developer console, mine on MAC, yours may be diff key combo
action.send_keys(keys.Keys.COMMAND+keys.Keys.ALT+'i')
action.perform()
time.sleep(3)
# this below ENTER is to rid of the above "i"
action.send_keys(keys.Keys.ENTER)
# inject the JavaScript...
action.send_keys("document.querySelectorAll('label.boxed')[1].click()"+keys.Keys.ENTER)
action.perform()
Alternatively to replace all the ActionChains commands, you can simply run execute_script like this:
driver.execute_script("document.querySelectorAll('label.boxed')[1].click()")
There you go, at least on my local file anyway... Hope this helps!
What worked for me was to find the element just before the problematic one (that is, just before it in terms of tab order), then call Tab on that element.
from selenium.webdriver.common.keys import Keys
elem = br.find_element_by_name("username")
elem.send_keys(Keys.TAB) # tab over to not-visible element
After doing that, I was able to send actions to the element.
The actual solution of this thread did not work for me.
however,
this one did :
element = WebDriverWait(driver, 3).until(EC.visibility_of_element_located((By.XPATH, xpaths['your_xpath_path'])))
the trick is to use :
EC.visibility_of_element_located
the WebDriverWait
WebDriverWait
from this import :
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
I suggest you use xpath with explicit wait
//input[contains(#id,'spsel')][#value='nba']
if "Element is not currently visible" then make it VISIBLE
f.e.
>>> before is hidden top is outside of page
<input type="file" style="position: absolute;top:-999999" name="file_u">
>>> after move top on in page area
DRIVER.execute_script("document.getElementByName('file_u').style.top = 0;")
time.sleep(1); # give some time to render
DRIVER.find_element_by_name("file_u").send_keys("/tmp/img.png")
Instead of get_element_by_id() you can try elem = browser.find_element_by_css_selector('#elemId') (go to that webpage and the element, right click it and Copy CSS Selector, or something like that.) This is what i did and it works. You also try find_element_by_link_text(text), find_element_by_partial_link_text(text), find_element_by_tag_name(tagName_case_insensitive_here), find_element_by_name(name) etc. Something will work. After the id the CSS Selector is your best bet.
I ended up using #twasbrillig's solution, but instead of finding the previous element and sending a TAB keypress, I find the desired element, send a TAB keypress with that element, and then a SHIFT + TAB keypress to the driver:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Firefox()
el = driver.find_element_by_id("-spel-nba")
el.send_keys(Keys.TAB)
webdriver.ActionChains(driver).key_down(Keys.SHIFT).send_keys(Keys.TAB).key_up(Keys.SHIFT)
I tried using the other methods but in the end found that the simplest way was to just try and click the button, and catch the error. This allows me to perform other actions based on if it worked (True) or didn't (False).
def click_button(html_object):
try:
html_object.click()
except:
return False #most likely because it is NotVisible object and can be ignored
return True
...
...
click_button(actions)
The way I solved this in python was:
try:
# the element you want to scroll to
element = driver.find_element_by_id("some_id")
ActionChains(driver).move_to_element(element).perform()
element.send_keys(Keys.TAB).key_up(Keys.SHIFT)
#element.click()
except Exception as e:
print(e)