I need to find and store the location of some elements so the bot can click on those elements even the if page changes. I have read online that for a single element storing the location of that element in a variable can help however I could not find a way to store locations of multiple elements in python. Here is my code
comment_button = driver.find_elements_by_css_selector("svg[aria-label='Comment']")
for element in comment_button:
comment_location = element.location
sleep(2)
for element in comment_location:
element.click()
this code gives out this error:
line 44, in <module>
element.click()
AttributeError: 'str' object has no attribute 'click'
Is there a way to do this so when the page refreshes the script can store the locations and move on to the next location to execute element.click without any errors?
I have tried implementing ActionChains into my code
comment_button = driver.find_elements_by_css_selector("svg[aria-label='Comment']")
for element in comment_button:
ac = ActionChains(driver)
element.click()
ac.move_to_element(element).move_by_offset(0, 0).click().perform()
sleep(2)
comment_button = driver.find_element_by_css_selector("svg[aria-label='Comment']")
comment_button.click()
sleep(2)
comment_box = driver.find_element_by_css_selector("textarea[aria-label='Add a comment…']")
comment_box.click()
comment_box = driver.find_element_by_css_selector("textarea[aria-label='Add a comment…']")
comment_box.send_keys("xxxx")
post_button = driver.find_element_by_xpath("//button[#type='submit']")
post_button.click()
sleep(2)
driver.back()
scroll()
However this method gives out the same error saying that the page was refreshed and the object can not be found.
selenium.common.exceptions.StaleElementReferenceException: Message: The element reference of <svg class="_8-yf5 "> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
Edited:
Assuming No. of such elements are not changing after refresh of page, you can use below code
commentbtns = driver.find_elements_by_css_selector("svg[aria-label='Comment']")
for n in range(1, len(commentbtns)+1):
Path = "(//*[name()='svg'])["+str(n)+"]"
time.sleep(2)
driver.find_element_by_xpath(Path).click()
You can use more sophisticated ways to wait for element to load properly. However for simplicity purpose i have used time.sleep.
Related
i've written the below lines of code:
elem = driver.find_elements(By.XPATH, "//span[#class='drop__expand']")
for i in elem:
i.click()
and i get the below error:
selenium.common.exceptions.ElementClickInterceptedException: Message: Element <span class="drop__expand"> is not clickable at point (112,20) because another element <div class="wrapper"> obscures it
i tried that without any result:
driver.find_elements(By.XPATH, "//div[#class='wrapper']").click()
elem = driver.find_elements(By.XPATH, "//span[#class='drop__expand']")
for i in elem:
i.click()
how can i handle this?
There are two functions:
find_elements() with s at the end - to get list with all matching elements - even if there is one element or there is no elements.
find_element() without s at the end - to get only first matching elements.
So to click element you may need second function (without s)
driver.find_element(By.XPATH, "//div[#class='wrapper']").click()
or you have to get first element from list when you use first function (with s)
driver.find_elements(By.XPATH, "//div[#class='wrapper']")[0].click()
or use for-loop
for item in driver.find_elements(By.XPATH, "//div[#class='wrapper']"):
item.click()
But wrapper may not be clickable - if it is some popup message then you may need to find button on this wrapper. But you didn't show URL for this page so only you have access to full HTML to check if it has button and find xpath for this button. And here I can't help.
You may also try to use JavaScript to click hiddent element(s) and maybe it will work.
Something like this
elem = driver.find_elements(By.XPATH, "//span[#class='drop__expand']")
for i in elem:
driver.execute_script("arguments[0].click()", i)
But all this is only guess because you didn't show url for this page and we can't test it.
I have seen other questions about this error but my case is that in my program the other element should receive the click. In details: the webdriver is scrolling through google search and it must click every website it finds but the program is preventing that. How can I make it NOT search the previous site it clicked?
This is the function. The program is looping it and after the first loop it scrolls down and the error occurs:
def get_info():
browser.switch_to.window(browser.window_handles[2])
description = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.TAG_NAME, "h3"))
).text
site = WebDriverWait(browser, 10).until(
EC.presence_of_element_located((By.TAG_NAME, "cite"))
)
site.click()
url=browser.current_url
#removes the https:// and the / of the url
#to get just the domain of the website
try:
link=url.split("https://")
link1=link[1].split("/")
link2=link1[0]
link3=link2.split("www.")
real_link=link3[1]
except IndexError:
link=url.split("https://")
link1=link[1].split("/")
real_link=link1[0]
time.sleep(3)
screenshot=browser.save_screenshot("photos/"+"(" + real_link + ")" + ".png")
global content
content=[]
content.append(real_link)
content.append(description)
print(content)
browser.back()
time.sleep(5)
browser.execute_script("window.scrollBy(0,400)","")
time.sleep(5)
You can create a list of clicked website and check every time if that link is clicked or not. Here's the demo code :
clicked_website=[]
url=browser.current_url
clicked_website.append(url)
# Now while clicking
if <new_url> not in clicked_website:
<>.click()
This is just an idea how to implement. Your code is mess, I didn't understand clearly so, implement in your code by yourself.
connections=driver.find_elements_by_css_selector("a span[class='mn-connection-card__name t-16 t-black t-bold']")
print(len(connections))
for connection in connections:
if connection.text == "XXX":
connection.click()
break
I am getting the following error in the if statement:
stale element reference: element is not attached to the page document
Stale element exception happen when properties of element on which your script is trying to perform some operation is changed. If you want to click a span with text "XXX" you can directly click on that:
WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, "//a[span[text()='XXX']]")))
If your requirement is to loop trough all such elements then:
connections=driver.find_elements_by_css_selector("a span[class='mn-connection-card__name t-16 t-black t-bold']")
print(len(connections))
for i in range(len(connections)):
connections=driver.find_elements_by_css_selector("a span[class='mn-connection-card__name t-16 t-black t-bold']") #Created Fresh element list, so it wont be stale
if connections[i].text == "XXX"
connections[i].click
break
Trying to break a bigger problem I have into smaller chunks
main question
I am currently inputting a boxer's name into an autocomplete box, selecting the first option that comes up (boxer's name) then clicking view more until I get a list of all the boxer's fights and the view more button stops appearing.
I am then trying to create a list of onclick hrefs I would like to click then iteratively clicking on each and getting the html from each page/fight. I would ideally want to extract the text in particular.
This is the code I have written:
page_link = 'http://beta.compuboxdata.com/fighter'
chromedriver = 'C:\\Users\\User\\Downloads\\chromedriver'
cdriver = webdriver.Chrome(chromedriver)
cdriver.maximize_window()
cdriver.get(page_link)
wait = WebDriverWait(cdriver,20)
wait.until(EC.visibility_of_element_located((By.ID,'s2id_autogen1'))).send_keys('Deontay Wilder')
wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'select2-result-label'))).click()
while True:
try:
element = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'view_more'))).click()
except TimeoutException:
break
# fighters = cdriver.find_elements_by_xpath("//div[#class='row row-bottom-margin-5']/div[2]")
links = [x.get_attribute('onclick') for x in wait.until(EC.visibility_of_element_located((By.XPATH, "//*[contains(#onclick, 'get_fight_report')]/a")))]
htmls = []
for link in links:
cdriver.get(link)
htmls.append(cddriver.page_source)
Running this however gives me the error message:
ElementClickInterceptedException Traceback (most recent call last)
<ipython-input-229-1ee2547c0362> in <module>
10 while True:
11 try:
---> 12 element = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'view_more'))).click()
13 except TimeoutException:
14 break
ElementClickInterceptedException: Message: element click intercepted: Element <a class="view_more" href="javascript:void(0);" onclick="_search('0')"></a> is not clickable at point (274, 774). Other element would receive the click: <div class="col-xs-12 col-sm-12 col-md-12 col-lg-12">...</div>
(Session info: chrome=78.0.3904.108)
UPDATE
I have tried looking at a few answers with similar error messages and tried this
while True:
try:
element = cdriver.find_element_by_class_name('view_more')
webdriver.ActionChains(cdriver).move_to_element(element).click(element).perform()
# element = wait.until(EC.visibility_of_element_located((By.CLASS_NAME, 'view_more'))).click()
except TimeoutException:
break
links = [x.get_attribute('onclick') for x in wait.until(EC.visibility_of_element_located((By.XPATH, "//*[contains(#onclick, 'get_fight_report')]/a")))]
htmls = []
for link in links:
cdriver.get(link)
htmls.append(cddriver.page_source)
but this seems to create some sort of infinite loop at the ActionChains point. Seems to be constantly waiting for the view more href to appear
click function should already move the window so the element is in the viewable window. So you don't need that action chain (I think...) but the original error shows some other element OVER the view more button.
You may need to remove (or hide) this element from the DOM, or if it's a html window, "dismiss" it. So pinpointing this covering element is key and then deciding on a strategy to uncover the view more button.
Your site http://beta.compuboxdata.com/fighter doesn't seem to be working at the time so I can't dig in further.
I want to locate the elements of a popup on some page,
the popup html is written in an iframe,
also the popup is triggered by clicking a link on the main page.
The weird thing is, if I run the whole code, I can not locate the 'target' element:
dr = webdriver.Chrome('chromedriver.exe', options=chrome_options)
modify = (By.CLASS_NAME, "modify")
ec_visible(dr, modify).click()
popup = (By.CLASS_NAME, "add-addr-iframe")
dr.switch_to.frame(ec_visible(dr, popup))
target = (By.CLASS_NAME, "cndzk-entrance-division-header-click")
ec_visible(dr, target).click()
def ec_visible(driver, locator):
return WebDriverWait(driver, 5).until(EC.visibility_of_element_located(locator))
But, if I first open the popup then locate, it works.
First:
modify = (By.CLASS_NAME, "modify")
ec_visible(dr, modify).click()
#popup = (By.CLASS_NAME, "add-addr-iframe")
#dr.switch_to.frame(ec_visible(dr, popup))
#target = (By.CLASS_NAME, "cndzk-entrance-division-header-click")
#ec_visible(dr, target).click()
Then: (works too if I manually open the popup and run this code)
#modify = (By.CLASS_NAME, "modify")
#ec_visible(dr, modify).click()
popup = (By.CLASS_NAME, "add-addr-iframe")
dr.switch_to.frame(ec_visible(dr, popup))
target = (By.CLASS_NAME, "cndzk-entrance-division-header-click")
ec_visible(dr, target).click()
Appreciate if you can point out my problem!
Here is the exception from shell:
raise TimeoutException(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message:
This is the html screenshot,
sometimes the content in iframe can not even be seen.
iframe not extendable
when iframe extendable
7/26 update
I am wondering whether I asked the right question which may lead you guys just focusing on my code part. Since my code works(seperately), the elements and frames approach are good.
I step back and find one detail which may help but I don't know how it matters.
Here are the two shots about ctrl-F some element in the page source:
Normal result: target found and highlighted
Weird result: target found and no highlight
I mean when the page exists 'weird result', my code does not work.
PS. The page is the order-confirmation part of an e-commerical site, but the site groups its goods into two types which led to TWO types order page.
For ifarmes you have frame_to_be_available_and_switch_to_it as the EC.
So try this:
WebDriverWait(driver, 5).until(EC.frame_to_be_available_and_switch_to_it((By.CLASS_NAME, "add-addr-iframe")))
target = (By.CLASS_NAME, "span.cndzk-entrance-division-header-click")
ec_visible(dr, target).click()