Develop Reference

Python element click intercepted Error? and advice on how click a view more button

I have been writing a simple code for a 5 days now which I am doing to improve my knowledge on web scraping using different packages, I have already wrote one that downloads all the URL's and has the choice of downloading all images or videos but when I visited 'https://www.pixwox.com/' it has a different html design where the urls are hidden so I googled and started using Selenium. It was all going well until I hit a wall and the limits of my python knowledge.
The error below is what I have been getting for about 4 of them days, sometimes the code will work fine and others it will show the error below:
selenium.common.exceptions.ElementClickInterceptedException: Message: element click intercepted: Element <a class="downbtn" href="https://scontent-frt3-2.cdninstagram.com/v/t51.2885-15/313202955_1363105014229232_3490221399588911023_n.jpg?stp=dst-jpg_e35_p828x828&_nc_ht=scontent-frt3-2.cdninstagram.com&_nc_cat=108&_nc_ohc=cB6igIdJd0UAX-sHSgL&edm=ACWDqb8BAAAA&ccb=7-5&ig_cache_key=Mjk2MDU4OTQ5NjY4NTI4NzQwMQ%3D%3D.2-ccb7-5&oh=00_AfAL1tPs2in8qcStQLZMdlDGZxdNRp5H5nnV4LpHWR07gg&oe=6363A260&_nc_sid=1527a3&dl=1">...</a> is not clickable at point (156, 814). Other element would receive the click: <iframe id="h12_frm_bl9ib7ijd9k" scrolling="no" frameborder="0" width="100%" height="auto" marginheight="0" marginwidth="0" style="margin: 0px; padding: 0px; width: 100%; height: 84.6186px; overflow: hidden;"></iframe>
(Session info: chrome=106.0.5249.119)
I know this means that the element I'm trying to click on has another element currently over top of it but currently even with stepping through the code I have not been able to see what is covering the element.
View more
My code is below, Please excuse the code for being messy, I am still learning.
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.ui import WebDriverWait
option = webdriver.ChromeOptions()
option.add_argument(" — incognito")
#option.add_argument("start-maximized")
#option.add_argument("--window-size=1400,600")
user = input("User: ")
full_url = 'https://www.pixwox.com/profile/' + user + '/'
driver = webdriver.Chrome()
driver.get(full_url)
print(driver.title)
i = 0
while i < int(12):
driver.execute_script("window.scrollTo(0,document.body.scrollHeight);")
i += 1
time.sleep(2)
download_button = "downbtn"
WebDriverWait(driver, 10).until(ec.presence_of_element_located((By.CLASS_NAME, download_button)))#.click()
elements = driver.find_elements(By.CLASS_NAME, download_button)
time.sleep(5)
for element in elements:
if element.text == 'Download': element.click()##### Added this sleep time ? time.sleep(2)
print(f"Dowdloading: {element}")
time.sleep(5)
else:
pass
also I know compared to other peoples code this is quite basic but I am still learning so any help would be great to further my learning, please excuse the while loop, I didn't know how else to keep scrolling down without using it.
Fix's tried:
Tried extending the sleep time to see whether it is a loading issue.
Tried to scroll down to see if that was causing the issue but that raises another issue ie the view more button which is another obstacle.
Stepped through the code using Thonny but was not able to find what is intercepting the element.
Tried https://stackoverflow.com/questions/44724185/element-myelement-is-not-clickable-at-point-x-y-other-element-would-receiv
Tried https://stackoverflow.com/questions/62260511/org-openqa-selenium-elementclickinterceptedexception-element-click-intercepted
I was expecting my code to be able to take a user input and download all images and videos on the whole page, currently it is very intermittent 99% of the time it downloads one image and then raises an 'ElementClickInterceptedException' error, that 1% downloads roughly 50 images and videos before raising the same error. I was also expecting it to scroll to the very bottom of the page so all images/videos load but the
View more
button stops the code from continuing.
Any help would be greatly appreciated.
Thank you

Identifying html structures with data-v-xxxxxxxx and pressing them using selenium

Trying to identify a javascript button on a website and press it to extend the page.
The website in question is the tencent appstore after performing a basic search. At the bottom of the page is a button titled "div.load-more-new" where upon pressing will extend the page with more apps.
the html is as follows
<div data-v-33600cb4="" class="load-more-btn-new" style="">
<a data-v-33600cb4="" href="javascript:void(0);">加载更多
<i data-v-33600cb4="" class="load-more-icon">
</i>
</a>
</div>
At first I thought I could identify the button using BeautifulSoup but all calls to find result as empty.
from selenium import webdriver
import BeautifulSoup
import time
url = 'https://webcdn.m.qq.com/webapp/homepage/index.html#/appSearch?kw=%25E7%2594%25B5%25E5%25BD%25B1'
WebDriver = webdriver.Chrome('/chromedriver')
WebDriver.get(url)
time.sleep(5)
# Find using BeuatifulSoup
soup = BeautifulSoup(WebDriver.page_source,'lxml')
button = soup.find('div',{'class':'load-more-btn-new'})
[0] []
After looking around here, it became apparent that even if I could it in BeuatifulSoup, it would not help in pressing the button. Next I tried to find the element in the driver and use .click()
driver.find_element_by_class_name('div.load-more-btn-new').click()
[1] NoSuchElementException
driver.find_element_by_css_selector('.load-more-btn-new').click()
[2] NoSuchElementException
driver.find_element_by_class_name('a.load-more-new.load-more-btn-new[data-v-33600cb4]').click()
[3] NoSuchElementException
but all return with the same error: 'NoSuchElementException'

Your selections wont work, cause they do not point on the <a>.
This one selects by class name and you try to click the <div> that holds your <a>:
driver.find_element_by_class_name('div.load-more-btn-new').click()
This one is very close but is missing the a in selection:
driver.find_element_by_css_selector('.load-more-btn-new').click()
This one try to find_element_by_class_name but is a wild mix of tag, attribute and class:
driver.find_element_by_class_name('a.load-more-new.load-more-btn-new[data-v-33600cb4]').click()
How to fix?
Select your element more specific and nearly like in your second apporach:
driver.find_element_by_css_selector('.load-more-btn-new a').click()
or
driver.find_element_by_css_selector('a[data-v-33600cb4]').click()
Note:
While working with newer selenium versions you will get DeprecationWarning: find_element_by_ commands are deprecated. Please use find_element()*
from selenium.webdriver.common.by import By
driver.find_element(By.CSS_SELECTOR, '.load-more-btn-new a').click()

Python selenium ElementNotInteractableException: Message: element not interactable

I am trying to login to beeradvocate.com to scrape (crawl) some beer data.
I tried with selenium but have been brutally failing.
here is the html
<input type="text" name="login" value="" id="ctrl_pageLogin_login" class="textCtrl" tabindex="1" autofocus="autofocus" style="background-image: url("data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR4nGP6zwAAAgcBApocMXEAAAAASUVORK5CYII="); cursor: auto;">
i tried using name and value and class but everything has failed.
I attempted Xpath as my final try, but have failed as well.
website and inspection
My code:
driver=webdriver.Chrome("~~~~\\chromedriver.exe")
driver.get("https://www.beeradvocate.com/community/login/")
from selenium.common.exceptions import TimeoutException
driver.maximize_window()
while True:
try:
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.XPATH,'//*[#id="ctrl_pageLogin_login"]'))).send_keys("scentmaster")
break
except TimeoutException:
print("too much time")
I've made the button work with:
button = driver.find_element_by_xpath('//*[#id="pageLogin"]/dl[3]/dd/input')
driver.execute_script("arguments[0].click();", button)
However, I need to be able to perform sent_keys to type in id and pw to log in...
Anybody got some idea?

There are 2 fields on the page if you use xpath //*[#id = "ctrl_pageLogin_login"], the input field you are referring to is the second. Sadly, the selenium find element by default refers to the first. It will work if you make it like this: (//*[#id = "ctrl_pageLogin_login"])[2].
But I have another suggestion, try to locate element by css selector with this value : form#pageLogin input#ctrl_pageLogin_login
WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.CSS_SELECTOR,'form#pageLogin input#ctrl_pageLogin_login'))).send_keys("scentmaster")
#for password
driver.find_element_by_css_selector('form#pageLogin input#ctrl_pageLogin_password').send_keys('your_password')
#for submit
driver.find_element_by_css_selector('form#pageLogin input[type=submit]').click()

To feed the username, try this xpath:
'//form/dl/dd/input[#id="ctrl_pageLogin_login"]'
for the password:
'//form/dl/dd/input[#id="ctrl_pageLogin_password"]'

WebDriverWait(driver, 30).until(EC.element_to_be_clickable((By.CSS_SELECTOR,'form#pageLogin input#ctrl_pageLogin_login'))).send_keys("scentmaster")
#for password
driver.find_element_by_css_selector('form#pageLogin input#ctrl_pageLogin_password').send_keys('your_password')
#for submit
driver.find_element_by_css_selector('form#pageLogin input[type=submit]').click()
The solution above given by frianH worked! :)

How to click the icon using selenium

I am trying to automate my certain activities using selenium. I was launching a webpage and a security windows popup appears for the login credentials.
I have automated that using Autoit. Now after login I need to click on the option and I have tried it based on the find_element_by_text and find_element_by_id.
I was getting an error as Unable to find the element with CSS selector and I have seen some other post in the StackOverflow with the same issue but I could not able to fix it by myself.
Here is how my HTML looks like. Could you please guide me on this and also please share any document for further checking. Thanks.
driver = webdriver.Ie(r"C:\\IEDriverServer\\IEDriverServer.exe")
driver.get('URL of the page')
#driver.implicitly_wait(10) # seconds
autoit.win_wait("Windows Security")
# now make that crap active so we can do stuff to it
autoit.win_activate("Windows Security")
# type in the username
autoit.send('username')
# tab to the password field
autoit.send("{TAB}")
# type in the password
autoit.send('password')
# kill!
autoit.send("{ENTER}")
driver.maximize_window()
driver.implicitly_wait(120) # seconds
#submit_button_incidents = driver.find_element_by_link_text("3-Normal Incidents")
submit_button_incidents= driver.find_element_by_id("nodeImgs13pm")
submit_button_incidents.click()
driver.implicitly_wait(10)
++ updating the info
I have tried to copy the whole HTML but the page was restricted so I cant able to view the full HTML page other than the basic templates. Adding some more screenshots of the developer tools.
Here how my webpage looks like.

try with this code :
submit_button_incidents = WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.XPATH, "//span[contains(text(),'My Group'])")))
submit_button_incidents.click()
and do not use implicit wait too many times in code.
Implicit wait is set for life time of web driver instance.
Reference :
Selenium official document for wiats
Xpath tutorial
cssSelector tutorial
UPDATE :
As OP has shared the HTML code. As per the requirement you can go ahead with this code.
As there are two elements with text as My Groups's queue.
For first one you can write XPATH as :
//a[#class='scbdtext']/span[contains(text(),'My Group')]
Code:
submit_button_incidents = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='scbdtext']/span[contains(text(),'My Group')]")))
submit_button_incidents.click()
For second one you can write XPATH as :
//a[#class='scbdtextfoc']/span[contains(text(),'My Group')]
Code :
submit_button_incidents = WebDriverWait(driver, 10).until(
EC.element_to_be_clickable((By.XPATH, "//a[#class='scbdtextfoc']/span[contains(text(),'My Group')]")))
submit_button_incidents.click()
Hope this will help.

Use ActionChains with double click for this to work in Python.
from selenium.webdriver import ActionChains
# Get the element however you want
element = driver.find_element_by_xpath("//a[#class='scbdtextfoc']/span[contains(text(),'My Group')]")
ActionChains(driver).double_click(settings_icon).perform()

Clicking Specific button with Selenium

I am trying to click a particular button with Selenium in Python, but am having trouble identifying that particular button. For example, if I was on the google page of this, and I wanted to have the translation bar drop down, how would I go about referencing that specific element. Inspecting it in my browser I see some of what I assume to be its data as:
<div style="clear: both;" aria-controls="uid_0" aria-expanded="false"
class="_LJ _qxg xpdarr _WGh vk_arc" data-fbevent="fastbutton" jsaction="kx.t;
fastbutton: kx.t" role="button" tabindex="0" data-ved="0ahUKEwiwn-6K17XLAhVLWD4KHTk9CTkQmDMILzAA">
However, from this point I'm not sure how I would use the find element by functions to reference what I need to in order to call it properly.
driver.find_element_by_*("?").click()
import unittest
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
#comment
print ("Let's talk about Python.")
driver = webdriver.Firefox()
driver.get("http://www.google.com")
assert "Google" in driver.title
elem = driver.find_element_by_name("q")
elem.send_keys("ignominious")
elem.send_keys(Keys.RETURN)
driver.find_element_by_*("?").click()
assert "No results found." not in driver.page_source
driver.close()

You can use css_selector with the class attribute
driver.find_element_by_css_selector("._LJ._qxg.xpdarr._WGh.vk_arc").click()
Or class_name with any one of the classes
driver.find_element_by_class_name("_LJ").click()
# or
driver.find_element_by_class_name("_qxg").click()
# or
driver.find_element_by_class_name("xpdarr").click()
# or
driver.find_element_by_class_name("_WGh").click()
# or
driver.find_element_by_class_name("vk_arc").click()
Sending click to the element child will also work
driver.find_element_by_class_name("vk_ard").click()

For a better maintainability you should try to work with ids.
With your example the selector would be:
driver.find_element_by_css_selector("#uid_1 > div[role='button']").click()

Do you want to click on arrow.
If yes then below code is working for me:-
driver.manage().timeouts().implicitlyWait(50, TimeUnit.SECONDS);
driver.get("https://www.google.com/");
driver.findElement(By.name("q")).sendKeys("ignominious");
driver.findElement(By.name("q")).sendKeys(Keys.RETURN);
driver.findElement(By.className("vk_ard")).click();
Hope it will help you :)