Develop Reference

Identifying html structures with data-v-xxxxxxxx and pressing them using selenium

Trying to identify a javascript button on a website and press it to extend the page.
The website in question is the tencent appstore after performing a basic search. At the bottom of the page is a button titled "div.load-more-new" where upon pressing will extend the page with more apps.
the html is as follows
<div data-v-33600cb4="" class="load-more-btn-new" style="">
<a data-v-33600cb4="" href="javascript:void(0);">加载更多
<i data-v-33600cb4="" class="load-more-icon">
</i>
</a>
</div>
At first I thought I could identify the button using BeautifulSoup but all calls to find result as empty.
from selenium import webdriver
import BeautifulSoup
import time
url = 'https://webcdn.m.qq.com/webapp/homepage/index.html#/appSearch?kw=%25E7%2594%25B5%25E5%25BD%25B1'
WebDriver = webdriver.Chrome('/chromedriver')
WebDriver.get(url)
time.sleep(5)
# Find using BeuatifulSoup
soup = BeautifulSoup(WebDriver.page_source,'lxml')
button = soup.find('div',{'class':'load-more-btn-new'})
[0] []
After looking around here, it became apparent that even if I could it in BeuatifulSoup, it would not help in pressing the button. Next I tried to find the element in the driver and use .click()
driver.find_element_by_class_name('div.load-more-btn-new').click()
[1] NoSuchElementException
driver.find_element_by_css_selector('.load-more-btn-new').click()
[2] NoSuchElementException
driver.find_element_by_class_name('a.load-more-new.load-more-btn-new[data-v-33600cb4]').click()
[3] NoSuchElementException
but all return with the same error: 'NoSuchElementException'

Your selections wont work, cause they do not point on the <a>.
This one selects by class name and you try to click the <div> that holds your <a>:
driver.find_element_by_class_name('div.load-more-btn-new').click()
This one is very close but is missing the a in selection:
driver.find_element_by_css_selector('.load-more-btn-new').click()
This one try to find_element_by_class_name but is a wild mix of tag, attribute and class:
driver.find_element_by_class_name('a.load-more-new.load-more-btn-new[data-v-33600cb4]').click()
How to fix?
Select your element more specific and nearly like in your second apporach:
driver.find_element_by_css_selector('.load-more-btn-new a').click()
or
driver.find_element_by_css_selector('a[data-v-33600cb4]').click()
Note:
While working with newer selenium versions you will get DeprecationWarning: find_element_by_ commands are deprecated. Please use find_element()*
from selenium.webdriver.common.by import By
driver.find_element(By.CSS_SELECTOR, '.load-more-btn-new a').click()

Selenium click on button trouble (Python web scraping)

I am trying to web scraping a popular movie database on the Internet. After a few thousands of requests, my spider is detected and I have to manually click on a simple button to continue my data collection.
I have tried to automatize this process by clicking automatically in that button. For that purpose, I am using Selenium library. I have located the xpath of the button, but, for some reason, .click() method doesn't work.
Here is my code:
def click_button():
options = Options()
options.headless = True
options.add_argument("--window-size=1920,1200")
DRIVER_PATH = 'geckodriver'
driver = webdriver.Firefox(options=options, executable_path=DRIVER_PATH)
driver.get('https://www.filmaffinity.com/es/main.html')
element = driver.find_element(By.XPATH,"//input[#value='Send']")
element.click()
driver.quit()
Also, I have tried other common alternatives, such as waiting until the element gets clickable or visible, but this didn't work. I have also tried to execute the click as a javascript script.
This is the window I see when my spider is detected:
Too many request window (picture)
As you see, there is a reCaptcha-protection icon on the lower right corner, but I don't have to solve any captcha puzzle to confirm I am not a robot, I have just to click on the send button you can see on the picture.
The html which contains the button is the following one:
<div class="content">
<h1>Too many requests</h1>
<div class="image">
<img height="400" src="/images/too-many-request.jpg">
</div>
<div class="text">
Are you sure you do not dream of electric sheep?
</div>
<form name="toomanyrequest">
<div class="alert">
please enter the Captcha to continue.
</div>
<div>
<input type="submit" value="Send">
</div>
</form>
</div>
Which do you think is the trouble with my code or approach? I have to find a way to click on that button to continue my scraping, but I don't know what I am doing wrong (I have little experience at this field).
Maybe the xpath is not correct? Maybe the captcha protection is blocking the click action? I don't get any exception when I execute my code; but nothing happens and the "Too many request" window does not disappear.
Thank you very much.

Try this:
driver.execute_script("arguments[0].click()", element)
or:
driver.execute_script("arguments[0].click();", element)

Selenium driven browser shows different HTML code

I am not a very experienced coder so apologies if I say smth stupid.
I am using Python (in Spyder) to get Selenium to fill in a website form containing username and password. Here's the target - link.
When I lookup the "username" element by pressing F12 in a regular browser I get the following:
<input class="slds-input input" type="text" aria-describedby="" placeholder="Username" id="172:0" data-aura-rendered-by="176:0" data-interactive-lib-uid="2">
So I attempt to locate the element using the ID. However when I run the script, I get the following error in Chrome:
NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"[id="172:0"]"}
Same when I run it in Firefox instead:
NoSuchElementException: Unable to locate element: [id="172:0"]
When I check HTML in the Selenium driven browser, I can see that the page code is (ie element ID) different, as below
<input class="slds-input input" type="text" aria-describedby="" placeholder="Username" id="78:2;a" data-aura-rendered-by="82:2;a" data-interactive-lib-uid="2">
My best guess is that the difference in HTML code is the reason for error. I found people posting similar issues but those were slightly different and I was not able to solve my issue using the solutions proposed there. I would appreciate is someone could help with my case.

Use xpath instead of id since it changes dynamically
Xpath for UserName: //label/following-sibling::input
Xpath for Password: //lightning-input//div//input
Sample working code which works in java convert with using above xpath in python and also add implicitlyWait and pageLoadTimeout befor launching the website
WebDriver driver = new ChromeDriver();
driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://lta-tennis.force.com/"); // WebElement
driver.findElement(By.xpath("//label/following-sibling::input")).sendKeys("dummy");
driver.findElement(By.xpath("//lightning-input//div//input")).sendKeys("dummy");
System.out.println(driver.getTitle());
Edit 1: Based on OP comment
This is working xpath

try these xpaths
//input[#placeholder="Username"]
//input[#placeholder="Password"]
here is the full code
from selenium import webdriver
import time
browser = webdriver.Chrome('C:\\driverpath\\chromedriver.exe')
url = 'https://lta-tennis.force.com/s/login/'
get = browser.get(url)
time.sleep(5)
browser.find_element_by_xpath('//input[#placeholder="Username"]').send_keys('hello')
browser.find_element_by_xpath('//input[#placeholder="Password"]').send_keys('pass')

Using Python & Selenium to send message to friend in Facebook

I am currently writing a script in Python to log in on facebook and I want to send message to a friend. The script logs in to my facebook, and it manages to find my friend, but the message is not sent. I'm not 100% sure, but I think the problem is in the div tag/CSS on the text area (I commented out that piece of code).
Screenshot:
Text doesn't appear here
Here is my code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('C:\path\chromedriver.exe')
driver.get('https://www.facebook.com/')
username_box = driver.find_element_by_id('email')
username_box.send_keys(USEREMAIL)
passElem = driver.find_element_by_id("pass")
passElem.send_keys(USERPASSWORD)
passElem.send_keys(Keys.RETURN)
userTargetUrl = "https://www.facebook.com/messages/t/" + "USERTAGET"
driver.get(userTargetUrl)
//The problem is here I think
elem = driver.find_element_by_css_selector("div textarea.uiTextareaNoResize")
while True:
elem.send_keys("Test")
elem.send_keys(Keys.RETURN)
driver.find_element_by_id("u_0_t").click()
The error code i get is:
selenium.common.exceptions.NoSuchElementException:
Message: no such element: Unable to locate element:
{"method":"css selector","selector":"div textarea.uiTextareaNoResize"}
(Session info: chrome=63.0.3239.132)
(Driver info: chromedriver=2.35.528161
(5b82f2d2aae0ca24b877009200ced9065a772e73),platform=Windows NT 10.0.15063 x86_64)

I solved this problem on RU SO)))
Briefly in steps:
Making the library import
from selenium.webdriver.common.action_chains import ActionChains
Visited the site
Logged in
You have a friend list, you clicked on your friend's icon and a chat window has opened.
now what to do?))
You need to place the cursor in the input line, in which the text is not written) Use for this .click (), you can use any of your XPath, but here is a shorter one))
driver.find_element_by_xpath("//*[#data-editor]").click()
After pressing the .click () method, the cursor is placed in the text input area. And now we need to enter the text, but use .send_keys() not with the pointer to the element, as you tried, but separately as an action (this is what the imported library is for)
actions = ActionChains(driver)
actions.send_keys('HI')
actions.perform()
Woo a la))))
Well, after that .click() on the send icon or press Enter)))))

Try to find the element by name, id, text or any other unique parameter - it would be better (they sometimes change the design and all xpaths and css-selectors become useless).
I made bold what I'd use:
<div aria-autocomplete="list" aria-controls="js_7" aria-describedby="js_0"
aria-expanded="false" <b>aria-label="Введите сообщение..."</b>
class="notranslate _5rpu" role="combobox" spellcheck="true"
style="outline: medium none currentcolor; -moz-user-select: text;
white-space: pre-wrap; overflow-wrap: break-word;" <b>tabindex="9999"<b>
<b>id="js_d"</b> contenteditable="true"><div data-contents="true">

I m not able to click on a button in a web page using selenium in python

What is wrong in the below code
import os
import time
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("http://x.x.x.x/html/load.jsp")
elm1 = driver.find_element_by_link_text("load")
time.sleep(10)
elm1.click()
time.sleep(30)
driver.close()
The page source is
<body>
<div class="formcenterdiv">
<form class="form" action="../load" method="post">
<header class="formheader">Loader</header>
<div align="center"><button class="formbutton">load</button></div>
</form>
</div>
</body></html>
I want to click on button load. when I ran the above code getting this error
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: load

As the documentation says, find_elements_by_link_text only works on a tags:
Use this when you know link text used within an anchor tag. With this
strategy, the first element with the link text value matching the
location will be returned. If no element has a matching link text
attribute, a NoSuchElementException will be raised.
The solution is to use a different selector like find_element_by_class_name:
elm1 = driver.find_element_by_class_name('formbutton')

Did you try using Xpath?
As the OP said, find_elements_by_link_text works on a tags only:
Below code might help you out
driver.get_element_by_xpath("/html/body/div/form/div/button")