Develop Reference

Selenium driven browser shows different HTML code

I am not a very experienced coder so apologies if I say smth stupid.
I am using Python (in Spyder) to get Selenium to fill in a website form containing username and password. Here's the target - link.
When I lookup the "username" element by pressing F12 in a regular browser I get the following:
<input class="slds-input input" type="text" aria-describedby="" placeholder="Username" id="172:0" data-aura-rendered-by="176:0" data-interactive-lib-uid="2">
So I attempt to locate the element using the ID. However when I run the script, I get the following error in Chrome:
NoSuchElementException: no such element: Unable to locate element: {"method":"css selector","selector":"[id="172:0"]"}
Same when I run it in Firefox instead:
NoSuchElementException: Unable to locate element: [id="172:0"]
When I check HTML in the Selenium driven browser, I can see that the page code is (ie element ID) different, as below
<input class="slds-input input" type="text" aria-describedby="" placeholder="Username" id="78:2;a" data-aura-rendered-by="82:2;a" data-interactive-lib-uid="2">
My best guess is that the difference in HTML code is the reason for error. I found people posting similar issues but those were slightly different and I was not able to solve my issue using the solutions proposed there. I would appreciate is someone could help with my case.

Use xpath instead of id since it changes dynamically
Xpath for UserName: //label/following-sibling::input
Xpath for Password: //lightning-input//div//input
Sample working code which works in java convert with using above xpath in python and also add implicitlyWait and pageLoadTimeout befor launching the website
WebDriver driver = new ChromeDriver();
driver.manage().timeouts().pageLoadTimeout(10, TimeUnit.SECONDS);
driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS);
driver.get("https://lta-tennis.force.com/"); // WebElement
driver.findElement(By.xpath("//label/following-sibling::input")).sendKeys("dummy");
driver.findElement(By.xpath("//lightning-input//div//input")).sendKeys("dummy");
System.out.println(driver.getTitle());
Edit 1: Based on OP comment
This is working xpath

try these xpaths
//input[#placeholder="Username"]
//input[#placeholder="Password"]
here is the full code
from selenium import webdriver
import time
browser = webdriver.Chrome('C:\\driverpath\\chromedriver.exe')
url = 'https://lta-tennis.force.com/s/login/'
get = browser.get(url)
time.sleep(5)
browser.find_element_by_xpath('//input[#placeholder="Username"]').send_keys('hello')
browser.find_element_by_xpath('//input[#placeholder="Password"]').send_keys('pass')

how to get anchor tag href attribute when it's hidden with selenium python

I'm currently using Selenium to make a simple python crawler.
Is there any method that I can get a link address(url) of a anchor tag? When I see the html source, it's hidden just like :
<a id='foo' href='#'></a>
I can actually click and load the page and get the url address, but then I need to wait a while.

For HTML you have shared :
<a id='foo' href='#'></a>
You can have a web element like this :
element = driver.find_element_by_id('foo')
once you have the web element, you can get the attribute like this :
href_val = element.get_attribute("href")
print(href_val)

Click a button using Selenium and Python

I have the following code:
<a class="sectionname" href="#" onclick="expandAll();return false;">Expand all</a>
When I click on expand all, the whole page loads. How can I do it using WebDriver for Python?

As per the HTML you can use the find_element_by_link_text and invoke click() method as follows :
driver.find_element_by_link_text("Expand all").click()
You can get more granualar with find_element_by_xpath as follows :
driver.find_element_by_xpath("//a[#class='sectionname' and contains(.,'Expand all')]").click()
Update
As you still don't see the expansion you can try the Javascript way as follows :
myElement = driver.find_element_by_xpath("//a[#class='sectionname' and contains(.,'Expand all')]")
driver.execute_script("arguments[0].click();", myElement)

How to click automatically on "Content" button

This is element HTML:
<button class="search-vertical-filter__filter-item-button button-tertiary-medium-muted" data-ember-action="" data-ember-action-5975="5975"> Content </button>
Code:
driver.find_element_by_link_text('Content').click()
Even i also tried:
driver.find_element_by_class_name('search-vertical-filter__filter-item-button button-tertiary-medium-muted').click()
I want to click on "Content" button on LinkedIn automatically and I am using Selenium and Python for that.

Try to use the following code:
driver.find_element_by_css_selector(".search-vertical-filter__filter-item-button.button-tertiary-medium-muted").click()
Hope it helps you!

As per the HTML provided to click on the button with text as Content you can use the following line of code :
driver.find_element_by_xpath("//button[#class='search-vertical-filter__filter-item-button button-tertiary-medium-muted' and normalize-space()='Content']").click()

Python selenium - get element by its xPath and access it

I am using Python and Selenium to scrape a webpage, in some cases, I can't get it to work,
*
I would like to access the element with text 'PInt', wich is the second link in the below code.
The xPath for it (copied from Developer console) is: //[#id="submenu1"]/a[2]
<div id="divTest" onscroll="SetDivPosition();" style="height: 861px;">
<div class="menuTitle" id="title1">
</div>
<div class="subtitle" id="submenu1">
<img src="images/spacer.gif" border="0" width="2px" height="12px">
Mov<br>
<img src="images/spacer.gif" border="0" width="2px" height="12px">
PInt<br>
<img src="images/spacer.gif" border="0" width="2px" height="12px">
SWAM / SWIF<br>
</div>
...
A snipped of my code is:
try:
res = driver.find_elements_by_link_text('PInt')
print("res1:{}".format(res))
res = driver.find_element(By.XPATH,'//*[#id="submenu1"]/a[3]')
print("res:{} type[0]:{}".format(res,res[0]))
itm1 = res[0]
itm1.click()
I get the error:
Unable to locate element:
{"method":"xpath","selector":"//*[#id="submenu1"]/a[2]"}
My question is, how can I get the right xPath of the element or any other way to access the element?
UPDATE:
This might be important, the issue with
Message: invalid selector: Unable to locate an element with the xpath expression (and I've tried all proposed solutions) might be that this is after authenticate in the webpage (User + Pwd) before, everything works.
I noticed that the url driver.current_url after login is static (asp page).
Also this part I am trying to access in a frameset and frame
html > frameset > frameset > frame:nth-child(1)

Thanks to #JeffC to point me in the right direction.
since the page has some frames, I manage to access the element first by switching to the right frame (using xPath)
and then access the element.
driver.switch_to.default_content()
driver.switch_to.frame(driver.find_element_by_xpath('html / frameset / frameset / frame[1]'))
driver.find_element_by_xpath("//a[contains(text(),'PInt')]").click()
BTW, in case you want to run the script from a crontab,you need to setup a display:
30 5 * * * export DISPLAY=:0; python /usr/.../main.py

To see a full list of all the ways of selecting elements using selenium, you can read all about it in the documentation.
Using xpath:
res = driver.find_element_by_xpath(u'//*[#id="submenu1"]/a[2]')
Using css selector:
res = driver.find_element_by_css_selector('#submenu1 a:nth-of-type(2)')

Try with any of the below xpath. Sometimes automatically generated xpath does not work.
//a[contains(text(),'PInt')]
or
//div[#id='submenu1']//a[contains(text(),'PInt')]
Also I would suggest you to set some wait time before clicking on above link in case if above xpath does not work

To find xPath in chrome:
Right click the element you want
Inspect, which will open the developer window and highlight the selected element.
Right click the highlighted element and choose copy > copy by xpath
Here is a list of all the different ways to locate an element Locating Elements