Develop Reference

Unable to grab selenium element for one particular class name

So I am trying to find elements by CSS selector using the class name. One of my code works while the other does not.
Screenshot of the html code that works:
The code for the above screenshot, that works:
driver.get(url)
print('a') # to check if code has run till here
disabled_next = driver.find_elements(By.CSS_SELECTOR, '.page-link.next.disabled')
print(disabled_next)
Screenshot of the html code that does NOT work:
The code for the above screenshot, that does not work:
driver.get(url)
print('a')
enabled_next = driver.find_elements(By.CSS_SELECTOR, '.page-link.next')
print(enabled_next)
Just trying to understand why the second one does not work. It does not even print ('a'). The error I am getting is along the lines of this:
selenium.common.exceptions.WebDriverException: Message: unknown error: cannot determine loading status
from unknown error: unexpected command response
(Session info: chrome=103.0.5060.66)
I am using chromedriver103 with chrome version 103. I know there have been some issues relating to it. Could that be why? Or have I used the wrong format for '.page-link.next'.
Also, a semi-related question: How do I grab the text after <a class = "text here"? I would like to just grab "text here". So in my examples above I would love to grab "page-link next disabled" or "page-link next".
Thank you in advance!

This is not related to your code at all, your code should do fine. The error message pertains to the chromedriver version and the current chrome browser you are using. Try to double check the chromedriver and chrome browser you are using for testing.
cannot determine loading status

Unable to switch to iFrame using Selenium

I'm currently trying out Selenium for Python to do some web automation.
I got stuck a the checkout where you have to enter your credit card data.
I'm aware that I have to switch to iFrame to locate the input box.
But I was never able to switch the frame.
I tried it for ID or for example for this:
time.sleep(4)
frame = self.driver.find_element_by_xpath("//iframe[#name= '__privateHeidelpayFrame--heidelpay-holder-iframe-1616489954834']")
self.driver.switch_to.frame(frame)
name_input = self.driver.find_element_by_id("card-holder")
name_input.clear()
name_input.send_keys(credit_card.NAME)
I'm always getting "Unable to locate element : {"method":"xpath","selector":"//iframe[#name= '__privateHeidelpayFrame--heidelpay-holder-iframe-1616489954834']"}".
I added a pic from the Source Code.
Source Code from the Webpage
I'm thankful for any advise!
Best, Jannic

Possibly id & name attribute is dynamic.
Try with starts-with() or use other attribute like class to identify the iframe.
frame = self.driver.find_element_by_xpath("//iframe[starts-with(#name, '__privateHeidelpayFrame--heidelpay-holder-iframe-')]")
self.driver.switch_to.frame(frame)
OR
frame = self.driver.find_element_by_xpath("//iframe[#class='heidelpayUIIframe']")
self.driver.switch_to.frame(frame)

Have you tried using a simpler API to find the iFrame?
Does
self.driver.find_element(By.NAME, "__privateHeidelpayFrame--heidelpay-holder-iframe-1616489954834'")
Return anything?

How do I get the element Datetime using selenium without using XPath?

how do i find that class datetime element using selenium.
not xpath please as that dont work on headless chrome and i do not want to see the browser showing up when i am coding this to run in background every hr.
i've tried this
pp = p.find_element_by_class_name('line-height-3')
print(pp.get_attribute('datetime'))
results
none
i've figured it out
Announcement = driver.find_element_by_class_name('discussion.subscribed')
if Announcement.is_displayed():
Announcement_id = Announcement.get_attribute('data-discussionid')
Announcement_title = Announcement.find_element_by_class_name('w-100.h-100.d-block').get_attribute('title')
TIME = Announcement.find_element_by_id('time-created-'+str(Announcement_id))
Time = TIME.get_attribute('datetime')
so i just putting it out incase anyone else came by such problem.
The reason why I am not using xpath is that whenever it runs in headless mode
i get nosucherror but in normal mode. it works fine.

Your id is dynamic so you can not use it. You can use cssSelector if xpath is not nice for you.
See more on https://www.browserstack.com/guide/css-selectors-in-selenium
I do not have more HTML of your page, but you can try with:
time[datatimeformat='part that is hard to see on your picture']

Try getting it by id.
pp = p.find_element_by_id('time-created-82672')
print(pp.get_attribute('datetime'))

You can use CSS selector
pp = p.find_element_by_css('div.author-info time')
print(pp.get_attribute('datetime'))
If you will change your mind there xpath :)
.//div[contains(#class, 'author-info')]//time

Search results don't change URL - Web Scraping with Python and Selenium

I am trying to create a python script to scrape the public county records website. I ultimately want to be able to have a list of owner names and the script run through all the names and pull the most recent deed of trust information (lender name and date filed). For the code below, I just wrote the owner name as a string 'ANCHOR EQUITIES LTD'.
I have used Selenium to automate the entering of owner name into form boxes but when the 'return' button is pressed and my results are shown, the website url does not change. I try to locate the specific text in the table using xpath but the path does not exist when I look for it. I have concluded the path does not exist because it is searching for the xpath on the first page with no results shown. BeautifulSoup4 wouldn't work in this situation because parsing the url would only return a blank search form html
See my code below:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome()
browser.get('http://deed.co.travis.tx.us/ords/f?p=105:5:0::NO:::#results')
ownerName = browser.find_element_by_id("P5_GRANTOR_FULLNAME")
ownerName.send_keys('ANCHOR EQUITIES LTD')
docType = browser.find_element_by_id("P5_DOCUMENT_TYPE")
docType.send_keys("deed of trust")
ownerName.send_keys(Keys.RETURN)
print(browser.page_source)
#lenderName = browser.find_element_by_xpath("//*[#id=\"report_results\"]/tbody[2]/tr/td/table/tbody/tr[25]/td[9]/text()")
enter code here
I have commented out the variable that is giving me trouble.. Please help!!!!
If I am not explaining my problem correctly, please feel free to ask and I will clear up any questions.

I think you almost have it.
You match the element you are interested in using:
lenderNameElement = browser.find_element_by_xpath("//*[#id=\"report_results\"]/tbody[2]/tr/td/table/tbody/tr[25]/td[9]")
Next you access the text of that element:
lenderName = lenderNameElement.text
Or in a single step:
lenderName = browser.find_element_by_xpath("//*[#id=\"report_results\"]/tbody[2]/tr/td/table/tbody/tr[25]/td[9]").text

have you used following xpath?
//table[contains(#summary,"Search Results")]/tbody/tr
I have checked it's work perfect.In that, you have to iterate each tr

Extracting hidden element in Selenium

I have an element of type hidden in an iframe. I am wondering if there would be any way to get this value as I am using selenium. More specifically it is a captcha field. I've tried pulling it with something along the lines of
#!/usr/bin/env python
from selenium import webdriver
driver=webdriver.Chrome(chrome_bin_path)
driver.get('http://websitehere.com')
print driver.find_element_by_xpath('//*[#id="recaptcha-token"]').text
but because of it's hidden nature it returns nothing.
Below is a snippet of the source.
Highlighted is the string of interest. (value)

driver.switch_to_frame('undefined')
token_value = driver.find_element_by_id('recaptcha-token').get_attribute('value')
driver.switch_to_default_content()
Moving between windows and frames.

Use this method
hidden_text = element.get_attribute("textContent")