Develop Reference

How do I click on an item on my google search page with Selenium Python?

Good time of the day!
Faced with a seemingly simple problem,
But it’s been a while, and I’m asking for your help.
I work with Selenium on Python and I need to curse about
20 items on google search page by random request.
And I’ll give you an example of the elements below, and the bottom line is, once the elements are revealed,
Google generates new such elements
Problem:
Cannot click on the element. I will need to click on existing and then on new, generated elements in this block: click for open see blocks with element
Tried to click on xpath, having collected all the elements:
xpath = '//*[#id="qmCCY_adG4Sj3QP025p4__16"]/div/div/div[1]/div[4]'
all_elements = driver.find_element(By.XPATH, value=xpath)
for element in all_elements:
element.click()
sleep(2)
Important note!
id xpath has constantly changing and is generated by another on the google side
Tried to click on the class
class="r21Kzd"
Tried to click on the selector:
#qmCCY_adG4Sj3QP025p4__16 > div > div > div > div.wWOJcd > div.r21Kzd
Errors
This is when I try to click using xpath:
Message: no such element: Unable to locate element: {"method":"xpath","selector"://*[#id="vU-CY7u3C8PIrgTuuJH4CQ_9"]/div/div[1]/div[4]}
In other cases, the story is almost the same, the driver does not find the element and cannot click on it. Below I apply a scratch tag on which I need to click
screenshot tags on google search
Thanks for the help!

In case iDjcJe IX9Lgd wwB5gf are a fixed class name values of that element all you need is to use CSS_SELECTOR instead of CLASS_NAME with a correct syntax of CSS Selectors.
So, instead of driver.find_element(By.CLASS_NAME, "iDjcJe IX9Lgd wwB5gf") try using this:
driver.find_element(By.CSS_SELECTOR, ".iDjcJe.IX9Lgd.wwB5gf")
(dots before each class name, no spaces between them)

How to switch to the child frame within #shadow-root (open) using Selenium

I'm trying to make a web scraper that downloads an image that's inside of an iframe with a child.
I can't get Selenium for Chrome to find the correct iframe to switch into. The main issue is the iframe in question doesn't have a name or id so I searched by index. I managed to get inside of the parent, but I can't get inside of the sub-child. If I set the index to 1 I get the next iframe in the outermost scope.
From looking into my webdriver object I think the search is limited to Red Rectangle, as thats what's inside the page source attribute of my var "driver".
The Object I want to reach is the img with the id pbk-page in the Green Rectangle
My code so far just gets the url then waits for the page to load using sleep (once I can navigate to the correct element I'll implement WebDriverWait). This is the navigation bit of code:
driver.switch_to.frame(0)
Image_link = driver.find_element(By.ID,'pbk-page')
Oh! I'm using python

Like any other element iframe can be located by XPath or CSS Selector. They can use any attribute value making that locator unique. I believe here you could uniquely locate both the iframes by their src value, but since you marked them out I can't see their values.

As per the given HTML:
The desired element:
<img id="pbk-page"....>
is within the child <iframe> which is within a #shadow-root (open) marked with a blue rectangle.
Solution
To access the desired element you need to:
Switch within the parent iframe
Switch within the shadow-root
Switch within the child iframe
Then locate the element
Effectively, your code block will be:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"parent_iframe_css_selector")))
shadow_host = driver.find_element(By.CSS_SELECTOR, 'mosaic-book')
shadow_root = shadow_host.shadow_root
shadow_content = shadow_root.find_element(By.CSS_SELECTOR, 'child_iframe_css_selector')
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR,"child_iframe_css_selector")))
element = WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//img[#id='pbk-page']")))

I was stuck doing the exact same thing you were (maybe even scraping the same website?), and this is what worked for me:
My solution:
iframe1 = driver.find_elements(By.XPATH, value="//iframe")[0]
driver.switch_to.frame(iframe1)
iframe2 = driver.execute_script("return document.querySelector(\"body > mosaic-book\").shadowRoot.querySelector(\"iframe\")")
driver.switch_to.frame(iframe2)
img = driver.find_elements(By.ID, value="pbk-page")
I am very much an amateur at using Selenium, but this is my best understanding of how this works: First, we're able to find the parent iframe iframe1, but our driver can't see anything inside of the shadow DOM. However, we can access inside of the shadow DOM using javascript, so starting from the iframe, we can find the shadow host element mosaic-book, enter the shadow DOM, and return/pass out the child iframe iframe2. Then we can switch our driver into this iframe2 and access the image.
There very well might be a more elegant way to do this, but this is what worked for me.
Reading I did to come up with this solution:
What even is a shadow root/shadow DOM or whatever??? https://www.geeksforgeeks.org/what-is-shadow-root-and-how-to-use-it/
How to access elements inside of a shadow DOM using javascript: https://www.youtube.com/watch?v=PQcRaIoc2AM

Selenium Python Error: "stale element reference: element is not attached to the page document"

HTML Image
I'm using selenium to get some information from a site, but the links that I need to click on require double clicking. Here's the code (I've included an attachment of the image of the HTML)
PATH = "C:\Program Files (x86)\msedgedriver.exe"
driver = webdriver.Edge(PATH)
action = ActionChains(driver)
from selenium import webdriver
from selenium.webdriver import ActionChains
frame = search_by_XPATH('//[#id="frmMain"]/table/tbody/tr/td[2]/div[2]').get_attribute('id').replace("_Div","_Frame")
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.ID, frame)))
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"fraTree_Frame")))
rows = driver.find_elements(By.XPATH,'//*[#id="ctl00_treChart_RadTreeView"]/ul/li/div/span[2]')
for row in rows:
# row.click()
action.double_click(row).perform()
sleep(1)
I'm accessing different pages to get the links, and although each page has the same content the "id" of the frame changes with each new page I access. So that's why I use the 'get_attribute' function to get the id of the frame of the page.
I'm using action chains right now to perform the double click, but every time I do I get the stale element reference: element is not attached to the page document error. I know it's not an issue with the XPATH I'm using to access the element because I can perform a element.click() function on it just fine and the element receives the click. Nothing changes after the first click except the element gets highlighted. The element's xpath doesn't change and its still on the page. I also tried doing the action.click().perform() on the element but that produces the same error.
I've tried clicking on other parts of the element that could receive the click (the div, the span2,etc) but I always get the error. The 'span2' is the part that holds the text of the link (although its not an 'a' tag so its not really a link in that sense.)
I don't understand why the element would be able to receive the click for the .click() function but then be unable to receive the double click. I'm using the double_click() function on another element previously in my code and that works as expected.

Entering values into iframe python selenium shopify payments

I am trying to use selenium + python to enter credit card values into a Shopify site. The boxes to enter the card values are in an iframe and I am unsure how to switch to this iframe.
I currently have this code:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,
'//*[#id="card-fields-number-950kvfi9pbn00000"]'))
).send_keys(card_number, Keys.TAB, name_on_card, Keys.TAB,expiry_date, cvv)
driver.switch_to.default_content()
But this returns the error:
WebDriverWait(driver, 10).until(EC.frame_to_be_available_and_switch_to_it(...
selenium.common.exceptions.TimeoutException: Message:
So effectively, the element could not be found...
This is the HTML of the page:
(https://gyazo.com/80d9d3c941c62ededc81d5fbc327a71f)
I would like some help on how to access this element, I have also tried accessing it by changing the id to a parent of this tag. I have also added a time.sleep(20) so I can be sure the page has fully loaded and I still got the same error.

You can switch to an iframe using the switch_to_frame method.
iframe = driver.find_element(By.XPATH, '//*[#id="card-fields-number-950kvfi9pbn00000"]')
driver.switch_to_frame(iframe)
#You're now 'in' the iframe

The problem was that the name and id of the element are dynamic and change for each unique checkout window this is a working code:
iframe = driver.find_element_by_class_name('card-fields-iframe')
driver.switch_to.frame(iframe)
driver.find_element_by_name('number').send_keys(card_number, Keys.TAB,name_on_card,Keys.TAB,expiry_date,Keys.TAB, cvv)
driver.switch_to.default_content()

You should try
driver.switch_to_frame("Insert 'Name'of Iframe here"), then find your element. After finding the element. You can try:
ELEMENT.send_keys(card_number, Keys.TAB, name_on_card, Keys.TAB,expiry_date, cvv)
driver.switch_to.default_content()
But I'd advise search for each element instead of using the tab key. It'll make it easier to fix or adjust in the future as websites can change at any time. Essentially, You can use if statements to detect the fields,and if their present, fill them in.

How do I click a button using Selenium in Python?

I'm trying to crawl review data on an app from a Google Play website page using Python and Selenium. What I'm trying to do here is to click the "Full Review" button to see the entire text of each review.
But I keep running into these errors each time:
ElementNotInteractableException: Message: element not interactable
or
AttributeError: 'list' object has no attribute 'click'
or
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":".LkLjZd ScJHi OzU4dc "}
The element for the button is this:
<button class="LkLjZd ScJHi OzU4dc " jsaction="click:TiglPc" jsname="gxjVle">Full Review</button>
The xpath for the button is this:
//*[#id="fcxH9b"]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div/div[26]/div/div[2]/div[2]/span[1]/div/button
And this is the code I'm using for xpath:
full_review = driver.find_elements_by_xpath('//*[#id="fcxH9b"]/div[4]/c-wiz/div/div[2]/div/div[1]/div/div/div[1]/div[2]/div/div[26]/div/div[2]/div[2]/span[1]/div/button')
full_review.click()
I can't find the button by class name, xpath or anything.
Could anyone help me figure out this problem? In addition, is it possible to find the element by jsname?
I would really appreciate if you could help.

Avoid using xpath whenever possible, it's the most brittle selector.
id > CSS > Xpath
For your button, this css selector should work:
button[jsname='gxjVle']
You'll probably need to specify the child as that probably won't be unique

Your XPath selector is a little bit flaky, you can simplify it to be something like:
//button[text()='Full Review']
You're using the wrong function, if you're looking for a single unique element you should be using WebDriver.find_element_by_xpath one, mind that element, not elements
It's better to use Explicit Wait just in case element is not immediately available in DOM (for example if you're testing application built using AJAX technology)
Assuming all above:
full_review = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Full Review']")))