Develop Reference

How do I click on an item on my google search page with Selenium Python?

Good time of the day!
Faced with a seemingly simple problem,
But it’s been a while, and I’m asking for your help.
I work with Selenium on Python and I need to curse about
20 items on google search page by random request.
And I’ll give you an example of the elements below, and the bottom line is, once the elements are revealed,
Google generates new such elements
Problem:
Cannot click on the element. I will need to click on existing and then on new, generated elements in this block: click for open see blocks with element
Tried to click on xpath, having collected all the elements:
xpath = '//*[#id="qmCCY_adG4Sj3QP025p4__16"]/div/div/div[1]/div[4]'
all_elements = driver.find_element(By.XPATH, value=xpath)
for element in all_elements:
element.click()
sleep(2)
Important note!
id xpath has constantly changing and is generated by another on the google side
Tried to click on the class
class="r21Kzd"
Tried to click on the selector:
#qmCCY_adG4Sj3QP025p4__16 > div > div > div > div.wWOJcd > div.r21Kzd
Errors
This is when I try to click using xpath:
Message: no such element: Unable to locate element: {"method":"xpath","selector"://*[#id="vU-CY7u3C8PIrgTuuJH4CQ_9"]/div/div[1]/div[4]}
In other cases, the story is almost the same, the driver does not find the element and cannot click on it. Below I apply a scratch tag on which I need to click
screenshot tags on google search
Thanks for the help!

In case iDjcJe IX9Lgd wwB5gf are a fixed class name values of that element all you need is to use CSS_SELECTOR instead of CLASS_NAME with a correct syntax of CSS Selectors.
So, instead of driver.find_element(By.CLASS_NAME, "iDjcJe IX9Lgd wwB5gf") try using this:
driver.find_element(By.CSS_SELECTOR, ".iDjcJe.IX9Lgd.wwB5gf")
(dots before each class name, no spaces between them)

scraping SVG data by CSS Selector and Id (Selenium)

I'm looking to scrape a label from an SVG that only arrives with a mouse hover.
I'm working with this link for the data contained with the [+] expand button to the right in each of the table rows. When you press [+] expand, an SVG table pops up that shows elements that contain elements. When you hover on each of the elements, a element appears called "Capacity Impact" with a value for each of the bars. These values are the values I want to scrape.
See a screenshot below.
So far, my code is successful in opening each of the [+] expand buttons, and identifying the polygons but I can't get to the labels using either XPATH or CSS Selectors. See code below.
driver.get(url)
table_button_xpath = "//table[#class='data-view-table redispatching dataTable']//tr//td[#class = 'button-column']//a[#class='openIcon pre-table-button operation-detail-expand small-button ui-button-light ui-button ui-widget ui-corner-all ui-button-text-only']"
driver.find_element(By.ID, "close-button").click()
driver.find_element(By.ID, "cookieconsent-button").click()
# open up all the "+" buttons
table_buttons = driver.find_element(By.XPATH, table_button_xpath)
for i in list(range(1, 10)):
driver.find_element(By.XPATH, table_button_xpath).click()
# find all the polygons
polygons = driver.find_elements(By.TAG_NAME, 'path')
label_xpath = "//*[name()='svg']//*[name()='g' and #id = 'ballons')]//*[name()='g']//*[name()='tspan']"
for polygon in polygons :
action.move_to_element(polygon)
labels_by_xpath = driver.find_elements(By.XPATH, label_xpath)
labels_by_css_selector = driver.find_elements(By.CSS_SELECTOR, "svg>#ballons>g>text>tspan")
Both labels_by_xpath and labels_by_css_selector return a list of 0 elements. I've tried many versions of both the xpath and css selector approach, along with using WebDriverWait, but I can't get it to return the capacity impact values.
HTML screenshot is also copied below (to be clear, the number I need to scrape is the "50" text in the tag.
Any help is appreciated! Thank you,
Sophie

The solution to your problem is with the locator.
Here is the updated locator to select the desired element.
CSS Selector :
svg>[id^='balloons']>g:nth-child(2)>text:nth-child(2)>tspan
try this to get the element Capacity 50

x = driver.find_elements(By.CSS_SELECTOR, "svg>[id^='balloons']>g>text>tspan")

How do I find a more descriptive XML Path for my Selenium webscrape?

I'm building a website scraper using Selenium and I want to "click" the highlighted div in the image below.
My current code (which works, but isn't very descriptive) is:
button = driver.find_element_by_xpath("//div/div/div/div/div/div/div/div[5]/div[8]")
button.click()
I'm glad it works, but it feels fragile, since I'm accessing the divs purely by index, without any other identifying features. Is there a way, at least for the last div, that I can specify my choice by the text within span? What would the syntax be for choosing the div that contains a span with the text "Grandmaster"?
It's worth noting that this is the only div in any of the "filter-group"s that contains the text "Grandmaster". Is there a way to select this div specifically, without listing all the nested divs (as I've done in my code above)?
Any other ideas on how to make the XML path's code a bit more robust would be appreciated.

What would the syntax be for choosing the div that contains a span with the text "Grandmaster"?
The syntax would be:
driver.find_element_by_xpath("//*[contains(text(), 'Grandmaster')]")

What would the syntax be for choosing the div that contains a span
with the text "Grandmaster"?
You can use this xPath:
//span[contains(., 'Grandmaster')]/parent::div
more information you can get here.

Selenium by data-react-id attribute on web scrape

I'm trying to scrape some data from Yahoo! Finance, but I've noticed that most of the elements have something called data-reactid. So when using selenium locate the element by, when I try to name or id I get an error each time. I've never used the XPath method, but could someone take a look at https://finance.yahoo.com/quote/IBM.
I want to save data-reactid='35' which are the $165 close price to a variable name data for example and then print the variable.

Use the following CSS selector (here I used a nested element structure):
price_per_share = driver.find_element_by_css_selector("#quote-header-info > div > div > div > span[data-reactid='35']")
print(price_per_share.text)
It's more accurate. Hope it helps you!
PS: data-reactid is custom attribute of the span element.

css_locator = 'div.quote-header-section span[data-reactid="35"]'
price = driver.find_element_by_css_selector(css_locator).text
print price

How to search two driver.find_element_by_partial_link_text

I am trying to create an automated bot to purchase items from supreme python/selenium.
When I am on the products page and I use a driver.find_element_by_partial_link_text('Flight Pant') to find the product I want to buy, however I also want to select the colour of the product so I use a driver.find_element_by_partial_link_text('Black') but by doing this I am returned with the first Black product on the page instead of Flight pants that are Black. Any idea how I would achieve this goal?
here is the site link where I am try to achieve this,
http://www.supremenewyork.com/shop/all/pants
Note - I am unable to use xpaths for this, as the products change on a weekly bases so I would be unable to get the xpath for the product before it goes live on the site.
Any advice or guidance would be greatly appreciated.

You can use XPath, but the maneuver is slightly trickier. The XPath would be:
driver.find_element_by_xpath('//*[contains(text(), "Flight Pant")]/../following-sibling::p/a[contains(text(), "Black")]')
Assuming the structure of the page doesn't change on a weekly basis... To explain my XPath:
//*[contains(text(), "Flight Pant")]
Select any node that contains the text "Flight Pant". These are all <a> tags.
/../following-sibling::p
Notice how the DOM looks:
<h1>
<a class="name-link" href="/shop/pants/dfkjdafkj">Flight Pant</a>
</h1>
<p>
<a class="name-link" href="/shop/pants/pvfcp0txzy">Black</a>
</p>
So we need to go to the parent and find its sibling that is a <p> element.
/a[contains(text(), "Black")]
Now go to the <a> tag that has the text Black.
The reason there's not really any other alternative to XPath is because there's no unique way to identify the desired element by any other means (tag name, class, link text, etc.)

After finding elements by the link text "Flight pants" , iterate over each found result and extract its css color attributes. Its a psuedo-code. You have to fine tune the specific color extraction web elements.
elements = driver.find_elements_by_partial_link_text("Flight Pants")
for element in elements :
if(element.get_css_value('color').lower() == "black")
element.click()
break