Develop Reference

Is it possible to use selenium to click this button?

The xpath is correct and I've tried both variations but it returns NoneType
from selenium import webdriver
from bs4 import BeautifulSoup
import time
import pyautogui
# Set up a webdriver instance using the Chrome browser
driver = webdriver.Chrome()
# Navigate to the staking page
driver.get('https://staking.chain.link/')
# Wait for the page to load
time.sleep(15)
# Find the "Next" button
next_button = driver.find_element_by_xpath('/html/body/div[1]/main/div/div[3]/div[3]/button[2]')
# Click the "Next" button to retrieve the next 10 results
next_button.click()
I want to be able to click through the table displaying the oracles answers. Is it possible to use Selenium to click this button?
I also tried:
next_button = class_element= driver.find_element_by_class('button_secondary__WYcyn paragraph-100')

The find_element_by_xpath has been deprecated in newer versions of Selenium
add this import
from selenium.webdriver.common.by import By
change this line
next_button = driver.find_element(By.XPATH, '/html/body/div[1]/main/div/div[3]/div[3]/button[2]')
https://selenium-python.readthedocs.io/

Unable to locate "Accept" Button - Selenium - Beginner Web Scraping

I am trying to use Selenium in order to learn different ways of web scraping.
When the code is executed Firefox starts and the "accept cookies" or what ever pops up. I am unable to locate the "accept" button when inspecting the page.
my code so far:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
import pandas as pd
import time
PATH = "C:/Users/myuser/Desktop/Driver/geckodriver.exe"
driver = webdriver.Firefox(executable_path=PATH)
driver.maximize_window() # For maximizing window
driver.get("https://www.immonet.de/")
button_pos = driver.find_element(by=By.CLASS_NAME, value="sc-gsDKAQ fILFKg")
button_pos.click()
print(driver.title)
input = input()
I get the following error: Unable to locate element: .sc-gsDKAQ fILFKg
My thought was locating the button via the inspect tool as follows:
What am I missing or doing wrong? How would i find the right element?
Thanks!
Pat

First of all, to display this url,accepting the cookies is a must but to accept and click on the cookie button isn't a easy task because cookies button is under shadow root (open) selenium and webdriverWait can do nothing on shadow root,so to execute shadow root you need to apply JavaScript querySelector.
#To execute shadow root and accept cookies
driver.execute_script('''return document.querySelector('div#usercentrics-root').shadowRoot.querySelector('button[data-testid="uc-accept-all-button"]')''').click()

Class attribute in the html element can contain multiple classes separated by space. i.e. "sc-gsDKAQ fILFKg", contains two classes, sc-gsDKAQ and fILFKg.
You can user either but both are random and can be changed next time css is recompiled. I recommend to think of xpath using data-testid attribute

xpath for click to a button - python/selenium

I'm trying to access a homepage, make login, click the login button and click a button (in the second page) using python/selenium.
The login button I wrote using Xpath and it's working well. The browser opens, writes user and login and click the login button.
Unfortunately, in the next page I'm not able to click on the button that I need to. It didn't work using Xpath and I can't understand why. The html is completely different from the first button, the button name is 'Reservar' and it is inside a class named <app-menu__menu>, written as:
<a href="/Services" id="advanced" class="element ">
<span>Reservar</span>
</a>
The xpath I got and tried:
xpath = "//*[#id="advanced"]"
Then I tried a second verion (it was gotten as the second line code xpath):
xpath = "//*[#id="advanced"]/span"
When I first tried to used them, I got an error. Then I change the "" to ' ' and the error was gone. But the program can't locate the button.
I'm using google-chrome, ubuntu, python3 and selenium package:
driver.find_element_by_xpath(xpath).click()
Thanks for any help.

You probably missing a delay / wait.
After clicking the submit / login button on the login page it takes some time to make the internal page loaded.
The recommended way to do that is to use Expected Conditions explicit waits, something like this:
wait.until(EC.visibility_of_element_located((By.XPATH, "//*[#id='advanced']"))).click()
To use the wait object here you will need to import the following inports:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
And to initialize the wait object with
wait = WebDriverWait(driver, 20)
You will have to validate your locator is unique. As well as to validate the element you are trying to access is not inside iframe and not in a new window / tab etc.

Using Selenium in iframe: NoSuchElementException

I am running a streamlit app, in which I try to run selenium (click a search box and) inside iframe component, but I keep getting an error "NoSuchElementException".
It successfully opens the iframe on streamlit app, but it does not execute the selenium code inside the iframe. The selenium code itself runs fine when I test it without the iframe, but I can't get it to work inside the iframe.
Here is my code:
import streamlit as st
from dateutil.parser import parse
import streamlit.components.v1 as components
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
st.title("Auto Search App")
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 20)
# url = 'https://wego.here.com/'
# driver.get(url)
components.iframe("https://wego.here.com/", height = 500)
## Give time for iframe to load ##
time.sleep(3)
## You have to switch to the iframe like so: ##
driver.switch_to.frame(driver.find_element_by_tag_name("iframe"))
search_input = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input.input_search")))
search_input.click()
search_input.send_keys('Seattle')
search_input.send_keys(Keys.ENTER)
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "button.btn"))).send_keys(Keys.ENTER)
internal_search_input = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, "input#itinerary_item_input_0")))
internal_search_input.click()
internal_search_input.send_keys('Chicago')
internal_search_input.send_keys(Keys.ENTER)
This is the streamlit page with the error message:
EDIT: Based on the answer, I figured that I don't have to switch to iframe in order to access the search input element. I guess the problem is due to the fact that when I run "streamlit run app5.py" on cmd, it generates two separate browsers with an empty browser opening most recently, so that the selenium codes is executing on an empty page not on the streamlit localhost:8501 page. How can I not generate the empty browser and only open the localhost:8501 page, so that the selenium code runs on the streamlit page?
EDIT2: I added more selenium execution code for your reference. The reason that I am using selenium here is to do some automation done on the iframe so that it populates the navigation page at the end. I was able to get it done on a separate chrome by using driver.get(url) but not on streamlit with iframe. This is the final result I want to see in the iframe:

From your code i understand that you are trying to run the selenium code to set the value of the input in the iframe inside the Streamlit page that opens.
This is not possibble. I will try to explain using the order of events when you run this code.
With your current code this is what happens:
you run the streamlit run app.py command.
Streamlit opens this code in a chrome browser.
The selenium code starts running and because you don't use driver.get(url) it runs on an empty page causing the NoSuchElementException.
If you use the driver.get(url) and set the url to the Streamlit app url='http://localhost:8501/' then it also wont work because this is what happens:
you run the streamlit run app.py command.
Streamlit opens this code in a chrome browser.
The selenium code starts running and opens a new chrome tab of the selenium code and you cant make selenium get an already opened tab (session).
I would suggest not using selenium for this and to create a text_input using Streamlit and generate the iframe using the geolocation of the input.
Code:
import streamlit as st
import streamlit.components.v1 as components
from geopy.geocoders import Nominatim
bt = st.text_input("Enter Location")
if bt:
geolocator = Nominatim(user_agent="my_app")
location = geolocator.geocode("Seattle")
components.iframe(f"https://wego.here.com/?map={location.latitude},{location.longitude},18,normal&x=ep", height = 500)
Output:
Edit 1 - if you just want to view direction from Chicago to Seattle this can be done by changing only the url.
code:
import streamlit as st
import streamlit.components.v1 as components
from geopy.geocoders import Nominatim
components.iframe("https://wego.here.com/directions/mix/Chicago/Seattlel", height = 500)
Output:

The input.input_search element is NOT inside any iframe on that page.
There are 4 iframes there, however the search input is not inside any of them.
So you should not switch to iframe in order to access the search input element.

You need to consider a couple of things here.
The desired <input> isn't within any <iframe> as such, so you don't need to switch to any frame.
To click() on any clickable element instead of presence_of_element_located() you need to induce WebDriverWait for the element_to_be_clickable() and you can use the following Locator Strategy:
Code Block:
driver.get("https://wego.here.com/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.input_search"))).send_keys("Seattle" + Keys.RETURN)
Browser Snapshot:

Selenium not finding an element

I am trying to retrieve an element that I would like to click on. Here's the opening of the website with Selenium in Python:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--dns-prefetch-disable')
driver = webdriver.Chrome("./chromedriver", options=chrome_options)
website = "https://www.agronet.gov.co/estadistica/Paginas/home.aspx?cod=4"
driver.get(website) # loads the page
Then, I look for the element I'm interested in:
driver.find_element_by_xpath('//*[#id="cmbDepartamentos"]')
which raises a NoSuchElementException error. When looking at the html source (driver.page_source), indeed "cmbDepartamentos" does not exist! and the text of the dropdown menu I am trying to locate which is "Departamentos:" does not exist either. How can I deal with this?

This should work:
iframe=driver.find_element_by_xpath('//div[#class="iframe"]//iframe')
driver.switch_to.frame(iframe)
driver.find_element_by_xpath('//*[#id="cmbDepartamentos"]').click()
Notes:
The reason for NoSuchElementException error is that the element is
inside an iframe. Unless you switch your driver to that iframe,
the identification will not work.
CTRL + F in the Dev Tools panel, then search for the xpath you
defined in your script is always a good way to rule out issues with
your xpath definition, as cause for NoSuchElementException error (and in your case, the xpath is correct)
You might want to consider adding a WebdriverWait for a complete load of the search area/iframe before attempting to find the "Departamentos" field