Develop Reference

Python Selenium accept cookies

I need to accept cookies on a specific website but I keep getting the NoSuchElementException. This is the code for entering the website:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import time
chrome_options = Options()
driver = webdriver.Chrome(executable_path='./chromedriver', options=chrome_options)
page_url = 'https://www.boerse.de/historische-kurse/Erdgaspreis/XD0002745517'
driver.get(page_url)
time.sleep(10)
I tried accepting the cookie button using the following:
driver.find_element_by_class_name('message-component message-button no-children focusable button global-font sp_choice_type_11 last-focusable-el').click()
driver.find_element_by_xpath('//*[#id="notice"]').click()
driver.find_element_by_xpath('/html/body/div/div[2]/div[4]/div/button').click()
I got the xpaths from copying the xpath and the full xpath from the element while using google chrome.
I am a beginner when it comes to selenium, just wanted to use it for a short workaround. Would appreciate some help.

The button Zustimmen is in iframe so first you'd have to switch to the respective iframe and then you can interact with that button.
Code:
driver.maximize_window()
page_url = 'https://www.boerse.de/historische-kurse/Erdgaspreis/XD0002745517'
driver.get(page_url)
wait = WebDriverWait(driver, 30)
try:
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, "//iframe[starts-with(#id,'sp_message_iframe')]")))
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Zustimmen']"))).click()
print('Clicked successfully')
except:
print('Could not click')
pass
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

I've got a similar problem with a page. I've tried to address the iframe and the "accept all" button with selenium (as suggested by #cruisebandey above).
However, the pop-up on this page seems to work differently:
https://www.kreiszeitung-wochenblatt.de/hanstedt/c-panorama/mega-faslams-umzug-in-hanstedt_a270327
This is what I've tried:
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(executable_path="C:\\Users\\***\\chromedriver.exe")
driver.maximize_window()
try:
driver.get("https://www.kreiszeitung-wochenblatt.de/hanstedt/c-panorama/mega-faslams-umzug-in-hanstedt_a270327")
except:
print('Site not found')
wait = WebDriverWait(driver,10)
try:
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,'/html/body/iframe')))
except:
print('paywall-layover not found')
try:
cookie = wait.until(EC.element_to_be_clickable((By.XPATH,'//*[#id="consentDialog"]/div[3]/div/div[2]/div/div[2]/div/div[1]/div[2]/div')))
cookie.click()
except:
print('Button to accept all not found')

Why DOM-element is not displayed to selenium webdriver?

I need to press green button in to accept cookies on https://garantex.io/
My code:
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.implicitly_wait(10)
driver.get("https://garantex.io/")
try:
cookie_access = driver.find_element_by_class_name('btn.btn-success')
print("Is displayed = " + str(cookie_access.is_displayed()))
except Exception as e:
driver.close()
print(e)
I have Is displayed = FALSE. How can I interact with this button?

You were almost correct. but that is CSS selector not class name.
driver = webdriver.Chrome(ChromeDriverManager().install())
# driver.implicitly_wait(10)
driver.maximize_window()
wait = WebDriverWait(driver, 30)
driver.get("https://garantex.io/")
try:
wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".btn.btn-success"))).click()
print('Clicked on cookies button successfully.')
#print("Is displayed = " + str(cookie_access.is_displayed()))
except Exception as e:
driver.close()
print(e)
You will need these imports as well
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

The button is clicked with me normally, maybe you need to add some wait like this:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
...
driver.get("https://garantex.io/")
cookie_access = WebDriverWait(driver, 30).until(EC.presence_of_element_located((By.CLASS_NAME, "btn.btn-success")))
cookie_access.click()
You also don't need driver.implicitly_wait(10) at all before loading the page, maybe after it.

How do i make explicit wait check for page source

This is just an example, but I want if Hello in html source on explicit wait, how do I make it work?
if "Hello" in html_source:
WebDriverWait wait = new WebDriverWait(driver,10)
wait.until(ExpectedConditions.visibilityOfElementLocated(By.linkText("foobar")))

From selenium documentation
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//*[text()='Hello']")))
print(driver.page_source)
except:
print("element could not be found within the 10 second timeout period")
If you want a better answer, I suggest you ask a better question and use examples.

Script fails to keep clicking on load more button

I've written a script in Python in association with selenium to keep clicking on MORE button to load more items until there are no new items left to load from a webpage. However, my below script can click once on that MORE button available on the bottom of that page.
Link to that site
This is my try so far:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
print(elems.get_attribute("href"))
try:
loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
driver.execute_script("arguments[0].scrollIntoView();", loadmore)
loadmore.click()
except Exception:break
driver.quit()
How can I keep clicking on that MORE button until there are no such button left to click and parse the links as I've already tried using for loop.

I've managed to solve the problem pursuing sir Andersson's logic within my exising script. This is what the modified script look like.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "https://angel.co/companies?company_types[]=Startup&company_types[]=Private+Company&company_types[]=Mobile+App&locations[]=1688-United+States"
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get(link)
while True:
try:
loadmore = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR,"[class='more']")))
driver.execute_script("arguments[0].click();", loadmore)
wait.until(EC.staleness_of(loadmore))
except Exception:break
for elems in wait.until(EC.presence_of_all_elements_located((By.CSS_SELECTOR,".results .name a.startup-link"))):
print(elems.get_attribute("href"))
driver.quit()

why not just?
while (driver.FindElements(By.ClassName("more")).Count > 0)
{
driver.FindElement(By.ClassName("more")).Click();
//Some delay to wait lazyload to complete
}
c# example. pretty sure that it can be done with python as well

Python Selenium Xpath from firebug not found

I am trying to login to the ESPN website using selenium. Here is my code thus far
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.maximize_window()
url = "http://www.espn.com/fantasy/"
driver.get(url)
login_button = driver.find_element_by_xpath("/html/body/div[6]/section/section/div/section[1]/div/div[1]/div[2]/a[2]")
login_button.click()
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "/html/body/div[2]/div/div/section/section/form/section/div[1]/div/label/span[2]/input")))
except:
driver.quit()
Basically, there are 2 steps, first I have to click the login button and then I have to fill in the form. Currently, I am clicking the login button and the form is popping up but then I can't find the form. I have been using firebug to get the xpath as suggested in other SO questions. I don't really know much about selenium so I am not sure where to look

Try to use
driver.switch_to_frame('disneyid-iframe')
# handle authorization pop-up
driver.switch_to_default_content() # if required

This works for me, switching to the iframe first. Note that you will need to switch back out of the iframe after entering the credentials.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.maximize_window()
url = "http://www.espn.com/fantasy/"
driver.get(url)
login_button = driver.find_element_by_xpath("/html/body/div[6]/section/section/div/section[1]/div/div[1]/div[2]/a[2]")
login_button.click()
iframe = driver.find_element_by_id("disneyid-iframe")
driver.switch_to.frame(iframe)
try:
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.XPATH, "/html/body/div[2]/div/div/section/section/form/section/div[1]/div/label/span[2]/input")))
element.send_keys("my username")
import time
time.sleep(100)
finally:
driver.quit()