How to get the updated url from firefox browser using selenium and Python? The below code is a very good working example of what I am trying to do. The script opens up a url, looks for the search bar in the webpage, pastes a particular product and then executes the search.
I am trying to extract the updated url after the search is completed which should be https://www.myntra.com/avene-unisex-thermal-spring-water-50-ml but I am getting https://www.myntra.com/. How can I get the required url?
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
firefoxOptions = webdriver.FirefoxOptions()
firefoxOptions.set_preference("dom.webnotifications.enabled", False)
driver = webdriver.Firefox(firefox_options = firefoxOptions)
driver.implicitly_wait(5)
# Maximize the browser window
driver.maximize_window()
# navigate to the home page
driver.get("https://www.myntra.com/")
# Locate the text field to update values
text_field = driver.find_element_by_class_name("desktop-searchBar")
# Clears any value already present in text field
text_field.clear()
# Updates the string in search bar
text_field.send_keys("Avene Unisex Thermal Spring Water 50 ml")
text_field.send_keys(Keys.ENTER)
new_page = driver.current_url
print(new_page)
driver.close()
Seems you were pretty close. You need to induce WebDriverWait for the url to get changed and you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
driver=webdriver.Firefox(executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("https://www.myntra.com/")
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input.desktop-searchBar"))).send_keys("Avene Unisex Thermal Spring Water 50 ml")
driver.find_element_by_css_selector("a.desktop-submit").click()
WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "h1.title-title")))
print(driver.current_url)
driver.quit()
Console Output:
https://www.myntra.com/avene-unisex-thermal-spring-water-50-ml
You need to wait after text_field.send_keys(Keys.ENTER) to get information updated,
if you don't want waits then try to use click() method instead of send_keys
Related
I am trying to get the webdriver to click a button on the site random.org The button is a generator that generates a random integer between 1 and 100. It looks like this:
After inspecting the webpage, I found the corresponding element on the webpage looks something like this:
It is inside an iframe and someone suggested that I should first switch over to that iframe to locate the element, so I incorporated that in my code but I am constantly getting NoSuchElementException error. I have attached my code and the error measage below for your reference. I can't understand why it cannot locate the button element despite referencing the ID, which is supposed to unique in the entire document.
The code:
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Edge()
driver.get("https://www.random.org/")
driver.implicitly_wait(15)
driver.switch_to.frame(driver.find_element(By.TAG_NAME, "iframe"))
button = driver.find_element(By.CSS_SELECTOR, "input[id='hnbzsqjufzxezy-button']")
button.click()
The error message:
Make sure that there are no more Iframes on the page. If there are a few an not only one do this:
iframes = driver.find_elements(By.CSS, 'iframe')
// try switching to each iframe:
driver.switch_to.frame(iframes[0])
driver.switch_to.frame(iframes[1])
You can't find the button because its name contain random letters. Every time you will refresh the page you can see that the name value will change. So, do this:
button = driver.findElement(By.CSS, 'input[type="button"][value="Generate"]')
button.click()
There are several issues with your code:
First you need to close the cookies banner
The locator of the button is wrong. It's id is dynamic.
You need to use WebDriverWait to wait for elements clickability.
The following code works:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = Options()
options.add_argument("start-maximized")
webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(service=webdriver_service, options=options)
url = 'https://www.random.org/'
driver.get(url)
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[onclick*='all']"))).click()
wait.until(EC.frame_to_be_available_and_switch_to_it((By.TAG_NAME, "iframe")))
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[id*='button']"))).click()
I am trying to do a tutorial and learn Selenium in python however i cant seem to get Selenium to click the "Checkout" button using "element_to_be_clickable((By.XPATH".
I am using:
Python v3.9
Chrome v87
This is the URL i am practicing on:
https://www.aria.co.uk/myAria/ShoppingBasket
And this is my current code for the clicking:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
import time
# Open Chromedriver
driver = webdriver.Chrome(r"C:\Users\Ste1337\Desktop\chromedriver\chromedriver.exe")
# Open webpage
driver.get("https://www.aria.co.uk/SuperSpecials/Other+products/ASUS+ROG+Pugio+2+Wireless+Optical+RGB+Gaming+Mouse?productId=72427")
#https://www.aria.co.uk/Products/Components/Graphics+Cards/NVIDIA+GeForce/GeForce+RTX+3060+Ti/Palit+GeForce+RTX+3060+Ti+Dual+8GB+GPU?productId=73054
# Click "Add to Basket" or refresh page if out of stock
try:
element = WebDriverWait(driver, 1).until(EC.presence_of_element_located((By.XPATH, "Out of Stock!")))
time.sleep(5)
browser.refresh()
except:
button = driver.find_element_by_id("addQuantityButton")
button.click()
basket = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID, "basketContent")))
basket.click()
checkout = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH("//img[contains(#src,'/static/images/checkoutv2.png.png')]"))).click()
I can see your xpath is not correct.
Your Xpath should be.
//img[contains(#src,'/static/images/checkoutv2.png')]
Your code should be.
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//img[contains(#src,'/static/images/checkoutv2.png')]"))).click()
The link you provided contains hCaptcha, which is actually responsible to check whether you are a human being or a bot. I guess that it's also the reason, why you can't click any of the items on the page, because Selenium actually is nothing less than a bot.
You first have to pass the test by clicking on the images, which are asked for.
Hoping you can help. I'm relatively new to Python and Selenium. I'm trying to pull together a simple script that will automate news searching on various websites. The primary focus was football and to go and get me the latest Manchester United news from a couple of places and save the list of link titles and URLs for me. I could then look through the links myself and choose anything I wanted to review.
In trying the the independent newspaper (https://www.independent.co.uk/) I seem to have come up against a problem with element not interactable when using the following approaches:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome('chromedriver')
driver.get('https://www.independent.co.uk')
time.sleep(3)
#accept the cookies/privacy bit
OK = driver.find_element_by_id('qcCmpButtons')
OK.click()
#wait a few seconds, just in case
time.sleep(5)
search_toggle = driver.find_element_by_class_name('icon-search.dropdown-toggle')
search_toggle.click()
This throws the selenium.common.exceptions.ElementNotInteractableException: Message: element not interactable error
I've also tried with XPATH
search_toggle = driver.find_element_by_xpath('//*[#id="quick-search-toggle"]')
and I also tried ID.
I did a lot of reading on here and then also tried using WebDriverWait and execute_script methods:
element = WebDriverWait(driver, 20).until(EC.presence_of_element_located((By.XPATH, '//*[#id="quick-search-toggle"]')))
driver.execute_script("arguments[0].click();", element)
This didn't seem to error but the search box never appeared, i.e. the appropriate click didn't happen.
Any help you could give would be fantastic.
Thanks,
Pete
Your locator is //*[#id="quick-search-toggle"], there are 2 on the page. The first is invisible and the second is visible. By default selenium refers to the first element, sadly the element you mean is the second one, so you need another unique locator. Try this:
search_toggle = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//div[#class="row secondary"]//a[#id="quick-search-toggle"]')))
search_toggle.click()
First you need to open search box, then send search keys:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.chrome.options import Options
import os
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome(executable_path=os.path.abspath(os.getcwd()) + "/chromedriver", options=chrome_options)
link = 'https://www.independent.co.uk'
browser.get(link)
# accept privacy
button = browser.find_element_by_xpath('//*[#id="qcCmpButtons"]/button').click()
# open search box
li = browser.find_element_by_xpath('//*[#id="masthead"]/div[3]/nav[2]/ul/li[1]')
search_tab = li.find_element_by_tag_name('a').click()
# send keys to search box
search = browser.find_element_by_xpath('//*[#id="gsc-i-id1"]')
search.send_keys("python")
search.send_keys(Keys.RETURN)
Can you try with below steps
search_toggle = driver.find_element_by_xpath('//*[#class="row secondary"]/nav[2]/ul/li[1]/a')
search_toggle.click()
I'm trying to scrape an AJAX webpage using Python and Selenium. The problem is, when I change the dropdown value, the page content changes according to my selection, but the selenium returns the same old html code from the page. I'd appreciate if anyone can help. Here is my code:
from selenium import webdriver
from selenium.webdriver.support.ui import Select
import time
url = "https://myurl.com/PATH"
driver = webdriver.Chrome()
driver.get(url)
time.sleep(5)
# change the dropdown value
sprintSelect = Select(driver.find_element_by_id("dropdown-select"))
sprintSelect.select_by_visible_text("DropDown_Value2")
html = driver.execute_script("return document.documentElement.outerHTML")
print(html)
You need to wait for the ajax to load the website after your selection.
Try to put implicit or explicit wait after selection.
driver.implicitly_wait(10) # 10 seconds
or if you know the tag/id etc. of the web element you want, try the explicit
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
element = WebDriverWait(driver, 10).until(
EC.presence_of_element_located((By.ID, "some_ID"))
I am using python selenium
I am running this simple code
driver = webdriver.PhantomJS()
#Also use driver = webdriver.Chrome()
driver.get("my_url")
driver.find_element_by_xpath("//*[#id='lb_LoadMore_button_text_2']").click()
print [x.text for x in driver.find_elements_by_xpath("//font[#class='ProductTitle']")]
The button I am clicking is a 'load more' button. On the Chrome webdriver I see that the items are being loaded, but I don't know how to access them after the click was committed.
I have also tried driver.refresh() before trying to print the elements
I am admittedly fresh to selenium and have not been able to find a solution.
You might just need a delay after a click, but let's not just add a time.sleep() call, but explicitly wait for the product titles to be present via presence_of_all_elements_located():
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver.find_element_by_xpath("//*[#id='lb_LoadMore_button_text_2']").click()
# waiting
wait = WebDriverWait(driver, 10)
product_titles = wait.until(EC.presence_of_all_elements_located((By.XPATH, "//font[#class='ProductTitle']")))
print [x.text for x in product_titles]