Develop Reference

Can you help me to get Selenium to click image in Chromedriver?

I am trying to do a tutorial and learn Selenium in python however i cant seem to get Selenium to click the "Checkout" button using "element_to_be_clickable((By.XPATH".
I am using:
Python v3.9
Chrome v87
This is the URL i am practicing on:
https://www.aria.co.uk/myAria/ShoppingBasket
And this is my current code for the clicking:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.action_chains import ActionChains
import time
# Open Chromedriver
driver = webdriver.Chrome(r"C:\Users\Ste1337\Desktop\chromedriver\chromedriver.exe")
# Open webpage
driver.get("https://www.aria.co.uk/SuperSpecials/Other+products/ASUS+ROG+Pugio+2+Wireless+Optical+RGB+Gaming+Mouse?productId=72427")
#https://www.aria.co.uk/Products/Components/Graphics+Cards/NVIDIA+GeForce/GeForce+RTX+3060+Ti/Palit+GeForce+RTX+3060+Ti+Dual+8GB+GPU?productId=73054
# Click "Add to Basket" or refresh page if out of stock
try:
element = WebDriverWait(driver, 1).until(EC.presence_of_element_located((By.XPATH, "Out of Stock!")))
time.sleep(5)
browser.refresh()
except:
button = driver.find_element_by_id("addQuantityButton")
button.click()
basket = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.ID, "basketContent")))
basket.click()
checkout = WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH("//img[contains(#src,'/static/images/checkoutv2.png.png')]"))).click()

I can see your xpath is not correct.
Your Xpath should be.
//img[contains(#src,'/static/images/checkoutv2.png')]
Your code should be.
WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, "//img[contains(#src,'/static/images/checkoutv2.png')]"))).click()

The link you provided contains hCaptcha, which is actually responsible to check whether you are a human being or a bot. I guess that it's also the reason, why you can't click any of the items on the page, because Selenium actually is nothing less than a bot.
You first have to pass the test by clicking on the images, which are asked for.

Webdriver selenium - can't find any element from TradingView

I'm new to using Selenium but I watched enough videos and followed enough articles to know something is missing. I'm trying to get values from TradingView but the problem I'm running into is that I simply can't find any of the elements, not by Xpath or Css. I went ahead and tried to do a simple visibility element test as shown in the code below and to my surprise it times out.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
# Stops the UI interface (chrome browser) from popping up
# chrome_options.add_argument("--headless")
driver = webdriver.Chrome(executable_path='c:\se\chromedriver.exe', options=chrome_options)
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException
import time
driver.get("https://www.tradingview.com/chart/")
timeout = 20
try:
WebDriverWait(driver, timeout).until(EC.visibility_of_element_located((By.XPATH, "/html/body/div[1]")))
print("Page loaded")
except TimeoutException:
print("Timed out waiting for page to load")
driver.quit()
I tried to click on one of the chart buttons too using the following and that doesn't work either. I noticed that unlike many other websites for Tradingview the elements don't have names and don't generate a relative path (only full) using Xpath.
driver.find.element_by_xpath('/html/body/div[2]/div[5]/div/div[2]/div/div/div/div/div[4]').click()
Any help is greatly appreciated!

I think there must be an issue with xpath.
When I try to click the AAPL button it is working for me.
The xpath I used is:
(//div[contains(text(),'AAPL')])[1]
If you specify exactly which element to be clicked I will try.
And also be familiar with the concept of frames because these type of websites has lot of frames in it.

Can't login to Target.com with Selenium

I'm working on a online purchase bot for target.com and I've run into a blocker. Upon providing correct username and password combination on the login page and clicking "login", I get the following error message on the target login page: "Sorry, something went wrong. Please try again.". This only occurs when running through browser automation. Just wondering if there's a workaround for this issue. Here is my code thus far:
from selenium import webdriver
from selenium.common.exceptions import NoSuchElementException
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
import time
PATH="C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
btn_sign_in_nav = "//span[text()='Sign in']"
btn_sign_in_drop_down = "//div[#id='accountMenu']//div[text()='Sign in']"
input_username_login = "//input[#id='username']"
input_password_login = "//input[#id='password']"
btn_submit_login = "//button[#id='login']"
def xpath_explicit_wait(xpath_val, time_sec):
if type(time_sec) != int or type(xpath_val) != str:
print("NUMERICAL VALUES ONLY!")
driver.quit()
try:
element = WebDriverWait(driver, time_sec).until(
EC.presence_of_element_located((By.XPATH, xpath_val))
)
except NoSuchElementException:
driver.quit()
def login_user(url, username, password):
driver.get(url)
driver.implicitly_wait(3)
driver.find_element_by_xpath(btn_sign_in_nav).click()
xpath_explicit_wait(btn_sign_in_drop_down, 5)
driver.find_element_by_xpath(btn_sign_in_drop_down).click()
driver.implicitly_wait(3)
driver.find_element_by_xpath(input_username_login).send_keys(username)
driver.find_element_by_xpath(input_password_login).send_keys(password)
xpath_explicit_wait(btn_submit_login, 5)
driver.find_element_by_xpath(btn_submit_login).click()

There are detection mechanisms on sites like target that detect when you're using selenium and prevent the site from working.
More details can be found in the answer here: Can a website detect when you are using Selenium with chromedriver?
PhoenixBot implements a mechanism that changes the contents of the driver so that it's undetectable. Use this same mechanism and your problems will vanish, as mine did! :-D
https://github.com/Strip3s/PhoenixBot/blob/554441b3b6888a9be46b8aed6d364dc33da92e87/utils/selenium_utils.py#L150-L173

There is a bot Phoenixbot, I can't speak for the functionality 100% but the auto login portion definitely works and is python. https://github.com/Strip3s/PhoenixBot/blob/master/sites/target.py
I've been attempting to figure out how exactly it's accomplishing that but no success replicating successfully in my own code. Maybe take a look.... If you figure it out I'd love to know.

IF you have the ability to use Safari webdriver, then see the below code.
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as ec
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.common.action_chains import ActionChains
from selenium.webdriver.common.keys import Keys
driver = webdriver.Safari(executable_path='/usr/bin/safaridriver')
driver.get('https://www.target.com/')
action = ActionChains(driver)
driver.find_element(By.XPATH, '//*[#id="account"]').click()
WebDriverWait(driver, 30).until(ec.presence_of_element_located((By.XPATH, '//*[#id="accountNav-signIn"]')))
action.send_keys(Keys.ENTER)
action.perform()
WebDriverWait(driver, 10).until(ec.presence_of_element_located((By.XPATH, '//h2[#class="sc-hMqMXs sc-esjQYD eXTUDl"]')))
driver.find_element(By.ID, 'username').click()
driver.find_element(By.ID, 'username').send_keys('foo')
time.sleep(5)
driver.find_element(By.ID, 'password').click()
driver.find_element(By.ID, 'password').send_keys('bar')
time.sleep(5)
driver.find_element(By.XPATH, "//button[#id=\'login\']").send_keys(Keys.ENTER)
time.sleep(10)
driver.quit()
As #Decian Shanaghy mentioned, Target seems to be bot protected, but Safari webdriver still works.
The time.waits are not needed, you can remove them if you would like.

For anyone still seeking a solution to this, see this answer at Can a website detect when you are using Selenium with chromedriver?. I did a search through the chromedriver binary executable (MacOS in my case) and changed instances of $cdc_ to $abc_.
I originally suspected Target employs a JavaScript solution to hunting selenium users. I confirmed this suspicion by using selenium to open a browser at https://www.target.com without any automation. I attempted to manually login and obtained a 401 response with the well known "Sorry, something went wrong. Please try again." That didn't rule out a UI-based bot hunting solution but it pointed to a JavaScript hunter.

Python Selenium find element XPath doesn't work

I try to find refresing elements (time minute) on the webpage. My code worked only for simple text earlier. Now I use Ctrl+Shift+I and point out my element and "Copy Xpath".
Also, I have Chrome extension "XPath helper" and tried to do that with it one. There is more longer XPath, than in my code below. And it doesn't work too.
Error: NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id....
And, I also tried to use find by class, by tag, by CSS selector.. It only worked by tag and no perfect, on different page.
And I don't even say about print it, sometimes find_element(By.XPATH,'//*[...).text work, sometimes not.
I don't understand, why it work on one page and not on second.. I want to work with find elements by XPath in flash later.
UPDATE Now I retrying code and it work! But still doesn't work on the next webpage.. why it is so changeable? XPath change, when page reload or what? What is the simplest way to get text(refresing) info from flash, opened in chrome browser?
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(r"C:\Users\vishniakov\Desktop\python bj\driver\chromedriver.exe",chrome_options=options)
driver.get("https://www.betfair.com/sport/football/event?eventId=28935432")
print(driver.title)
elem =driver.find_element(By.XPATH,'//*[#id="yui_3_5_0_1_1538670363144_2571"]').text
print(elem)

This will work with assumption you want data of that page not of any specific element:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(chrome_options=options)
driver.get("https://www.betfair.com/sport/football/event?eventId=28935730")
print(driver.title)
elem =driver.find_element(By.CSS_SELECTOR,'.scroller.context-event').text
print(elem)

Assuming you do want spcipic data, you can use the contains() Xpath method... You can read about this here
For your case:
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
options = webdriver.ChromeOptions()
options.add_argument('headless')
driver = webdriver.Chrome(r"C:\Users\vishniakov\Desktop\python bj\driver\chromedriver.exe",chrome_options=options)
driver.get("https://www.betfair.com/sport/football/event?eventId=28935432")
print(driver.title)
elements =driver.find_elements(By.XPATH,'//*[contains(#id, "yui_3_5_0_1_")]')
print([i.text for i in elements])
You can play around with the contains() if my example didn't work... You must find what part of the id changes and exclude that part of the ID.
Hope this helps you!

Unable to use "explicit wait" in the right way

I've written a script using python with selenium to click on some links listed in the sidebar of google maps. When any of the items get clicked, the related information attached to each lead shows up in the right sided area. The script is doing fine. However, I've used hardcoded delay to do the job. How can I get rid of hardcoded delay by achieving the same with explicit wait. Thanks in advance.
Link to the site: website
The script I'm trying with:
import time
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
link = "replace_with_above_link"
driver = webdriver.Chrome()
driver.get(link)
wait = WebDriverWait(driver, 10)
for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "[id^='rlimg0_']"))):
item.location
time.sleep(3) #wish to try with explicit wait but can't find any idea
item.click()
driver.quit()
I tried with wait.until(EC.staleness_of(item)) instead of hardcoded delay but no luck.

If you want to wait until new data displayed after each clcik you may try below:
for item in wait.until(EC.visibility_of_all_elements_located((By.CSS_SELECTOR, "[id^='rlimg0_']"))):
div = driver.find_element_by_xpath("//div[#class='xpdopen']")
item.location
item.click()
wait.until(EC.staleness_of(div))