Our developer that does python left and I've been assigned the task of learning how to use the language. On top of that we can't find his source code so I'm having to rebuild the code from scratch. I have no experience in python and have spent all morning trying to find the answer to this. I realize there's 100 other questions similar to this and I promise I've looked through most of them.
I'm scraping a site and after logging in a modal is popping up forcing me to click a button to exit it. The button doesn't have an ID so I'm trying to find it by text. I've tried using XPATH, LINK_TEXT, and a couple of other things. Below is my code and some screenshots that I hope help identify what I'm doing wrong.
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I think this is a temporary modal so I'm trying to set up the code to look for my "real" button first. If it can't find it then throw the exception and look for the modal button.
try:
srch = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.ID, "1744")))
srch.send_keys(Keys.ENTER)
#NoSuchElementException thrown if not present
except:
btn_text = "Don't ask again"
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[text()=\"" + btn_text + "\"]"))).click()
This is giving me the following error while debugging:
I'm trying to select the Don't ask again button to close the modal. Here's what I see when I inspect it:
Curiously though, if I just start by inspecting the modal "from the beginning", when I drill down to where the button is it's properties don't come up, this is as far down as it goes:
And this is where I'm stuck. Does anybody have any suggestions?
EDIT: I'm adding the entire error that I'm getting below:
Without seeing the real page entire HTML we can only guess, so as a first guess I would suggest instead of this
btn_text = "Don't ask again"
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//*[text()=\"" + btn_text + "\"]"))).click()
I'd try this:
WebDriverWait(browser, 10).until(EC.element_to_be_clickable((By.XPATH, "//a[contains(.,'ask again')]"))).click()
Related
I just started trying to learn to write python code today. I am trying to make a web scraper to do some stuff, but i am having trouble getting selenium to click on the specific boxes/places to get to the correct page. Because it uses Ember.js, it is a Single Page Application. This throws things off a little. I found some stuff on stackoverflow about it, but i wasnt able to get it to work. I keep getting the "NoSuchElementException" error. Here is my code
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
PATH = r'C:\Users\dme03\Desktop\desktop\webdrivers\chromedriver_win32\chromedriver.exe'
driver = webdriver.Chrome(PATH)
driver.get('https://thedailyrecord.com/reader-rankings-interface/')
time.sleep(15)
mim = driver.find_element(By.XPATH, '//img[#src="//media.secondstreetapp.com/1973100?width=370&height=180&cropmode=Fill&anchor=Center"]/parent::div')
mim.click
For the XPATH, I had to select a child element and specify the parent to select the correct element. The key identifiers in the parent element kept changing, so i was unable to select just that. I believe the XPATH is correct because when I put that into the website, it showed up as the correct line.
I have also tried using the code below, as i have seen some people suggest. When using that, I get a timed out error.
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//img[#src="//media.secondstreetapp.com/1973100?width=370&height=180&cropmode=Fill&anchor=Center"]/parent::div')))
Any help would be appreciated
NEW CODE for iFrames
WebDriverWait(driver, 20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH, '//iframe[#src="https://embed-882706.secondstreetapp.com/embed/ae709978-d6d1-499e-9f5e-136cc36eab1b/"]')))
mim = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, '//img[#src="//media.secondstreetapp.com/1973100?width=370&height=180&cropmode=Fill&anchor=Center"]/parent::div')))
mim.click
I am having issues clicking the item. To be fair, i am not sure that you are supposed to click that element to change the webpage. That is probably an issue.
Edit: nvm im just dumb that totally worked lol :))
You are getting "NoSuchElementException" error because the desired element is inside the iframe .So you have to switch to iframe at first.See the below image screenshot
So I am trying to automate my daily thing to login to my ADP website but it seems whatever function I pick from selenium is does not find the ID that I am looking for which is "id="login-form_username".
Below is the part of the Selenim Python code.
user_textbox=driver.find_element_by_id('login-form_username')
user_textbox.send_keys(user)
user_textbox.click()
user_confirm=driver.find_element_by_id("verifUseridBtn")
user_pass_textbox=driver.find_element_by_id("login-form_password")
user_signin=driver.find_element_by_id("signBtn")
user_signin.send_keys(Keys.RETURN)
Also Attached screenshot of the site and error from my IDE. Im not sure if its because for security reasons that the Website I am trying to access wont allow this kind of thing.
Thanks
Joel
Since you didn't share the page link and not the error you seeing we can only guess what is wrong with your code.
So it can be:
You are missing a wait / delay before accessing that element.
In this case something like this should resolve your issue:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Chrome(executable_path='chromedriver.exe')
wait = WebDriverWait(driver, 20)
driver.get("https://your_site_url/")
user_textbox = wait.until(EC.visibility_of_element_located((By.ID, "login-form_username")))
user_textbox.send_keys(user)
user_textbox.click()
Alternatively you can simply add a delay before locating the elements, something like this:
time.sleep(5)
user_textbox=driver.find_element_by_id('login-form_username')
user_textbox.send_keys(user)
user_textbox.click()
time.sleep(5)
user_confirm=driver.find_element_by_id("verifUseridBtn")
user_pass_textbox=driver.find_element_by_id("login-form_password")
time.sleep(5)
user_signin=driver.find_element_by_id("signBtn")
user_signin.send_keys(Keys.RETURN)
but this is not the best approach.
2) Maybe there is an iframe there.
In this case you have to switch to that iframe.
3) Maybe you didn't set web driver to large enough size - if so please do that.
4) Possibly you have to scroll that input element into the view?
When trying to make a Selenium bot that comments on Instagram pictures, the bot will click on the comment box, but the keys will not send. Interestingly though, when I manually click the comment box myself, the keys will send. I'm unsure what's going on here as the box appears to be clicked just fine, but keys will not send without me manually clicking on it. I'm new to both Python and Selenium... And programming in general.
I've tried adding a delay between the click() and the send keys but to no avail. I still have the same issue. I've also tried sending the keys more than once, with again, a delay between them. But again, this does not work. I'm certain that I have the class name correct, as the bot appears to find the box, the problem is with the send key. I've searched and searched and I know others have had this same issue but no solution has worked for me, that's where I got the idea of sending the keys twice, as well as adding a time delay. This is the code in question below -
comment = driver.find_element_by_class_name("Ypffh").click()
time.sleep(5)
comment.send_keys("test")
time.sleep(5)
comment.send_keys("test")
And here is what I'm seeing when inspecting the Instagram comment box -
<textarea aria-label="Add a comment…" placeholder="Add a comment…" class="Ypffh" autocomplete="off" autocorrect="off" style="height: 18px;"></textarea>
EDIT -----
I have updated the code to this but still not having any luck.
comment = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "Ypffh")))
comment.click()
comment.send_keys("test")
You can use WebDriverWait to try and make sure the browser can click on the textarea before sending the keys to it. Also, its not completely necessary but I like to save the elements that I know I will use a lot at the top using By.
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium import webdriver
TextArea = (By.CLASS_NAME, "Ypffh")
#Initialize browser and go to proper page same as before...
WebDriverWait(driver, 10).until(EC.element_to_be_clickable(TextArea)).send_keys("Whatever you want to Type")
Another possible problem could be the time.sleep() calls. It doesn't take into account your operating systems process scheduler which can cause the sleep to be more or less than what you actually pass to it. A better option is selenium's implicit_wait().
You are trying to send_keys to a click that returns a NONE.
Just change it to:
comment = driver.find_element_by_class_name("Ypffh")
comment.click()
time.sleep(5)
comment.send_keys("test")
time.sleep(5)
comment.send_keys("test")
It is best practice to use WebDriverWait not time.sleep
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
comment = WebDriverWait(driver, 10).until(EC.element_to_be_clickable((By.CLASS_NAME, "Ypffh")))
comment.click()
comment.send_keys("test")
comment.send_keys("test")
I faced the same problem. Then i tried first click on the comment box element and then send keys .
It just worked out for me
driver.find_element_by_xpath('/html/body/div[5]/div[2]/div/article/div[3]/section[3]/div/form/textarea').click()
time.sleep(1.5)
driver.find_element_by_xpath("/html/body/div[5]/div[2]/div/article/div[3]/section[3]/div/form/textarea").send_keys("nice post,Keys.ENTER)#comment
I am trying to use Selenium to log into a printer and conduct some tests. I really do not have much experience with this process and have found it somewhat confusing.
First, to get values that I may have wanted I opened the printers web page in Chrome and did a right click "view page source" - This turned out to be not helpful. From here I can only see a bunch of <script> tags that call some .js scripts. I am assuming this is a big part of my problem.
Next I selected the "inspect" option after right clicking. From here I can see the actual HTML that is loaded. I logged into the site and recorded the process in Chrome. With this I was able to identify the variables which contain the Username and Password. I went to this part of the HTML did a right click and copied the Xpath. I then tried to use the Selenium find_element_by_xpath but still no luck. I have tried all the other methods to (find by ID, and name) however it returns an error that the element is not found.
I feel like there is something fundamental here that I am not understanding. Does anyone have any experience with this???
Note: I am using Python 3.7 and Selenium, however I am not opposed to trying something other than Selenium if there is a more graceful way to accomplish this.
My code looks something like this:
EDIT
Here is my updated code - I can confirm this is not just a time/wait issue. I have managed to successfully grab the first two outer elements but the second I go deeper it errors out.
def sel_test():
chromeOptions = Options()
chromeOptions.add_experimental_option("useAutomationExtension", False)
browser = webdriver.Chrome(chrome_options=chromeOptions)
url = 'http://<ip address>/'
browser.get(url)
try:
element = WebDriverWait(browser, 10).until(EC.presence_of_element_located((By.XPATH, '//*[#id="ccrx-root"]')))
finally: browser.quit()
The element that I want is buried in this tag - Maybe this has something to do with it? Maybe related to this post
<frame name="wlmframe" src="../startwlm/Start_Wlm.htm?arg11=">
As mentioned in this post you can only work with the current frame which is seen. You need to tell selenium to switch frames in order to access child frames.
For example:
browser.switch_to.frame('wlmframe')
This will then load the nested content so you can access the children
Your issue is most likely do to either the element not loading on the page until after your bot searches for it, or a pop-up changing the xpath of the element.
Try this:
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException
delay = 3 # seconds
try:
elementUsername = WebDriverWait(browser, delay).until(EC.presence_of_element_located((By.xpath, 'element-xpath')))
element.send_keys('your username')
except TimeoutException:
print("Loading took too much time!")
you can find out more about this here
I am trying to create a Python program that periodically checks a website for a specific update. The site is secured and multiple clicks are required to get to the page that I want to monitor. Unfortunately, I am stuck trying to figure out how to click a specific button. Here is the button code:
<input type="button" class="bluebutton" name="manageAptm" value="Manage Interview Appointment" onclick="javascript:programAction('existingApplication', '0');">
I have tried numerous ways to access the button and always get "selenium.common.exceptions.NoSuchElementException:" error. The obvious approach to access the button is XPath and using the Chrome X-Path Helper tool, I get the following:
/html/body/form/table[#class='appgridbg mainContent']/tbody/tr/td[2]/div[#class='maincontainer']/div[#class='appcontent'][1]/table[#class='colorgrid']/tbody/tr[#class='gridItem']/td[6]/input[#class='bluebutton']
If I include the above as follows:
browser.find_element_by_xpath("/html/body/form/table[#class='appgridbg mainContent']/tbody/tr/td[2]/div[#class='maincontainer']/div[#class='appcontent'][1]/table[#class='colorgrid']/tbody/tr[#class='gridItem']/td[6]/input[#class='bluebutton']").submit()
I still get the NoSuchElementException error.
I am new to selenium and so there could be something obvious that I am missing; however, after much Googling, I have not found an obvious solution.
On another note, I have also tried find_element_by_name('manageAptm') and find_element_by_class_name('bluebutton') and both give the same error.
Can someone advise on how I can effectively click this button with Selenium?
Thank you!
To follow your attempts and the #har07's comment, the find_element_by_name('manageAptm') should work, but the element might not be immediately available and you may need to wait for it:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
manageAppointment = wait.until(EC.presence_of_element_located((By.NAME, "manageAptm")))
manageAppointment.click()
Also, check if the element is inside an iframe or not. If yes, you would need to switch into the context of it and only then issue the "find" command:
driver.switch_to.frame("frame_name_or_id")
driver.find_element_by_name('manageAptm').click()