I am attempting to choose date on a calendar on this website. On the first calendar (date from) I can choose the desired date using Selenium, however, I get the following error while clicking on the desired month even though the exact element is found.
ElementNotInteractableException:element not interactable
To me, it seems weird because I can click on the month manually.
Here is what I have tried so far
from selenium import webdriver
import time
year = 2019
month = 'JAN'
driver_path = 'pathtochromedriver\chromedriver.exe'
url = 'https://app.cpcbccr.com/ccr/#/caaqm-dashboard-all/caaqm-landing/data'
driver = webdriver.Chrome(driver_path)
driver.get(url)
time.sleep(8)
# find desired calendar
to_date = driver.find_element_by_xpath('//*[#id="date2"]/angular2-date-picker/div/div[1]/i')
to_date.click()
# Click on year dropdown
to_year = driver.find_element_by_xpath('//*[#id="date2"]/angular2-date-picker/div/div[2]/div[3]/div')
to_year.click()
driver.find_element_by_xpath('//*[#id="{}"]'.format(year)).click()
# Click on month dropdown
to_month = driver.find_element_by_xpath('//*[#id="date2"]/angular2-date-picker/div/div[2]/div[2]/div')
to_month.click()
mm = driver.find_element_by_xpath('//*[#id="{}"]'.format(month))
mm.click()
Mistake in code mm = driver.find_element_by_xpath('//*[#id="{}"]'.format(month)). You find first element in DOM(which combined with element data instead data2) and it is not visible yet.
There is workig code
mm = driver.find_element_by_id('date2').find_element_by_class_name('months-view').find_element_by_id(month)
mm.click()
Also good deal, to use WebDriverWait, because the site is very slow.
for example
to_date = WebDriverWait(driver, 10).until(
expected_conditions.presence_of_element_located(
(By.XPATH, '//*[#id="date2"]/angular2-date-picker/div/div[1]/i')))
Related
I am trying to pull data from following page:
https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs
I have to select the name of the company and select a date from the calendar and click get data button.
I am trying to achieve this using Selenium Web Driver using Chrome in Python, I am stuck how do i pass the date parameter to the page.
it seems the page is postback after selection of date from the calendar.
Date needs to be selected from the calendar else the data is not returned by the webpage.
I have tried using requests Post method as well but am not able to get the NAV data.
I need to iterate this for a period of 5 years on daily (Trading Days) basis.
PS: I am bad at understanding DOM elements and have basic knowledge of Python and coding. by profession I am a data analyst.
Thanks in Advance.
Kiran Jain
edit: adding current code below:
from selenium import webdriver
url='https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs'
opt = webdriver.ChromeOptions()
prefs = {"profile.managed_default_content_settings.images": 2}
opt.add_argument("--start-maximized")
# opt.add_argument("--headless")
opt.add_argument("--disable-notifications")
opt.add_experimental_option("prefs", prefs)
driver = webdriver.Chrome(options=opt)
driver.get('https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs');
insurer = driver.find_element_by_id("MainContent_drpselectinscompany")
nav_date=driver.find_element_by_id('MainContent_txtdateselect')
get_data_btn=driver.find_element_by_id('MainContent_btngetdetails')
options=insurer.find_elements_by_tag_name("option")
data=[]
row={'FirmName','SFIN','fundName','NAVDate','NAV'}
for option in options:
print('here')
print(option.get_attribute("value") + ' ' + option.text)
if(option.text!='--Select Insurer--'):
option.click()
driver.find_element_by_id("MainContent_imgbtncalender").click()#Calender Icon
driver.find_element_by_link_text("June").click()#Date
driver.find_element_by_link_text("25").click()#Date
get_data_btn=driver.find_element_by_id('MainContent_btngetdetails') #this is put here again because on clicking the date, the page is reloaded
get_data_btn.click()
print('clicked')
driver.quit()
The date is in "a" tag. You can try to do select the date using "link-text".
driver.find_element_by_id("MainContent_imgbtncalender").click()#Calender Icon
driver.find_element_by_link_text("27").click()#Date
As per your comment I tried to traverse through dates but it only worked for that particular month. I tried to use "send_keys()" to that text box and its not working. Below is the code to traverse it for a month.
driver.get("https://www.lifeinscouncil.org/industry%20information/ListOfFundNAVs")
driver.find_element_by_id("MainContent_drpselectinscompany").click()
driver.find_element_by_xpath("//option[starts-with(text(),'Aditya')]").click()
driver.find_element_by_id("MainContent_imgbtncalender").click()
driver.find_element_by_link_text("1").click()
driver.find_element_by_id("MainContent_btngetdetails").click()
dateval = 2
while True:
if dateval == 32:
break
try:
driver.find_element_by_id("MainContent_imgbtncalender").click()
driver.find_element_by_link_text(str(dateval)).click()
driver.find_element_by_id("MainContent_btngetdetails").click()
dateval+=1
time.sleep(2)
except:
driver.switch_to.default_content()
dateval+=1
time.sleep(2)
time.sleep(5)
driver.quit()
I am trying to select dropdown year of 2021 on (https://www.theknot.com/registry/couplesearch) and am unable to figure out how to use the dropdown.
#This code is working
typetextfirst = driver.find_element_by_id("couples-search-first-name")
typetextfirst.clear()
typetextfirst.send_keys(row["First"])
typetextlast = driver.find_element_by_id("couples-search-last-name")
typetextlast.clear()
typetextlast.send_keys(row["Last"])
typetextyear = driver.find_element_by_id("couples-search-year")
#None of these options work to populate the year
typetextyear.selectByIndex(1)
typetextyear.select_by_index(1)
typetextyear.selectByVisibleText("2021")
typetextyear.select_by_visible_text("2021")
#This code is working
typetextlast.send_keys(Keys.ENTER)
Page doesn't use standard dropdown widget but it uses button and ul to emulate dropdown.
This code works for me on Firefox and Chrome on Linux Mint.
First I click button to open dropdown created with ul and later I search li with expected text and click it.
Because it may have text 2021 with some spaces/tabs/enters (which browser doesn't show) so I prefer contains instead of =
from selenium import webdriver
url = 'https://www.theknot.com/registry/couplesearch'
driver = webdriver.Firefox()
#driver = webdriver.Chrome()
driver.get(url)
year_dropdown = driver.find_element_by_id("couples-search-year")
year_dropdown.click()
year = year_dropdown.find_element_by_xpath(".//li[contains(text(), '2021')]")
#year = year_dropdown.find_element_by_xpath(".//li[text()='2021']")
year.click()
I am using Python selenium chrome driver and i am stuck at filling out the csc and the year of the creditcard information field ( look at picture ). The credit card number and month works fine with this code:
iframe = driver.find_element_by_xpath("//iframe[#class='js-iframe']")
driver.switch_to.frame(iframe)
inputCC = WebDriverWait(driver, 30).until(
lambda driver: driver.find_element_by_id("encryptedCardNumber")
)
inputCC.send_keys("1111222233334444")
driver.switch_to.default_content()
time.sleep(1)
iframe = driver.find_element_by_xpath("//iframe[#class='js-iframe']")
driver.switch_to.frame(iframe)
inputCC = WebDriverWait(driver, 30).until(
lambda driver: driver.find_element_by_id("encryptedExpiryMonth")
)
inputCC.send_keys("08")
driver.switch_to.default_content()
I tried to use the same for the csc and year with changing the id but it didnt work.
How to do it?
I don't run your code, but I checked the HTML & your code. Here's what I think:
Because //iframe[#class='js-iframe'] is a very general XPATH, you need to be more specific. In your site, you have many iframes with the same XPATH.
You can fill the Month because after calling iframe = driver.find_element_by_xpath("//iframe[#class='js-iframe']"), it gives you the FIRST iframe, which contains the Month.
Your code fails for Year/CSC because it uses the FIRST iframe (which contains Month) to locate Year & CSC.
To fix, you have 2 methods.
Write the correct XPATH.
Month iframe: //span[#data-cse="encryptedExpiryMonth"]/iframe
Year iframe: //span[#data-cse="encryptedExpiryYear"]/iframe
CSC iframe: //span[#data-cse="encryptedSecurityCode"]/iframe
Find a list of iframes
iframe_list = driver.find_elements_by_xpath("//iframe[#class='js-iframe']")
month_iframe = iframe_list[0]
year_iframe = iframe_list[1]
csc_iframe = iframe_list[2]
I am trying to scrape the Google News page in the following way:
from selenium import webdriver
import time
from pprint import pprint
base_url = 'https://www.google.com/'
driver = webdriver.Chrome('/home/vincent/wintergreen/chromedriver') ## change here to your location of the chromedriver
driver.implicitly_wait(30)
driver.get(base_url)
input = driver.find_element_by_id('lst-ib')
input.send_keys("brexit key dates timetable schedule briefing")
click = driver.find_element_by_name('btnK')
click.click()
news = driver.find_element_by_link_text('News')
news.click()
tools = driver.find_element_by_link_text('Tools')
tools.click()
time.sleep(1)
recent = driver.find_element_by_css_selector('div.hdtb-mn-hd[aria-label=Recent]')
recent.click()
# custom = driver.find_element_by_link_text('Custom range...')
custom = driver.find_element_by_css_selector('li#cdr_opt span')
custom.click()
from_ = driver.find_element_by_css_selector('input#cdr_min')
from_.send_keys("9/1/2018")
to_ = driver.find_element_by_css_selector('input#cdr_max')
to_.send_keys("9/2/2018")
time.sleep(1)
go_ = driver.find_element_by_css_selector('form input[type="submit"]')
print(go_)
pprint(dir(go_))
pprint(go_.__dict__)
go_.click()
This script manage to enter search terms, switch to the news tab, open the custom time period tab, fill in start and end date, but fails to click on the 'Go' button after that point.
From the print and pprint statement at the end of the script, I can deduct that it does find the 'go' button succesfully, but is somehow unable to click on it. The error displays as selenium.common.exceptions.ElementNotVisibleException: Message: element not visible
Could anyone experienced with Selenium have a quick run at it and give me hints as why it returns such error?
Thx!
Evaluating the css using developer tools in chrome yields 4 elements.
Click here for the image
use the following css instead:
go_ = driver.find_element_by_css_selector('#cdr_frm > input.ksb.mini.cdr_go')
I'd like to use python selenium to search at https://www.homeaway.com/
The following works:
from selenium import webdriver
driver = webdriver.Firefox()
driver.get("https://www.homeaway.com/")
driver.find_element_by_xpath("//input[#id='searchKeywords']").send_keys("Philadelphia, PA, USA")
But I'm running into an issue using the the calendar dropdown date picker, as it's not taking any of the values.
I've tried the following
Attempt 1 to enter a date into the start and end date field:
driver.find_element_by_xpath("//input[#id='stab-searchbox-start-date']").send_keys("02/01/2017")
driver.find_element_by_xpath("//input[#id='stab-searchbox-end-date']").send_keys("03/01/2017")
Note: It looks like homeaway website is completely ignoring the above commands unless you mannually click on the website with your mouse and then use the above selenium commands. In other words, the above commands are not working without a manual mouse click on the website first.
Attempt 2 to enter a date into the start and end date field:
driver.find_element_by_xpath("//div[#id='2017-02-01_2017-02']").click()
Attempt 3 to enter a date into the start and end date field
driver.execute_script("document.querySelectorAll('#stab-searchbox-start-date')[0].value = '02/01/2017'")
driver.execute_script("document.querySelectorAll('#stab-searchbox-end-date')[0].value = '03/01/2017'")
driver.find_element_by_xpath("//button[#class='btn btn-primary btn-lg searchbox-submit js-searchSubmit']").click()
Note: this looks like it works, but the dates are actually not registered when you click on search despite the dates being entered into the star and end date text boxes. It seems that homeaway will only register the dates if you use the calendar dropdown.
Note that you can not send_keys in these types when input text is not supported.
try this code, it will work:
from selenium import webdriver
url='http://www.homeaway.com/'
driver = webdriver.Chrome()
driver.maximize_window()
driver.get(url)
driver.find_element_by_id("search-location").send_keys("Philadelphia, PA, USA")
driver.find_element_by_id('search-checkin').click()
driver.find_element_by_xpath('//*[#id="ui-datepicker-div"]//*[text()="30"]').click()
driver.find_element_by_id('search-checkout').click()
driver.find_element_by_xpath('//*[#id="ui-datepicker-div"]//*[text()="31"]').click()