I am learning python and selenium right now by making a small script that posts birthday comments to my facebook friends. I am able to successfully login and navigate to the "friends' birthday" modal, but then I am unable to comment in the textbox.
Here is the error I am getting:
selenium.common.exceptions.NoSuchElementException:
Message: no such element: Unable to locate element:
{"method":"xpath","selector":"//*[#id="u_i_6"]"}
I haven't had any issues finding other page elements via XPath so I am not sure what is causing the issue. I also tried finding the textbox to comment via classname, but had the same result. Can anyone offer some thoughts?
EDIT: forgot to paste code
from selenium import webdriver
from pynput.keyboard import Key, Controller
import time
import config
# fetch facebook password from separate file in the same directory
pw = config.secret['pw']
keyboard = Controller()
driver = webdriver.Chrome()
options = webdriver.ChromeOptions()
options.add_argument('--disable-extensions')
options.add_argument('--disable-notifications')
# navigate to facebook url
driver.get('https://facebook.com')
# input email address
email_input = driver.find_element_by_xpath('//*[#id="email"]')
email_input.send_keys(<omitted>)
# input pw
password_input = driver.find_element_by_xpath('//*[#id="pass"]')
password_input.send_keys(pw)
# click login button element
login_button = driver.find_element_by_xpath('//*[#id="u_0_b"]')
login_button.click()
home_button = driver.find_element_by_xpath('//*[#id="u_0_c"]/a')
# wait 8 seconds for browser popup before pressing esc to get out
time.sleep(8)
keyboard.press(Key.esc)
# check if there are any birthdays today
birthday_section = driver.find_element_by_xpath('//*[#id="home_birthdays"]/div/div/div/div/a/div/div/span/span[1]')
birthday_section.click()
first_comment = driver.find_element_by_xpath('//*[#id="u_i_6"]')
first_comment.send_keys('happy birthday!')
# click post button to post comment
# close browser window after actions are complete
time.sleep(5)
driver.close()
It is hard to answer without the code, but here are a couple of ideas:
Make sure you are using some sort of Selenium wait, implicit or explicit, so your code does not search for an element before this element appears on the page. You can add this code after driver.get() for implicit wait:
driver.implicitly_wait(5)
Also, it looks like IDs are dynamic on that page, I get a different one on my FB page. Try using this xpath to find the textarea:
"//form[contains(#action, 'birthday')]//textarea"
Hope this is helpful, good luck!
Related
First of all sorry for my junky coding, I am not a coder of any sort.
I created (copied xd) an Instagram bot like a year ago, and it was working absolutely fine until two months ago. My goal with my Instagram bot is to identify new followers and unfollowers and stuff for my main account. I don't do rapid followings and unfollowings. The problem is when I open the following popup on Instagram, selenium cannot find any element that I specify. I've tried using XPATH, CSS Selector, by class name and all those methods. But I cannot even click the close button after I open that popup page, the code cannot find the button. Do you guys have any ideas why or any suggestions?
last_ht, ht = 0, 1
WebDriverWait(self.browser, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#class=_aano]")))
scroll_box = self.browser.find_element_by_xpath("//div[#class=_aano]")
while last_ht != ht:
last_ht = ht
time.sleep(2.5)
ht = self.browser.execute_script(
"arguments[0].scrollTo(0, arguments[0].scrollHeight);"
"return arguments[0].scrollHeight;", scroll_box)
The code above should wait for the Following button on your profile page and then click it (there is nothing wrong until there), and then wait for the popup to load and then scroll, but the WebDriverWait function raises an error, meaning the code cannot find the element, but like I can literally see it. It waits for 10 seconds.
I've tried Chrome and then Firefox, but it was even worse in Mozilla, it couldn't locate the Followings button on the profile page.
Since you haven't shared the applicable part of the DOM, my best guess is that the element you need to interact with is located inside an iframe. To interact with components inside an iframe, you must switch to the iframe first (which means you need to write a locator for the iframe, and then execute the rest of your code.
last_ht, ht = 0, 1
# you must provide the xpath string for the iframe
driver.switch_to.frame(driver.find_element_by_xpath("..."))
WebDriverWait(self.browser, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#class=_aano]")))
scroll_box = self.browser.find_element_by_xpath("//div[#class=_aano]")
while last_ht != ht:
last_ht = ht
time.sleep(2.5)
ht = self.browser.execute_script(
"arguments[0].scrollTo(0, arguments[0].scrollHeight);"
"return arguments[0].scrollHeight;", scroll_box)
# the rest of your code
Once you finish with the components in the iframe, you must switch back to the parent frame.
driver.switch_to.parent_frame()
Or switch to the top window
driver.switch_to.default_content()
Try them in that order. I am sure the first one will work for you.
My goal with my Instagram bot is to identify new followers and unfollowers and stuff for my main account
So you should click "followers" button instead of "following" button, like you said here.
The code above should wait for the Following button on your profile page
So, assume that's what you need, this code will help you scroll down the "followers" popup. I put the whole code here because i don't know if your code clicked the "followers" button or not. Also, the:
last_ht, ht = 0, 1
while last_ht != ht:
last_ht = ht
part will make your scrolling down action run only once, quite weird that they use 3 lines of code for doing nothing. just remove that part and your code will do exact same thing it previously does. So I use a for loop here instead.
Things to change to use the code:
Put your username and password in these two lines:
INS_EMAIL = "youremail#gmail.com"
INS_PASSWORD = "yourpassword"
Fix the link in this line to your ins account link:
self.driver.get("https://www.instagram.com/geeks_for_geeks/")
Change "followers" to "following" depend on your use case:
followers = WebDriverWait(self.driver, 10).until(
EC.presence_of_element_located((
By.PARTIAL_LINK_TEXT,
'followers')))
I'm using a 100 times for loop but if you want to scroll until it reaches the end of the file, guess I can change that later.
Next time you should post longer code so we can understand if other parts are working or not.
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import os
# ------------ CONSTANTS ------------- #
INS_EMAIL = 'youremail#gmail.com'
INS_PASSWORD = 'yourpassword'
INS_URL = 'https://instagram.com'
class InstaFollower:
def __init__(self):
s = Service(ChromeDriverManager().install())
self.driver = webdriver.Chrome(service=s)
def login(self):
self.driver.get(INS_URL)
time.sleep(3)
username_input_tag = WebDriverWait(self.driver,
10).until(EC.presence_of_element_located((By.XPATH,
"//input[#name='username']")))
username_input_tag.send_keys(INS_EMAIL)
password_input_tag = WebDriverWait(self.driver,
10).until(EC.presence_of_element_located((By.XPATH,
"//input[#name='password']")))
password_input_tag.send_keys(INS_PASSWORD)
password_input_tag.send_keys(Keys.ENTER)
def find_followers(self):
time.sleep(2)
self.driver.get('https://www.instagram.com/geeks_for_geeks/')
# change the link above to your instagram link
time.sleep(3)
followers = WebDriverWait(self.driver,
10).until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT,
'followers')))
# The line above is optional, you can change followers to following
followers.click()
time.sleep(3)
try:
popup = self.driver.find_element(By.CSS_SELECTOR, '._aano')
except:
print 'FAILED TO FIND POPUP ELEMENT'
else:
print 'Popup element is found'
for run in range(100):
print(f"scrolling down {run}")
self.driver.execute_script('arguments[0].scrollTop = arguments[0].scrollHeight'
, popup)
time.sleep(2)
insta_follower_bot = InstaFollower()
insta_follower_bot.login()
time.sleep(2)
insta_follower_bot.find_followers()
I'm curious to know your result because I'm currently working on the similar project.
Added a lot of time.sleep because the libary's internet is so badddd
I'm trying to automate MS Teams and can't get past the login because after entering the password I am having issues clicking the Sign In button.
If I try:
button = driver.find_element_by_xpath('//*[#id="idSIButton9"]')
button.click()
I get:
ElementNotInteractableException: Message: element not interactable.
If I try to solve it with Action Chains method (which seems to have solved many similiar issues for lots of people):
button = driver.find_element_by_xpath('//*[#id="idSIButton9"]')
driver.implicitly_wait(10)
ActionChains(driver).move_to_element(button).click(button).perform()
I get:
StaleElementReferenceException: Message: stale element reference: element is not attached to the page document
It is worth mentioning that when logging in manually I can get past this stage with pressing return after entering my password but I couldn't be successful with send keys methods neither.
If you still filled the form use submit() function - It is there to make your life easier.
Import webdriver:
from selenium import webdriver
Create webdriver object:
driver = webdriver.Firefox()
Get the website:
driver.get("https://www.YOUR_WEBSITE.de/")
Get element - In your case input for password
element = driver.find_element_by_xpath('//*[#id="i0118"]')
Send keys - Enter password
element.send_keys("YOUR_PASSWORD")
Submit contents
element.submit()
The element which you are trying to locate using the XPath driver.find_element_by_xpath('//*[#id="idSIButton9"]') might be the part of the previous screen and new screen, hence the driver might be referring to the element which is part of the previous screen, hence you are getting the StaleElementReferenceException.
The simplest way to resolve this is you can add, driver.refresh() just before the above xpath, this will load the page and DOM with the new elements.
Else, you can use a different locator strategy which will uniquely identify that element.
I am trying to scrape some information off this website using python's selenium.
First, I log into the website and get to the page. Then, I would like to click on the tab "Quickscan" to scrape some info. However, that's where I get stuck. I can't find a way to click on the tab.
Note that this problem would be surmounted if I managed to navigate to the page, though when I log in, even if I put such page in my WebDriver, I still get redirected to this one.
To get to the desired page I have tried finding the element both through the xpath and through the link, but it does not find the element.
import requests
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver =webdriver.Chrome(executable_path ="mypath")
driver.get("https://vc4a.com/ventures/autocollect/#quickscan-tab")
#find username and password bar
username = driver.find_element_by_id("user_login")
password = driver.find_element_by_id("user_pass")
#Input password and username
username.send_keys("username")
password.send_keys("password")
#click on submit
driver.find_element_by_name("wp-submit").click()
driver.find_element_by_name("rememberme").click()
#try to find element using text in the link
driver.find_elements_by_link_text('#quickscan-tab')[0].click()
#try to find element using xpath from the inspected element
driver.find_element_by_xpath('//*[#id="subnav"]/li[3]/a').click()
I would like to be able to open the tab so that I can scrape the content.
When I use the first code it returns the following error:
IndexError: list index out of range
However, by inspecting the page I can see there is indeed 2 elements with the text "#quickscan-tab", so I don't understand why the index 0 would be out of range.
When I use the second code it returns the following error:
NoSuchElementException: no such element: Unable to locate element: {"method":"xpath","selector":"//*[#id="subnav"]/li[3]/a"}
(Session info: chrome=74.0.3729.169)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Mac OS X 10.14.5 x86_64)
What I did was just copying the xpath.
I created an account on that page and tried this modified script and it works:
import requests
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
driver = webdriver.Chrome()
driver.get("https://vc4a.com/ventures/autocollect/#quickscan-tab")
#find username and password bar
username = driver.find_element_by_id("user_login")
password = driver.find_element_by_id("user_pass")
#Input password and username
username.send_keys("username")
password.send_keys("password")
#click on submit
driver.find_element_by_name("rememberme").click()
driver.find_element_by_name("wp-submit").click()
time.sleep(10)
#try to find element using text in the link
driver.find_elements_by_link_text('Quickscan')[0].click()
#try to find element using xpath from the inspected element
driver.find_element_by_xpath('//a[text()="Quickscan"]').click()
link_text means the text that you actually see. [Quickscan]
Login takes time and the script tries to locate before the tab is created thus causing error.
Your xpath would have worked if not for the login delay.
Click rememberme before submitting the form. Or don't, since selenium starts a clean session for every run.
driver.find_elements_by_link_text('#quickscan-tab')[0].click() - it's wrong
Link text doesn't work like this you need to create a different locator. try below XPath
driver.find_element_by_xpath((//*[#id='quickscan-tab'])[0])
Try this:
scanelements = driver.find_elements_by_xpath('//*[#id='quickscan-tab']')
for elt in scanelements :
elt.click()
break
I am trying to automate reading my phone bill from the carrier website. www.fido.ca
However, the site is built with angularjs and I can't find the element using python and selenium webdriver. Please see below for the codes I've tried.
driver = webdriver.Firefox()
url = 'https://www.fido.ca/pages/#/login?m=login'
driver.get(url)
wait = WebDriverWait(driver, 10)
wait.until(EC.visibility_of_element_located((By.XPATH, "//a[#id='BC']")))
It returns selenium.common.exceptions.TimeoutException: Message:
Note: I can see the element from the front end, but no idea why webdriver can't see it.
When you navigate to a page, you would see the overlay "Welcome to Fido!" screen which makes your desired element invisible - hence the timeout error.
Handle the screen by selecting a region and clicking "Continue" or "X" (close).
I'm trying to automate the search process in this website: https://www.bcbsga.com/health-insurance/provider-directory/searchcriteria
The process involves clicking on the "Continue" button to search under the 'guest' mode. The next page has got a list of drop-down items to refine the search criteria. My code either produces the "Element not visible" exception (which I corrected by using a wait) or times out. Please help.
Here's my code:
# navigate to the desired page
driver.get("https://www.bcbsga.com/health-insurance/provider-directory/searchcriteria")
# get the guest button
btnGuest = driver.find_element_by_id("btnGuestContinue")
#click the guest button
btnGuest.click()
wait = WebDriverWait(driver,10)
#Find a Doctor Search Criteria page
element = wait.until(EC.visibility_of_element_located((By.ID,"ctl00_MainContent_maincontent_PFPlanQuestionnaire_ddlQuestionnaireInsurance")))
lstGetInsurance = Select(element)
lstGetInsurance.select_by_value("BuyMyself$14States")
# close the browser window
#driver.quit()
you can use input search and key.return:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
divID = 'ctl00_MainContent_maincontent_PFPlanQuestionnaire_ddlQuestionnaireInsurance_chosen'
inputID = 'ctl00_MainContent_maincontent_PFPlanQuestionnaire_ddlQuestionnaireInsurance_chosen_input'
inputValue = 'I buy it myself (or plan to buy it myself)'
driver = webdriver.Chrome()
driver.get("https://www.bcbsga.com/health-insurance/provider-directory/searchcriteria")
driver.find_element_by_id("btnGuestContinue").click()
driver.implicitly_wait(10)
driver.find_element_by_id(divID).click()
driver.find_element_by_id(inputID).send_keys(inputValue)
driver.find_element_by_id(inputID).send_keys(Keys.RETURN)
time.sleep(6)
driver.close()