Develop Reference

Python Selenium Webdriver Cannot Locate an Element on a Pop Up Page in Instagram

First of all sorry for my junky coding, I am not a coder of any sort.
I created (copied xd) an Instagram bot like a year ago, and it was working absolutely fine until two months ago. My goal with my Instagram bot is to identify new followers and unfollowers and stuff for my main account. I don't do rapid followings and unfollowings. The problem is when I open the following popup on Instagram, selenium cannot find any element that I specify. I've tried using XPATH, CSS Selector, by class name and all those methods. But I cannot even click the close button after I open that popup page, the code cannot find the button. Do you guys have any ideas why or any suggestions?
last_ht, ht = 0, 1
WebDriverWait(self.browser, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#class=_aano]")))
scroll_box = self.browser.find_element_by_xpath("//div[#class=_aano]")
while last_ht != ht:
last_ht = ht
time.sleep(2.5)
ht = self.browser.execute_script(
"arguments[0].scrollTo(0, arguments[0].scrollHeight);"
"return arguments[0].scrollHeight;", scroll_box)
The code above should wait for the Following button on your profile page and then click it (there is nothing wrong until there), and then wait for the popup to load and then scroll, but the WebDriverWait function raises an error, meaning the code cannot find the element, but like I can literally see it. It waits for 10 seconds.
I've tried Chrome and then Firefox, but it was even worse in Mozilla, it couldn't locate the Followings button on the profile page.

Since you haven't shared the applicable part of the DOM, my best guess is that the element you need to interact with is located inside an iframe. To interact with components inside an iframe, you must switch to the iframe first (which means you need to write a locator for the iframe, and then execute the rest of your code.
last_ht, ht = 0, 1
# you must provide the xpath string for the iframe
driver.switch_to.frame(driver.find_element_by_xpath("..."))
WebDriverWait(self.browser, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#class=_aano]")))
scroll_box = self.browser.find_element_by_xpath("//div[#class=_aano]")
while last_ht != ht:
last_ht = ht
time.sleep(2.5)
ht = self.browser.execute_script(
"arguments[0].scrollTo(0, arguments[0].scrollHeight);"
"return arguments[0].scrollHeight;", scroll_box)
# the rest of your code
Once you finish with the components in the iframe, you must switch back to the parent frame.
driver.switch_to.parent_frame()
Or switch to the top window
driver.switch_to.default_content()
Try them in that order. I am sure the first one will work for you.

My goal with my Instagram bot is to identify new followers and unfollowers and stuff for my main account
So you should click "followers" button instead of "following" button, like you said here.
The code above should wait for the Following button on your profile page
So, assume that's what you need, this code will help you scroll down the "followers" popup. I put the whole code here because i don't know if your code clicked the "followers" button or not. Also, the:
last_ht, ht = 0, 1
while last_ht != ht:
last_ht = ht
part will make your scrolling down action run only once, quite weird that they use 3 lines of code for doing nothing. just remove that part and your code will do exact same thing it previously does. So I use a for loop here instead.
Things to change to use the code:
Put your username and password in these two lines:
INS_EMAIL = "youremail#gmail.com"
INS_PASSWORD = "yourpassword"
Fix the link in this line to your ins account link:
self.driver.get("https://www.instagram.com/geeks_for_geeks/")
Change "followers" to "following" depend on your use case:
followers = WebDriverWait(self.driver, 10).until(
EC.presence_of_element_located((
By.PARTIAL_LINK_TEXT,
'followers')))
I'm using a 100 times for loop but if you want to scroll until it reaches the end of the file, guess I can change that later.
Next time you should post longer code so we can understand if other parts are working or not.
from selenium import webdriver
from selenium.webdriver import ActionChains
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.common.keys import Keys
import time
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import os
# ------------ CONSTANTS ------------- #
INS_EMAIL = 'youremail#gmail.com'
INS_PASSWORD = 'yourpassword'
INS_URL = 'https://instagram.com'
class InstaFollower:
def __init__(self):
s = Service(ChromeDriverManager().install())
self.driver = webdriver.Chrome(service=s)
def login(self):
self.driver.get(INS_URL)
time.sleep(3)
username_input_tag = WebDriverWait(self.driver,
10).until(EC.presence_of_element_located((By.XPATH,
"//input[#name='username']")))
username_input_tag.send_keys(INS_EMAIL)
password_input_tag = WebDriverWait(self.driver,
10).until(EC.presence_of_element_located((By.XPATH,
"//input[#name='password']")))
password_input_tag.send_keys(INS_PASSWORD)
password_input_tag.send_keys(Keys.ENTER)
def find_followers(self):
time.sleep(2)
self.driver.get('https://www.instagram.com/geeks_for_geeks/')
# change the link above to your instagram link
time.sleep(3)
followers = WebDriverWait(self.driver,
10).until(EC.presence_of_element_located((By.PARTIAL_LINK_TEXT,
'followers')))
# The line above is optional, you can change followers to following
followers.click()
time.sleep(3)
try:
popup = self.driver.find_element(By.CSS_SELECTOR, '._aano')
except:
print 'FAILED TO FIND POPUP ELEMENT'
else:
print 'Popup element is found'
for run in range(100):
print(f"scrolling down {run}")
self.driver.execute_script('arguments[0].scrollTop = arguments[0].scrollHeight'
, popup)
time.sleep(2)
insta_follower_bot = InstaFollower()
insta_follower_bot.login()
time.sleep(2)
insta_follower_bot.find_followers()
I'm curious to know your result because I'm currently working on the similar project.
Added a lot of time.sleep because the libary's internet is so badddd

How do I use Selenium and Python to click on a button when the text changes to something different?

I have been testing some new browser automation tools. One of them being Selenium. One thing I am working on is using Python to open a web page. Go to that web page and look for a certain button. If the button says "Yes Button" do nothing and just refresh the page. If the button changes after a refresh to "No Button" then click on that button. I will post the code below. The only thing I left out is my website address. Any help is really appreciated. Currently my script below does the refresh part but when the button changes to "No Button" it just stops and will not click on the button. I am not sure if my while loop is wrong or my understanding of Selenium.
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions
from selenium.webdriver.common.by import By
import time
################import the chrome web driver and define the location###############
PATH = "c:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
###################################################################################
###########open the web page and print the title##############
driver.get("https://mywebsite.com")
print(driver.title)
time.sleep(1)
##############################################################
#Look for search button and wait for it to change to something else.
while True:
searchbutton = WebDriverWait(driver, 60).until(expected_conditions.presence_of_element_located((By.LINK_TEXT, "Yes Button")))
driver.refresh()
else:
searchbutton = driver.find_element_by_link_text("No Button")
searchbutton.click()

else:
searchbutton = driver.find_element_by_link_text("No Button")
searchbutton.click()` It never reaches here because `While is always True
The program never reaches this part because it never gets out of While True. It should be something like this. Also Excpected Conditions will make the driver wait for the element to appear failing which, it will give a timeout error. So, we do it like this.
while True:
button1 = driver.find_element_by_xpath('//*[#id="comp-khm867e6"]/a/span')
if 'Yes' in button1.text:
driver.refresh()
time.sleep(10)
elif 'No' in button1.text:
button1.click()
break

Is there a way to click a list of multiple buttons, one by one, in selenium?

I am a student working on a scraping project with selenium on python. I am trying to scrape data from multiple profiles, that are all formatted the same. There is a directory website with buttons that lead to all of the profiles, but the problem that I am having is being able to click every button one at a time because they are all formatted the same. My goal is to be able to open the 1st link in a new tab, scrape the data from that profile, then close the first profile's tab and move on too the second. I hope to make this process repeatable. Here is what I have so far:
from selenium import webdriver
from selenium.webdriver.common.action_chains import ActionChains
import time
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome(executable_path="MY PATH TO MY CHROME DRIVER")
driver.implicitly_wait(5)
driver.get("http://directory.bcsp.org")
buttons = driver.find_elements_by_link_text('View Profile')
Please let me know if you have any solutions to my problem. Thank you :)

"body" for to go down on the page
"profile_count" for get link
".get_attribute('href')" for to scrape the link from the "View Profile" button
"temp" the information in the old window does not switch to the new window so we need temp
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from time import sleep
driver = webdriver.Edge()
driver.get("https://directory.bcsp.org/")
count = int(input("Count : "))
body = driver.find_element_by_xpath("//body") #
profile_count = driver.find_elements_by_xpath("//div[#align='right']/a")
while len(profile_count) < count: # Get links up to "count"
body.send_keys(Keys.END)
sleep(1)
profile_count = driver.find_elements_by_xpath("//div[#align='right']/a")
for link in profile_count: # Calling up links
temp = link.get_attribute('href') # temp for
driver.execute_script("window.open('');") # open new tab
driver.switch_to.window(driver.window_handles[1]) # focus new tab
driver.get(temp)
# you can do
# what you want
# to do here
driver.close()
driver.switch_to.window(driver.window_handles[0])
driver.close()

Python Selenium Webdriver Select Dropdown Value

I'm trying to automate the search process in this website: https://www.bcbsga.com/health-insurance/provider-directory/searchcriteria
The process involves clicking on the "Continue" button to search under the 'guest' mode. The next page has got a list of drop-down items to refine the search criteria. My code either produces the "Element not visible" exception (which I corrected by using a wait) or times out. Please help.
Here's my code:
# navigate to the desired page
driver.get("https://www.bcbsga.com/health-insurance/provider-directory/searchcriteria")
# get the guest button
btnGuest = driver.find_element_by_id("btnGuestContinue")
#click the guest button
btnGuest.click()
wait = WebDriverWait(driver,10)
#Find a Doctor Search Criteria page
element = wait.until(EC.visibility_of_element_located((By.ID,"ctl00_MainContent_maincontent_PFPlanQuestionnaire_ddlQuestionnaireInsurance")))
lstGetInsurance = Select(element)
lstGetInsurance.select_by_value("BuyMyself$14States")
# close the browser window
#driver.quit()

you can use input search and key.return:
import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
divID = 'ctl00_MainContent_maincontent_PFPlanQuestionnaire_ddlQuestionnaireInsurance_chosen'
inputID = 'ctl00_MainContent_maincontent_PFPlanQuestionnaire_ddlQuestionnaireInsurance_chosen_input'
inputValue = 'I buy it myself (or plan to buy it myself)'
driver = webdriver.Chrome()
driver.get("https://www.bcbsga.com/health-insurance/provider-directory/searchcriteria")
driver.find_element_by_id("btnGuestContinue").click()
driver.implicitly_wait(10)
driver.find_element_by_id(divID).click()
driver.find_element_by_id(inputID).send_keys(inputValue)
driver.find_element_by_id(inputID).send_keys(Keys.RETURN)
time.sleep(6)
driver.close()

Selenium with Python - Wait indefinitely until an input box is present

I want a WebDriver instance to monitor a page indefinitely until an input box appears with the name 'move.' Once the input box appears, I want to fill it with some text and click a submit button adjacent to the form. What is the easiest way to do this?
I have something like this now:
try:
move = WebDriverWait(driver, 1000).until(
EC.presence_of_element_located((By.NAME, "move"))
)
finally:
wd.quit()
And the button adjacent to the form has no name or id, so I am locating it by XPATH. I want to wait until that form is present before click the button.
How do I do this?

monitor a page indefinitely until an input box appears
An Explicit wait you've used in the example requires a timeout value defined. Either you set a very high value for the timeout, or it is not an option.
Alternatively, you can have a while True loop until an element would be found:
from selenium.common.exceptions import NoSuchElementException
while True:
try:
form = driver.find_element_by_name("move")
break
except NoSuchElementException:
continue
button = form.find_element_by_xpath("following-sibling::button")
button.click()
where I'm assuming the button element is a following sibling of the form.