Develop Reference

Why Selenium method get(), doesnt work second time?

Help me figure out why when using the get () method a second time, the page does not go? The method works only if you use a time delay time.sleep ()
Not working:
LOGIN = 'something#mail.com'
PASS = 'somepass'
LINK = 'https://stepik.org/'
browser = webdriver.Chrome()
browser.get(LINK)
browser.implicitly_wait(5)
browser.find_element_by_id('ember232').click()
username = browser.find_element_by_name('login').send_keys(LOGIN)
pas = browser.find_element_by_name('password').send_keys(PASS)
button = browser.find_element_by_xpath('//button[#type = "submit"]').click()
browser.get('https://stepik.org/lesson/237240/step/3?unit=209628')
Working
LOGIN = 'something#mail.com'
PASS = 'somepass'
LINK = 'https://stepik.org/'
browser = webdriver.Chrome()
browser.get(LINK)
browser.implicitly_wait(5)
browser.find_element_by_id('ember232').click()
username = browser.find_element_by_name('login').send_keys(LOGIN)
pas = browser.find_element_by_name('password').send_keys(PASS)
button = browser.find_element_by_xpath('//button[#type = "submit"]').click()
time.sleep(5)
browser.get('https://stepik.org/lesson/237240/step/3?unit=209628')

You are trying to login into the web site and then to navigate to some internal page.
By clicking the submit button
button = browser.find_element_by_xpath('//button[#type = "submit"]').click()
You are trying to log into the site.
This process takes some time.
So if immediately after clicking the submit page, while login is still not proceed, you are trying to navigate to some internal page this will not work since you still not logged in.
However you do not need to use a hardcoded sleep of 5 seconds.
You can use an explicit wait of expected conditions like presence_of_element_located() of some internal element to indicate you are inside the web site. Once this condition is fulfilled you can navigate to the desired internal page.

Try an alternative way:
driver.navigate().to("https://stepik.org/lesson/237240/step/3?unit=209628")

Injecting token received from anticaptcha for recaptcha v2 using selenium python

I am using anticaptcha to help out with bypassing recaptcha on a webpage I'm crawling.
I have managed to work out the api part of this solution. It's quite straightforward.
The part I am struggling with is the injection of the token received from anti-captcha into the webpage.
Haven't found too many resources on this. I am using Selenium and Python alongside the anticaptchaofficial module.
The script I am executing does change the innerHtml of the textarea with id g-recaptcha-response but the webpage does nothing and the checkbox doesn't load the spinner or get verified.
Here's my code:
from anticaptchaofficial.recaptchav2proxyless import recaptchaV2Proxyless
from selenium import webdriver
import os
import time
driver = webdriver.Chrome(os.path.normpath(os.getcwd()+"\\chromedriver.exe"))
driver.get("https://www.google.com/recaptcha/api2/demo")
time.sleep(1)
data_sitekey = driver.find_element_by_class_name('g-recaptcha').get_attribute('data-sitekey')
solver = recaptchaV2Proxyless()
solver.set_verbose(1)
solver.set_key("<--my-key-->")
solver.set_website_url("https://www.google.com/recaptcha/api2/demo")
solver.set_website_key(data_sitekey)
g_response = solver.solve_and_return_solution()
driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "{}";'.format(g_response)) # target textarea that is supposed to be injected with the token, I found upon some research
driver.execute_script("onSuccess('{}')".format(g_response))
time.sleep(1)

Turns out I was under the assumption that the recaptcha frame would show visible feedback on injection of the token (or some other equivalent action) but it turns out just the line:
driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "{}";'.format(g_response))
which updates the textarea's innerHtml is enough. So you would basically need to continue with your task ie: click submit, if it is a recaptcha on form or reload the page if it is just randomly triggered
from anticaptchaofficial.recaptchav2proxyless import recaptchaV2Proxyless
from selenium import webdriver
import os
import time
driver = webdriver.Chrome(os.path.normpath(os.getcwd()+"\\chromedriver.exe"))
driver.get("https://www.google.com/recaptcha/api2/demo")
time.sleep(1)
data_sitekey = driver.find_element_by_class_name('g-recaptcha').get_attribute('data-sitekey')
solver = recaptchaV2Proxyless()
solver.set_verbose(1)
solver.set_key("<--my-key-->")
solver.set_website_url("https://www.google.com/recaptcha/api2/demo")
solver.set_website_key(data_sitekey)
g_response = solver.solve_and_return_solution()
driver.execute_script('document.getElementById("g-recaptcha-response").innerHTML = "{}";'.format(g_response))
time.sleep(1)
# whatever the next step is. Could be clicking on a submit button
driver.refresh()

Logging into Gmail using Python with detecting failed login

I know that multiple people have asked a similar question about this however, I would like to know how to login to gmail (or google account) using python. I have a code already (see below) that can loggin the user to gmail using selenium. However I noticed a few problems.
The browser closes when the program stops/closes.
It can not detect a failed login.
Both problems really need to be solved for me to be able to work on my project. I don't mind using something else than selenium like pyautogui to open google. However, it needs to be able to detect a failed login and then close the browser, if the login is successful the browser should stay open for the user to use.
My code:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
def gmail_login(username, password):
gmailId = username
passWord = password
driver = webdriver.Chrome(ChromeDriverManager().install())
try:
driver.get(r'https://accounts.google.com/signin/v2/identifier?continue=' + \
'https%3A%2F%2Fmail.google.com%2Fmail%2F&service=mail&sacu=1&rip=1' + \
'&flowName=GlifWebSignIn&flowEntry = ServiceLogin')
driver.implicitly_wait(3)
loginBox = driver.find_element_by_xpath('//*[#id ="identifierId"]')
loginBox.send_keys(gmailId)
nextButton = driver.find_elements_by_xpath('//*[#id ="identifierNext"]')
nextButton[0].click()
passWordBox = driver.find_element_by_xpath(
'//*[#id ="password"]/div[1]/div / div[1]/input')
passWordBox.send_keys(passWord)
nextButton = driver.find_elements_by_xpath('//*[#id ="passwordNext"]')
nextButton[0].click()
except:
driver.close()
gmail_login("email#gmail.com", "Password")
I tought of checking the url after the program has finished if it is equal to a logged in url however that didn't really work too well and now I am out of ideas.

Updated 02-14-2021
I was able to extract this error message:
using this code:
signin_failure = driver.find_elements_by_xpath("//*[contains(text(),'Couldn’t sign you in')]")
if signin_failure[1].text == 'Couldn’t sign you in':
print('something went wrong')
In a normal gmail login you will get one of these two error messages:
Couldn’t find your Google Account
Wrong password. Try again or click Forgot password to reset it
The XPATH to get these error messages is:
wrong_email = driver.find_elements_by_xpath("//*[contains(text(),'Couldn’t find your Google Account')]")
wrong_password = driver.find_elements_by_xpath("//*[contains(text(),'Wrong password. Try again or click Forgot password to reset it')]")
if you want to close the browser after an error message, such as Couldn’t sign you in then add a driver.close() statement.
signin_failure = driver.find_elements_by_xpath("//*[contains(text(),'Couldn’t sign you in')]")
if signin_failure[1].text == 'Couldn’t sign you in':
print('something went wrong')
driver.close()
If you want to keep the browser open then don't use the driver.close() statement, but add this experimental_option
chrome_options.add_experimental_option("detach", True)
I was also able to throw these error messages:
signin_failure = driver.find_elements_by_xpath("//*[contains(text(),'Couldn’t sign you in')]")
# this message was throw when the next button was clicked prior to entering a username
no_input = driver.find_elements_by_xpath("//*[contains(text(),'Enter a valid email of phone number')]")
PSEUDOCODE CODE:
This is how you could do it, but you might have to adjust the code as you test.
def gmail_login(username, password):
driver = webdriver.Chrome(ChromeDriverManager().install())
try:
driver.get(r'https://accounts.google.com/signin/v2/identifier?continue=' + \
'https%3A%2F%2Fmail.google.com%2Fmail%2F&service=mail&sacu=1&rip=1' + \
'&flowName=GlifWebSignIn&flowEntry = ServiceLogin')
driver.implicitly_wait(3)
loginBox = driver.find_element_by_xpath('//*[#id ="identifierId"]')
loginBox.send_keys(username)
nextButton = driver.find_elements_by_xpath('//*[#id ="identifierNext"]')
nextButton[0].click()
wrong_email = driver.find_elements_by_xpath("//*[contains(text(),'Couldn’t find your Google Account')]")
# you need to check this slice
if wrong_email[1].text == 'Couldn’t find your Google Account':
print('something went wrong')
driver.close()
else:
passWordBox = driver.find_element_by_xpath('//*[#id ="password"]/div[1]/div / div[1]/input')
passWordBox.send_keys(password)
nextButton = driver.find_elements_by_xpath('//*[#id ="passwordNext"]')
nextButton[0].click()
wrong_password = driver.find_elements_by_xpath("//*[contains(text(),'Wrong password. Try again or click Forgot password to reset it')]")
# you need to check this slice
if wrong_password[1].text == 'Wrong password. Try again or click Forgot password to reset it':
print('something went wrong')
driver.close()
except:
driver.close()
Original post
Google forbids using automated scripts for logging into Gmail. I have tried using selenium and I get this warning.
When I click Learn More I get this message.
Please note this line: Are being controlled through software automation rather than a human
Here are other question where they discuss work arounds for this Google direct login issue:
“This browser or app may not be secure” error while attempting to login in to Gmail account using Selenium
GMail is blocking login via Automation (Selenium)

You should be looking at the Gmail API for programmatic access, it will work a lot better than trying to drive the UI like selenium does.
https://developers.google.com/gmail/api/quickstart/python

I have no time right now to code, but:
To make your browser stay open just delete your driver.close() function, this is what makes it to stop when program reach it.
To solve your problem, just try to make a successfull login and a failing one, look for a WebElement in both that indicates one of the events in an unique way, then, put the driver.close() in an if declaration only if selenium finds the WebElement in viewport after the login (failed one), otherwise do not let it execute the instruction.
Easy as that.

always "wrong password" message in selenium automated login

I'm trying to automate a duolingo login with Selenium with the code posted below.
While everything seems to work as expected at first, I always get an "Wrong password" message on the website after the login button is clicked.
I have checked the password time and time again and even changed it to one without special characters, but still the login fails.
I have seen in other examples that there is sometimes an additional password input field, however I cannot find one while inspecting the html.
What could I be missing ?
(Side note: I'm also open to a completely different solution without a webdriver since I really only want to get to the duolingo.com/learn page to scrape some data, but as of yet I haven't found an alternative way to login)
The code used:
from selenium import webdriver
from time import sleep
url = "https://www.duolingo.com/"
def login():
driver = webdriver.Chrome()
driver.get(url)
sleep(2)
hve_acnt_btn = driver.find_element_by_xpath("/html/body/div/div/div/span[1]/div/div[1]/div[2]/div/div[2]/a")
hve_acnt_btn.click()
sleep(2)
email_input = driver.find_element_by_xpath("/html/body/div[1]/div[3]/div[2]/form/div[1]/div/label[1]/div/input")
email_input.send_keys("email#email.com")
sleep(2)
pwd_input = driver.find_element_by_css_selector("input[type=password]")
pwd_input.clear()
pwd_input.send_keys("password")
sleep(2)
login_btn = driver.find_element_by_xpath("/html/body/div[1]/div[3]/div[2]/form/div[1]/button")
login_btn.click()
sleep(5)
login()
I couldn't post the website's html because of the character limit, so here is the link to the duolingo page: Duolingo

Switch to Firefox or a browser which does not tell the page that you are visiting it automated. See my earlier answer for a very similar issue here: https://stackoverflow.com/a/57778034/8375783
Long story short: When you start Chrome it will run with navigator.webdriver=true. You can check it in console. Pages can detect that flag and block login or other actions, hence the invalid login. This is a read-only flag set by the browser during startup.
With Chrome I couldn't log in to Duolingo either. After I switched the driver to Firefox, the very same code just worked.
Also if I may recommend, try to use Xpath with attributes.
Instead of this:
hve_acnt_btn = driver.find_element_by_xpath("/html/body/div/div/div/span[1]/div/div[1]/div[2]/div/div[2]/a")
You can use:
hve_acnt_btn = driver.find_element_by_xpath('//*[#data-test="have-account"]')
Same goes for:
email_input = driver.find_element_by_xpath("/html/body/div[1]/div[3]/div[2]/form/div[1]/div/label[1]/div/input")
vs:
email_input = driver.find_element_by_xpath('//input[#data-test="email-input"]')

How to get HTML code after logging in?

I am quite new to Selenium, it would be great if you guys can point me to the right direction.
I'm trying to access the HTML code of a website AFTER the login sequence.
I've used Selenium to direct the browser to initiate the login sequence, the part of the HTML I need will show up after I login. But when I tried to call the HTML code after the login sequence with page_source, it just gave me the HTML code for the site before logging in.
def test_script(ticker):
base_url = "http://amigobulls.com/stocks/%s/income-statement/quarterly" %ticker
driver = webdriver.Firefox()
verificationErrors = []
accept_next_alert = True
driver.get(base_url)
driver.maximize_window()
driver.implicitly_wait(30)
driver.find_element_by_xpath("//header[#id='header_cont']/nav/div[4]/div/span[3]").click()
driver.find_element_by_id("login_email").clear()
driver.find_element_by_id("login_email").send_keys(email)
driver.find_element_by_id("login_pswd").clear()
driver.find_element_by_id("login_pswd").send_keys(pwd)
driver.find_element_by_id("loginbtn").click()
amigo_script = driver.page_source