Develop Reference

How to log into a website without input field and button ID using Selenium and Python [duplicate]

This question already has answers here:
What is the difference between "find_element_by_name('name')" and "find_element(By.NAME, 'name')"?
(2 answers)
Closed last year.
I'm creating a program that will help students track their marks, for that I need to web scrap the site where all our grades are stored. The idea was to automate the login and then there will be a separate script that will get the information that I need. I have trouble with automated login, because there is no input field and button ID. All I could find was names input fields and class for the button.
browser = webdriver.Chrome()
browser.get(("https://www.the-site.com/"))
usernameStr = "myUsername"
passwordStr = "myPassword"
username = browser.find_element("UserName")
username.send_keys(usernameStr)
password = browser.find_element("Password")
password.send_keys(passwordStr)
nextButton = browser.find_element("btn btn-default")
nextButton.click()
This is the error that I get:
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator
Here's the HTML for the input fields and button:
<form action="https://my.e-klase.lv?v=15" method="post" autocomplete="off">
<div class="form-group relative">
<input type="text" class="form-control" placeholder="Lietotājvārds"
autocomplete="off"
autocorrect="off"
autocapitalize="off"
spellcheck="false" name="UserName"
value="">
<div class="dummy-placeholder">Lietotājvārds</div>
</div>
<div class="form-group relative">
<input type="private" class="form-control upass"
placeholder="Parole"
autocomplete="off"
autocorrect="off"
autocapitalize="off"
name="Password">
<div class="dummy-placeholder">Parole</div>
</div>
<button type="submit" class="btn btn-default">
<span class="progress"></span>
<span class="content">Pieslēgties</span>
</button>
</form>

You can locate the elements based on any their unique attributes.
It can be id, class name(s), name and any other attribute values.
To give you more specific answer we need to see that web page elements HTML.
UPD
According to your HTML you can use the following locators:
browser = webdriver.Chrome()
browser.get(("https://www.the-site.com/"))
usernameStr = "myUsername"
passwordStr = "myPassword"
time.sleep(5)
username = browser.find_element_by_xpath("//input[#name='UserName']")
username.send_keys(usernameStr)
password = browser.find_element_by_xpath("//input[#name='Password']")
password.send_keys(passwordStr)
nextButton = browser.find_element_by_xpath("//button[#type='submit']")
nextButton.click()
This should basically work, but you should use ExpectedConditions explicit wait in order to wait for the page loaded

find_element()
find_element() finds an element given a By strategy and a locator.
You code was missing the By strategy and locator. Hence you see the error:
selenium.common.exceptions.InvalidArgumentException: Message: invalid argument: invalid locator
This usecase
As per the html you have shared, to fill in the UserName and Password field ideally you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
browser = webdriver.Chrome()
browser.get(("https://www.the-site.com/"))
WebDriverWait(browser, 20).until(EC.element_to_be_clickable((By.NAME, "UserName"))).send_keys("myUsername")
browser.find_element(By.NAME, "Password").send_keys("myPassword")
browser.find_element(By.XPATH, "//button[#class='btn btn-default']//span[text()='Pieslēgties']").click()

Click through dropdown menu with selenium

I am trying to create flashcards on quizlet.com with selenium. If you visit, you will see a "Create" button (or just a "+" depending on window size) in the navbar, when you click this it turns into a dropdown menu with 3 more buttons: 'Study Set', 'Folder' and 'Class'. (I am trying to click Study Set)
First, I am not even sure If I need to have selenium click the first 'Create' button to access the 'Study Set' button or if I can just jump straight to the 'Study Set' button. Anyway, here is the html related to the 'Create' button and 'Study Set' button, respectively:
<button type="button" aria-label="Create"><span>Create</span>/button>
and: (Note, all 3 buttons share the class 'UILink', the first button is the 'Study Set' button and the comments are mine)
<div class="s1ovpdog"> <!-- going to grab this div first because theres 85 elements with the class 'UILink' on the page, grabbing this div cuts it to 3-->
<button class="UILink" type="button"> <!-- then going to grab this button-->
<div class="c1ap9d88">
<div class="iiekfr8">
<div class="i1q3l8tw">
<svg aria-label="sets" class="AssemblyIcon AssemblyIcon--medium" role="img"><noscript></noscript>
<use xlink:href="#sets"></use><noscript></noscript></svg><span class="AssemblyMenuItem--title t1nsp0j0">Study set</span>
</div>
</div>
</div>
</button>
<button class="UILink" type="button">
<div class="c1ap9d88">
<div class="iiekfr8">
<div class="i1q3l8tw">
<svg aria-label="folder" class="AssemblyIcon AssemblyIcon--medium" role="img"><noscript></noscript>
<use xlink:href="#folder"></use><noscript></noscript></svg><span class="AssemblyMenuItem--title t1nsp0j0">Folder</span>
</div>
</div>
</div>
</button>
<button class="UILink" type="button">
<div class="c1ap9d88">
<div class="iiekfr8">
<div class="i1q3l8tw">
<svg aria-label="class" class="AssemblyIcon AssemblyIcon--medium" role="img"><noscript></noscript>
<use xlink:href="#class"></use><noscript></noscript></svg><span class="AssemblyMenuItem--title t1nsp0j0">Class</span>
</div>
</div>
</div>
</button>
</div>
Python code:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('./chromedriver')
driver.get("https://quizlet.com")
# Code to log in to quizlet omitted...
# Grab and click first 'Create' button
create_button = driver.find_element_by_xpath("//button[#aria-label='Create']") # only one element has this aria-label
create_button.send_keys(Keys.RETURN) # doesn't display drop-down menu, makes me think something is wrong here but also does not throw an error
# Grab and click 'Study Set' button
div_containting_study_set_button = driver.find_element_by_class_name('s1ovpdog') # only one element has this class
study_set_button = div_containting_study_set_button.find_elements_by_class_name('UILink')[0] # returns 3 buttons, only need first one
study_set_button.send_keys(Keys.RETURN) # Error,ElementNotInteractableException
When I run this, it throws
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.ElementNotInteractableException: Message: element not interac
table
Though study_set_button should definitely be referencing a button element, I believe? Thanks for any help with this.
EDIT:
Upon searching, I found is_displayed() and ran
create_button.is_displayed() # True
study_set_button.is_displayed() #False
this returns True and False, respectively. Think my problem lies somewhere there.

This is what I found to work. It was not easy to find a unique locator for "Study set" inside the dropdown. There are two "Study set" elements on the page and the first one in the DOM is not visible. I added the waits just to be safe since you are clicking and the dropdown has to load. You may not need the waits but it won't hurt to have them (it won't slow anything down) just in case.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button[aria-label='Create']"))).click()
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.UIOverlayTrigger-content svg[aria-label='sets'] + span"))).click()

selenium unable to locate input object in iframe

My goal is to automatically fill out the form at https://mail.protonmail.com/create/new?language=en using selenium.
I am able to fill in a password but I am still struggling to fill a username in.
The username field is inside an iframe. The HTML for said iframe is the following:
<iframe title="Registration form" scrolling="no" class="top" data-name="top" sandbox="allow-scripts allow-same-origin allow-popups allow-top-navigation" src="https://secure.protonmail.com/abusev2.iframe.html?name=top">
#document
<!DOCTYPE html>
<html lang="en">
<head>...</head>
<body>
<div id="app" data-name="top">
<div class="formList">
<div class="field field-usernameInput">
<div class="group-username">
<label for="username">Choose username</label>
<input placeholder="Choose username" required="" name="username" messages="[object Object]" iframename="top" pattern=".{1,40}" id="username" class="input">
<div class="field field-select"><label for="domain">Select a domain</label><div class="select-mask"><select component="domains" name="domain" id="domain"><option value="protonmail.com">protonmail.com</option><option value="protonmail.ch">protonmail.ch</option></select><i>▼</i></div></div>
</div>
</div>
</div>
</div>
</iframe>
In order to switch selenium to the iframe I used
driver.switch_to.frame(driver.find_element_by_tag_name('iframe'))
That does not give me an exception.
However I am now unable to find the input object to do .send_keys().
I've tried
driver.find_element_by_id('username')
driver.find_element_by_class_name('input')
driver.find_element_by_tag_name('input')
driver.find_element_by_name('username')
driver.find_element_by_xpath('//input[#id="username"]')
driver.find_element_by_css_selector('#username')
but every time I get the same exception:
selenium.common.exceptions.NoSuchElementException: Message: Unable to locate element: *whatever I searched for*
When I tried
driver.find_element_by_id('app') #first div object in the iframe
I didn't get any exceptions.
If I don't switch to the iframe I cannot find this which leads me to believe that switching to the iframe worked.
I am however unable to find any of the other div objects with any of the mentioned methods.
To conclude: I have no idea why I am unable to find the input object with selenium to automatically put in a username afterwords and would greatly appreciate ideas on what I am doing wrong.

That would get the first iframe use a more targeted xpath.
driver.switch_to.frame(driver.find_element_by_xpath("(//iframe[#title='Registration form'])[1]"))
So to target the first iframe and wait for the username and send a key to it.
wait = WebDriverWait(driver, 10)
driver.get('https://mail.protonmail.com/create/new?language=en')
wait.until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,"(//iframe[#title='Registration form'])[1]")))
wait.until(EC.element_to_be_clickable((By.XPATH,"(//div[#class='group-username'])[1]/input"))).send_keys('a')
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Python, selenium chrome driver to fill form (element ids' changing)

a part of company internal webpage looks like this:
It's the first form of the webpage. I want to key in numbers into this form.
The problem is that, the element id is changing when the webpage is refreshed. So I tired driver.find_element_by_xpath and driver.find_element_by_css_selector, it's not catching the form everytime.
So I am thinking maybe locate the line of text "Customer No.", then move to its next element, could be a choice. but again, the element id of "Student No." is also changing.
Is there a way to catch the form by the text "Student No."? Or there's a better option? thank you.
part of HTML code as below:
<div class="labeledField">
<div class="fieldLabel beanRequiredField">Student No.</div>
<div class="field"><span id="id16">
<div>
<input type="text" class="inputField" value="" name="tabs:r:0:c:0:c:field:c:p:r:0:c:0:c:field:c:frag:component" id="id4e" customid="StudentId" autocomplete="off">
</div>
</span></div>
</div>

driver.find_elements_by_class_name('inputField')[0].send_keys('Text')
It works!

Maybe this will solve your problem:
WebDriverWait wait = new WebDriverWait(driver, 15);
wait.until(ExpectedConditions.visibilityOfElementLocated(By.xpath("//input[#id='text3']")));

If you want to base on "Student No." string, then you can go with XPath selector:
//div[text()="Student No."]/../div[2]/span/div/input
So for Python it would be something like this:
input = driver.find_element(By.XPATH, '//div[text()="Student No."]/../div[2]/span/div/input')

Selenium with Python: First instance of the element is identified but the next occurance is ElementNotVisibleException

I have the following Selenium Test for Python/Django application:
class EmailRecordsTest(StaticLiveServerTestCase):
def test_can_store_email_and_retrieve_it_later(self):
self.browser.get(self.live_server_url)
emailbox = self.browser.find_element_by_xpath("//form[#class='pma-subscribe-form']/input[1]")
self.assertEqual(emailbox.get_attribute("placeholder"), 'Enter your Email')
print("tested until here")
print("The placeholder: ", emailbox.get_attribute("placeholder"))
print(emailbox)
emailbox.send_keys('vio#mesmerizing.com')
First occurance of emailbox is clearly identified as seen from the print runs and assert Equal for placeholder. The last instance of emailbox.send_keys throws following error:
selenium.common.exceptions.ElementNotVisibleException: Message:
Element is not currently visible and so may not be interacted with
Cannot find why the same element become Not Visible when using with send_keys.
The Html code being tested is as below:
<!-- Start footer -->
<footer id="pma-footer">
<!-- start footer top -->
<div class="pma-footer-top">
<div class="container">
<div class="pma-footer-top-area">
<div class="row">
<div class="col-lg-3 col-md-3 col-sm-3">
<div class="pma-footer-widget">
<h4>News letter</h4>
<p>Get latest update, news & offers</p>
<form class="pma-subscribe-form">
<input id="subscribe-email" type="email" placeholder="Enter your Email">
<button class="btn btn-danger btn-md" type="submit">Subscribe!</button>
</form>
</div>
</div>
</div>
</div>
</div>
</div>
<!-- end footer top -->
Kindly help.

Actually find_element returns element which would be present on the DOM no matter it's visible or not and you can get attribute of this element as well but send_keys does an action on element and selenium does action only visible element, So you need to be sure before doing action on element that it's visible using WebDriverWait as below :-
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
wait = WebDriverWait(driver, 10)
emailbox = wait.until(EC.visibility_of_element_located((By.ID, "subscribe-email")))
#do your all stuff before send keys
# now use send_keys
emailbox.send_keys('vio#mesmerizing.com')
Edited :- If you are still unable to interact with element try using execute_script() to set value as below :-
emailbox = wait.until(EC.presence_of_element_located((By.ID, "subscribe-email")))
#do your all stuff before send keys
# now use execute_script
driver.execute_script("arguments[0].value = 'vio#mesmerizing.com'", emailbox)

Another option which worked in this case is that you scroll to the specific element (which was at the bottom of the page)and then use send_keys it works.
emailbox = self.browser.find_element_by_xpath("//form[#class='mu-subscribe-form']/input[1]")
self.browser.execute_script("window.scrollTo(0, document.body.scrollHeight);")
emailbox.send_keys('vio#mesmerizing.com')