Develop Reference

Selenium webdriver can't find visible element

I have a problem with finding element on a webpage. It's a popup window with a confirmation [NO]/[YES]. Selenium can find a parent of a buttons but not a button itself.
Popup window view
HTML block of code:
<div tabindex="-1" role="alertdialog" style="position: fixed; height: auto; width: 300px; top: 188.5px; left: 800px; max-width: 100%;"
class="ui-dialog ui-dialog--notification ui-dialog--modern ui-widget ui-widget-content ui-front ui-dialog-buttons ui-draggable"
aria-describedby="ui-id-3" aria-labelledby="ui-id-4">
<div class="ui-dialog-titlebar ui-corner-all ui-widget-header ui-helper-clearfix ui-draggable-handle"></div>
<div class="ui-dialog-content ui-widget-content" style="width: auto; min-height: 0px; max-height: none; height: 32px;">
<div class="a-AlertMessage">
<div class="a-AlertMessage-body">
<div class="a-AlertMessage-details" id="ui-id-3">Czy chcesz wygenerować pozycje dla całej dostawy?</div>
</div>
</div>
</div>
<div class="ui-dialog-buttonpane ui-widget-content ui-helper-clearfix"><div
class="ui-dialog-buttonset">
<button type="button" class="ui-button ui-corner-all ui-widget">Nie</button>
<button type="button" class="js-confirmBtn ui-button ui-corner-all ui-widget ui-button--hot">Tak</button>
</div>
</div>
</div>
Here is a code snippet that selenium can't find and gives an error 'Unable to locate element':
pop_up_yes_button = driver.find_element(By.XPATH, '//*[#id="t_PageBody"]/div[15]/div[3]/div/button[2]').click()
Parent is avaible for selenium without a problem:
driver.find_element(By.XPATH, '//*[#id="t_PageBody"]/div[15]')
I also tried WebDriverWait and ExpectedConditions but it gives TimeoutException:
WebDriverWait(driver, 15).until(EC.presence_of_element_located((By.XPATH, '//*[#id="t_PageBody"]/div[15]/div[3]/div/button[2]'))).click()

As you said that it is a popup window, it is very likely that the button is in an iFrame. You have to wait until the iFrame loads and then search the element again.
Luckily, selenium has a built in function for dealing with iFrames:
your_button = WebDriverWait(driver,
20).until(EC.frame_to_be_available_and_switch_to_it((By.XPATH,'//*[#id="t_PageBody"]/div[15]/div[3]/div/button[2]')))
your_button.click()
Don't just copy and paste this code, understand this thing and implement it.

How can I programmatically download Tableau csv file from website with Python?

I am attempting to download the data from the following site programmatically with Python: https://health.wyo.gov/publichealth/infectious-disease-epidemiology-unit/disease/novel-coronavirus/covid-19-testing-data/
At the bottom right of the Tableau view there are 3 buttons: Share, Download, and Full Screen. After clicking Download, you are then taken to another pop up. I am then wanting to select Crosstab that then takes you to another pop up where I want to select Positivity and finally, Download which provides a csv.
I have essentially managed to navigate through some of the iframes but am a little lost on where to click since the buttons do not include id/link.
Below is some code from one approach:
from selenium import webdriver
url = 'https://public.tableau.com/profile/melissa.taylor#!/vizhome/WyomingCOVID-19TestingDataDashboard/Dashboard1'
driver = webdriver.Chrome()
driver.get(url)
#elem = driver.switch_to.frame(driver.find_element_by_xpath('//iframe[contains(text(), "googletagmanager"]'))
try:
time.sleep(4)
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to.default_content()
driver.switch_to.frame(iframe)
driver.find_elements_by_tag_name('iframe')
#driver.find_element_by_id('download-ToolbarButton').click()
print(driver.page_source)
finally:
driver.quit()
Below is the HTML that displays the 3 buttons from the Tableau page.
<div class="tab-nonVizItems tab-fill-right hideLabels"><div class="tabToolbarButton tab-widget undo disabled" role="button" data-tb-test-id="undo-ToolbarButton" id="undo-ToolbarButton" aria-disabled="true" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Undo"><span class="tabToolbarButtonImg tab-icon-undo"></span><span class="tabToolbarButtonText">Undo</span></div><div class="tabToolbarButton tab-widget redo disabled" role="button" data-tb-test-id="redo-ToolbarButton" id="redo-ToolbarButton" aria-disabled="true" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Redo"><span class="tabToolbarButtonImg tab-icon-redo"></span><span class="tabToolbarButtonText">Redo</span></div><div class="tabToolbarButton tab-widget revert disabled" role="button" data-tb-test-id="revert-ToolbarButton" id="revert-ToolbarButton" aria-disabled="true" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Reset"><span class="tabToolbarButtonImg tab-icon-revert"></span><span class="tabToolbarButtonText">Reset</span></div><div class="tabToolbarButton tab-widget share" role="button" data-tb-test-id="share-ToolbarButton" id="share-ToolbarButton" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Share"><span class="tabToolbarButtonImg tab-icon-share"></span><span class="tabToolbarButtonText">Share</span></div><div class="tabToolbarButton tab-widget download" role="button" data-tb-test-id="download-ToolbarButton" id="download-ToolbarButton" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Download"><span class="tabToolbarButtonImg tab-icon-download"></span><span class="tabToolbarButtonText">Download</span></div><div class="tabToolbarButton tab-widget enterFullscreen" role="button" data-tb-test-id="toggle-fullscreen-ToolbarButton" id="toggle-fullscreen-ToolbarButton" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Full Screen"><span class="tabToolbarButtonImg tab-icon-enterFullscreen"></span><span class="tabToolbarButtonText">Full Screen</span></div></div>
<div></div></div><div class="tab-ReactView tab-toolbar-dialoghost"></div></div></div>
Thanks for your help!!

Try the below code:
driver.get('https://public.tableau.com/profile/melissa.taylor#!/vizhome/WyomingCOVID-19TestingDataDashboard/Dashboard1')
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[title='Data Visualization']")))
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".tab-icon-download"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Crosstab']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='positivity']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Download']"))).click()
Following import:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

Clicking on Javascript tab using Selenium and Python without unique class id or element name

I have this HTML element code which I am currently struggling to figure out to use it for clicking on the tab that says Problem. As the "Problem" doesnt have a unique classname or element ID, I am unable to figure how to send a Click().
I have tried to check if z-index can be used as index(assumed) and used below line of code
browser.switch_to_frame(a[3])
but it seems I am wrong.
HTML code as below
<div class="TabsViewPort" style="position: relative; overflow: hidden; width: 896px; height: 22px; float: left;">
<div style="overflow: visible; float: left; width: 897px; top: 0px; left: 0px;">
<dl class="OuterOuterTab">
<dd class="OuterTab" artabid="955000038" arwindowid="0" style="top: 1px; z-index: 1; left: 0px; visibility: inherit; display: block;"><span class="TabLeftRounded"> </span>
<span class="Tab">My Profile
</span>
<span class="TabRight"> </span>
</dd>
<dd class="OuterTabSelected" artabid="600000203" arwindowid="0" style="top: 1px; z-index: 3; left: 63px; visibility: inherit; display: block;"><span class="TabLeft"> </span>
<span class="Tab">Approval
</span>
<span class="TabRight"> </span>
</dd>
<dd class="OuterTab" artabid="536870915" arwindowid="0" style="top: 1px; z-index: 1; left: 409px; visibility: inherit; display: block;"><span class="TabLeft"> </span>
<span class="Tab">Problem
</span>
<span class="TabRight"> </span>
</dd>
</dl>
</div>
</div>

The element with text as Problem is a JavaScript enabled element so to click() on the element you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using XPATH A:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='TabsViewPort']//dl[#class='OuterOuterTab']//dd[#class='OuterTab']//a[#class='btn f1' and text()='Problem']"))).click()
Using XPATH B (shortened):
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='btn f1' and text()='Problem']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

If the element present inside an iframe then you need to switch to iframe first to access the element.
You can Use following method to frame_to_be_available_and_switch_to_it()
By locator ID
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"id of the iframe")))
OR
By locator NAME
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"name of the iframe")))
Once You have switched to iframe you can access the element using following xpath
To click on the element Induce WebDriverWait and element_to_be_clickable()
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//span[#class='Tab']//a[text()='Problem']"))).click()
You need to import following to execute above code.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Hope this will help.

please find below xpath to click on third TAB
(//span[#class="Tab"])[3]/a

Use selenium to write in and run a code cell in jupyterlab

I would like to test a patched implementation of juptyerlab. I was hoping to use selenium to execute "hello world" in a code cell. So far I can log in and create a new notebook:
from selenium import webdriver
driver = webdriver.Firefox()
# assume jupyterlab is running and serving on localhost at port 8888
driver.get("http://localhost:8888")
elem = driver.find_element_by_id("password_input")
password = ""
elem.send_keys(password)
elem = driver.find_element_by_id("login_submit")
elem.click()
elem = driver.find_element_by_css_selector(".jp-Launcher-cwd+ .jp-Launcher-section .jp-LauncherCard")
elem.click()
This creates a new notebook, but now I'm stuck at the point of entering some code in a cell and running it. If I view the page source I don't see any html elements for the cells. But if I enter print("test") in a cell, then driver.page_source contains this (it's pretty nested in other stuff I've omitted too):
<div class="CodeMirror cm-s-jupyter CodeMirror-wrap jp-mod-readOnly">
<div style="overflow: hidden; position: relative; width: 3px; height: 0px; top: 0px; left: 0px;">
<textarea
style="position: absolute; bottom: -1em; padding: 0px; width: 1px; height: 1em; outline: currentcolor none medium;"
autocorrect="off" autocapitalize="off"
spellcheck="false" tabindex="0"
wrap="off"></textarea></div>
<div class="CodeMirror-vscrollbar" tabindex="-1"
cm-not-content="true"
style="display: block; bottom: 0px;">
<div style="min-width: 1px; height: 33px;"></div>
</div>
<div class="CodeMirror-hscrollbar" tabindex="-1"
cm-not-content="true">
<div style="height: 100%; min-height: 1px; width: 0px;"></div>
</div>
<div class="CodeMirror-scrollbar-filler"
cm-not-content="true"></div>
<div class="CodeMirror-gutter-filler"
cm-not-content="true"></div>
<div class="CodeMirror-scroll" tabindex="-1" draggable="true">
<div class="CodeMirror-sizer"
style="margin-left: 0px; padding-right: 0px; padding-bottom: 0px;">
<div style="position: relative;">
<div class="CodeMirror-lines" role="presentation">
<div style="position: relative; outline: currentcolor none medium;"
role="presentation">
<div class="CodeMirror-measure">
<pre><span>xxxxxxxxxx</span></pre>
</div>
<div class="CodeMirror-measure">
<pre class="CodeMirror-line"
role="presentation"><span
role="presentation"><span
class="cm-builtin">print</span>(<span
class="cm-string">"test"</span>)</span></pre>
</div>
<div style="position: relative; z-index: 1;"></div>
<div class="CodeMirror-cursors"></div>
<div class="CodeMirror-code"
role="presentation"></div>
</div>
</div>
</div>
</div>
<div style="position: absolute; height: 30px; width: 1px; border-bottom: 0px solid transparent;"></div>
<div class="CodeMirror-gutters"
style="display: none;"></div>
</div>
</div>
I can see where the text for print("text") is (i.e. the deepest nested elements in the above html snippet), but I can't figure out which element here I would be able to send text to or send keys to.
I came across robotframework-jupyterlibrary and it has some clues such as this and this . From those links I see
${JLAB CSS ACTIVE INPUT} ${JLAB CSS ACTIVE CELL} .CodeMirror
and
Add and Run JupyterLab Code Cell
[Arguments] ${code}=print("hello world")
[Documentation] Add a ``code`` cell to the currently active notebook and run it.
Click Element css:${JLAB CSS NB TOOLBAR} ${JLAB CSS ICON ADD}
Sleep 0.1s
${cell} = Get WebElement css:${JLAB CSS ACTIVE INPUT}
Click Element ${cell}
Set CodeMirror Value ${JLAB CSS ACTIVE INPUT} ${code}
Run Current JupyterLab Code Cell
Click Element ${cell}
which makes me think if I select the .CodeMirror element, then I just need to figure out what Get WebElement does in that weird language and how to do it in selenium.
Any ideas?
I've also tried (based on https://stackoverflow.com/a/48723135/1011724 and https://stackoverflow.com/a/50279295/1011724):
from selenium.webdriver.common.action_chains import ActionChains
actions = action_chains.ActionChains(driver)
textarea = driver.find_elements_by_css_selector('.CodeMirror textarea')[0] # tried for [0], [1] ,[2] and [3] which is all of them.
actions.move_to_element(textarea).click().send_keys("testing...").perform()
but I keep getting the error
selenium.common.exceptions.WebDriverException: Message: TypeError: rect is undefined

Code below tested with Chrome, Firefox and jupyterlab latest versions:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("http://localhost:8888")
token = "0107216930d05db8a7c36ad6a73573dd5349c3dd56fee852"
wait.until(EC.element_to_be_clickable((By.ID, "password_input"))).send_keys(token, Keys.ENTER)
# wait for "Python 3" Notebook menu or CodeMirror element if already launched.
wait.until(EC.presence_of_element_located(
(By.CSS_SELECTOR, "[title='Python 3'][data-category='Notebook'], .jp-Notebook .CodeMirror")))
# if "Python 3" Notebook menu found click to open new Notebook
if len(driver.find_elements_by_css_selector("[title='Python 3'][data-category='Notebook']")) > 0:
driver.find_element_by_css_selector("[title='Python 3'][data-category='Notebook']").click()
# wait for CodeMirror and click to focus
code_mirror = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, ".jp-Notebook .CodeMirror")))
code_mirror.click()
code_mirror.find_element_by_tag_name("textarea").send_keys("print('Hello World!')")
driver.find_element_by_css_selector("[data-icon='run']").click()
output = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".jp-OutputArea-output")))
print(output.text)
assert output.text.strip() == "Hello World!"
driver.quit()

To Open Jupyter-Notebook:
Either Open a command window and navigate to the repository folder or Open anaconda-command-prompt window and simply execute
jupyter notebook --NotebookApp.token='' --NotebookApp.password=''
After loading notebook using driver.get("http://localhost:8888"), Here is the most tricky part, that how to select a Dynamically changing object. Visit Reference: Dynamically Changing IDs.
By using find_element_by_xpath
By using find_element_by_css_selector
These both will give you the same selection-point but it would be better to use Xpath which is more convenient technique. You can move forward like this;
import time
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
driver = webdriver.Chrome('chromedriver.exe')
wait = WebDriverWait(driver, 20)
driver.maximize_window()
website_url = "http://localhost:8888/"
driver.get(website_url)
# Using Xpath
# I prefer using xpath, because it is simple to understand
# and if you want to dynamically enter data into fields, it would be an awesome approach
driver.find_element_by_xpath("//div[#id='new-buttons']").click()
if len(driver.find_elements_by_xpath("//div[#id='new-buttons']//li[#id='kernel-python3']")) > 0:
time.sleep(3)
driver.find_element_by_xpath("//div[#id='new-buttons']//li[#id='kernel-python3']").click()
driver.find_element_by_xpath('//div[#class="cell code_cell rendered selected"]').click()
# Using css_selector
#driver.find_element_by_css_selector("#notebook-container > div").click()
command = 'print("Hello World!")'
#a = driver.find_element_by_css_selector("#notebook-container > div > div.input > div.inner_cell >"
# "div.input_area > div > div:nth-child(1) > textarea")
time.sleep(3)
# To select Note-Book text-area and place command in it.
a = driver.find_element_by_xpath('//div[#class="input_area"]//textarea').click().send_keys(command)
# To run the Code in Selected Cell
time.sleep(3)
driver.find_element_by_xpath("//button[#title='Run']").click()
print("Test is done.")

How to locate the upload button related to the text Upload Image as per the HTML?

I'm using Selenium with Firefox (I've tried chrome but the success ratio on this one was significantly lower). I'm trying to locate an element (upload button) and the success ratio is nowhere near an acceptable percentage.
Now I know that I could try to use just selenium to the input but it wont locate it as well no matter what I do. So I used AutoIt for the uploading part.
frdriver.get('')
time.sleep(11)
try:
# el = WebDriverWait(frdriver, 15).until(EC.presence_of_element_located((By.XPATH, '/html/body/div[1]/header/div[2]/div[2]/div/div/form/i[2]')))
el = frdriver.find_element_by_xpath('/html/body/div[1]/header/div[2]/div[2]/div/div/form/i[2]')
try:
el.click()
print(el)
#ActionChains(frdriver).move_to_element(el).click().perform()
try:
#element = WebDriverWait(frdriver, 15).until(EC.presence_of_element_located((By.XPATH, '/html/body/div[1]/header/div[2]/div[2]/div/div/form/div[2]/div[3]/div[1]/div/div/div[1]')))
element = frdriver.find_element_by_xpath('/html/body/div[1]/header/div[2]/div[2]/div/div/form/div[2]/div[3]/div[1]/div/div/div[1]')
time.sleep(4)
try:
element.click()
#ActionChains(frdriver).move_to_element(element).click().perform()
autoit.run("/image.exe")
r = 100
except:
print('Firefox retries upload')
r += 1
except:
print('Firefox retries upload')
r += 1
except:
print('Firefox retries upload')
r += 1
except:
print('Firefox retries upload')
r += 1
As you can see I've tried many ways, each one had almost the same success.
I'm using a remote windows machine for the job and it really might be my idea, but when it isn't minimized it works a lot better.
Also there is something else, for some reason there are times that even if it hasn't successfully clicked a button it bypasses the try/except and proceeds.
Edit html of the page
<div class="ui-searchbar-keyword-panel ui-searchbar-keyword-hide ui-searchbar-static" style="z-index: 99; position: absolute; left: 0px; top: 35px;" data-widget-cid="widget-7">
<div class="ui-searchbar-img-search-box">
<div class="icbu-w-image-uploader-panel">
<div class="icbu-w-image-uploader-content">
<div class="upload-btn-wrapper">
<div class="upload-btn" data-spm-anchor-id="a2700.8293689.scGlobalHomeHeader.i1.2ce265aarlKA3D" style="z-index: 1;">Upload Image</div>
<div id="html5_1cjiqdvveht51pom1nbkqoiaop3_container" class="moxie-shim moxie-shim-html5" style="position: absolute; top: 14px; left: 224px; width: 109px; height: 28px; overflow: hidden; z-index: 0;">
<input id="html5_1cjiqdvveht51pom1nbkqoiaop3" type="file" style="font-size: 999px; opacity: 0; position: absolute; top: 0px; left: 0px; width: 100%; height: 100%;" multiple="" accept="image/jpeg,image/png,image/bmp">
</div>
</div>
<div class="response-text">Max 2MB per Image</div>
</div>
</div>
</div>
<div class="J-box-loadding-mask box-loadding-mask" data-role="loading" style="display: none;">
<div class="mh-mamo-loadding-box">
<i class="mh-mamo-icon-loadding"></i>
<span class="J-loading-text mh-mamo-loadding-text">Loading...</span>
</div>
</div>
</div>

You should use driver.implicitly_wait(X) where X is a number of seconds instead of reinventing the wheel with using time.sleep and retrying. That will increase selenium's tolerance towards the page loading and you will have better success in finding page elements.

To locate the upload button related to the text Upload Image you can use the following solution:
XPATH:
upload_button = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='upload-btn-wrapper']//div[#class='moxie-shim moxie-shim-html5' and starts-with(#id,'html5_')]/input[starts-with(#id,'html5_')]"))).click()
CSS_SELECTOR:
upload_button = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "div.upload-btn-wrapper div.moxie-shim.moxie-shim-html5[id^='html5_']>input[id^='html5_']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC