I am attempting to download the data from the following site programmatically with Python: https://health.wyo.gov/publichealth/infectious-disease-epidemiology-unit/disease/novel-coronavirus/covid-19-testing-data/
At the bottom right of the Tableau view there are 3 buttons: Share, Download, and Full Screen. After clicking Download, you are then taken to another pop up. I am then wanting to select Crosstab that then takes you to another pop up where I want to select Positivity and finally, Download which provides a csv.
I have essentially managed to navigate through some of the iframes but am a little lost on where to click since the buttons do not include id/link.
Below is some code from one approach:
from selenium import webdriver
url = 'https://public.tableau.com/profile/melissa.taylor#!/vizhome/WyomingCOVID-19TestingDataDashboard/Dashboard1'
driver = webdriver.Chrome()
driver.get(url)
#elem = driver.switch_to.frame(driver.find_element_by_xpath('//iframe[contains(text(), "googletagmanager"]'))
try:
time.sleep(4)
iframe = driver.find_elements_by_tag_name('iframe')[0]
driver.switch_to.default_content()
driver.switch_to.frame(iframe)
driver.find_elements_by_tag_name('iframe')
#driver.find_element_by_id('download-ToolbarButton').click()
print(driver.page_source)
finally:
driver.quit()
Below is the HTML that displays the 3 buttons from the Tableau page.
<div class="tab-nonVizItems tab-fill-right hideLabels"><div class="tabToolbarButton tab-widget undo disabled" role="button" data-tb-test-id="undo-ToolbarButton" id="undo-ToolbarButton" aria-disabled="true" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Undo"><span class="tabToolbarButtonImg tab-icon-undo"></span><span class="tabToolbarButtonText">Undo</span></div><div class="tabToolbarButton tab-widget redo disabled" role="button" data-tb-test-id="redo-ToolbarButton" id="redo-ToolbarButton" aria-disabled="true" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Redo"><span class="tabToolbarButtonImg tab-icon-redo"></span><span class="tabToolbarButtonText">Redo</span></div><div class="tabToolbarButton tab-widget revert disabled" role="button" data-tb-test-id="revert-ToolbarButton" id="revert-ToolbarButton" aria-disabled="true" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Reset"><span class="tabToolbarButtonImg tab-icon-revert"></span><span class="tabToolbarButtonText">Reset</span></div><div class="tabToolbarButton tab-widget share" role="button" data-tb-test-id="share-ToolbarButton" id="share-ToolbarButton" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Share"><span class="tabToolbarButtonImg tab-icon-share"></span><span class="tabToolbarButtonText">Share</span></div><div class="tabToolbarButton tab-widget download" role="button" data-tb-test-id="download-ToolbarButton" id="download-ToolbarButton" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Download"><span class="tabToolbarButtonImg tab-icon-download"></span><span class="tabToolbarButtonText">Download</span></div><div class="tabToolbarButton tab-widget enterFullscreen" role="button" data-tb-test-id="toggle-fullscreen-ToolbarButton" id="toggle-fullscreen-ToolbarButton" tabindex="-1" style="user-select: none; -webkit-tap-highlight-color: transparent;" title="Full Screen"><span class="tabToolbarButtonImg tab-icon-enterFullscreen"></span><span class="tabToolbarButtonText">Full Screen</span></div></div>
<div></div></div><div class="tab-ReactView tab-toolbar-dialoghost"></div></div></div>
Thanks for your help!!
Try the below code:
driver.get('https://public.tableau.com/profile/melissa.taylor#!/vizhome/WyomingCOVID-19TestingDataDashboard/Dashboard1')
wait = WebDriverWait(driver, 20)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.CSS_SELECTOR, "iframe[title='Data Visualization']")))
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR, ".tab-icon-download"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Crosstab']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//span[text()='positivity']"))).click()
wait.until(EC.element_to_be_clickable((By.XPATH, "//button[text()='Download']"))).click()
Following import:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Related
I am using Selenium with Python. I want to automate to click the one of menu in the dropdown menu written as below. Let's say "openExportDialog()" xpath is //*[#id="globalMenuItems"]/li[1]/a
Could anyone please shed some light?
<div id="nmapGlobalMenu" class="col-md-12 disable-user-select dropdown pr-0 show">
<button class="imgicon n-ui-icon n-ui-ic-operation myBasic_icon imgicon" data-toggle="dropdown" aria-haspopup="true" aria-expanded="true"></button><div id="globalMenuItems" class="dropdown-menu dropdown-menu-right show" x-placement="bottom-end" style="position: absolute; transform: translate3d(803px, 33px, 0px); top: 0px; left: 0px; will-change: transform;">
</li><li class=""><a class="dropdown-item" onclick="openImportDialog()"><span class="vakata-contextmenu"> Import Node</span></a>
</li><li class=""><a class="dropdown-item" onclick="openExportDialog()"><span class="vakata-contextmenu"> Export Node</span></a>
</li><li class=""><a class="dropdown-item" onclick="discoverNetwork()"><span class="vakata-contextmenu">Discover Network</span></a>
</li><li class=""><a class="dropdown-item" onclick="openEmailSettingsView()"><span class="vakata-contextmenu"> Email Notification Settings</span></a>
</li><li class=""><a class="dropdown-item" onclick="ICMPsettingsView()"><span class="vakata-contextmenu"> ICMP Health Check Settings</span></a>
</li><li class=""><a class="dropdown-item" onclick="customTypeManagement()"><span class="vakata-contextmenu"> Node Type Management</span></a></li></div>
</div>
I hope the question is clear.
Thank you
once the dialog box is open, you can use this //a[#onclick='openExportDialog()'] xpath to click on Export Node option.
Code :
wait = WebDriverWait(driver, 50)
wait.until(EC.element_to_be_clickable((By.XPATH, "//a[#onclick='openExportDialog()']"))).click()
Imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
First click on the Button and then try to find the a tag. Like below:
driver find_element_by_id("globalMenuItems").click()
driver.find _element_by_xpath("//a[contains(text(),'Export Node')]").click()
I am trying to make an automated checkout system but I can't seem to send keys in fields of this payment box. Even when I try implicit wait and wait for it to load in it still doesn't work.
checkout = driver.find_element_by_id("number")
checkout.send_keys("9848432")
<input required="" autocomplete="cc-number" id="number" name="number" type="tel" aria-describedby="error-for-number tooltip-for-number" data-current-field="number" placeholder="Card number" style="font-family: Roboto, sans-serif; padding: 0.94em 0.8em; transition: padding 0.2s ease-out 0s;">
Here's a link!
Here is the URL you need to fill in the info until you get to the payment page, which is where I am having the issues
It might be in a nested iframe. A similar checkout due to not proper reproducible code.
<iframe src="https://fs.global-e.com/Checkout/v2/598d87e9-cd85-4f61-84df-81e345d62f05?gaSesID=526107767.603750114.583&gaMerchantClientInfo=undefined#undefined&chkcuid=b51f4449-13e2-4883-92ba-2641bb06ec51&isNECAllowed=true&vph=663&ift=87" class="Intrnl_CO_Container" id="Intrnl_CO_Container" name="Intrnl_CO_Container" allowtransparency="true" width="100%" scrolling="no" marginheight="0" marginwidth="0" frameborder="0" height="1000px" style="height: 3101px;">
<iframe id="secureWindow" class="clear-fix clearfix" allowtransparency="true" style="width: 100%; min-height: 160px; height: 165px;" src="https://securev2.global-e.com/payments/CreditCardForm/598d87e9-cd85-4f61-84df-81e345d62f05/11">
<input autocomplete="off" class="form-control input-validation-error" data-type="unknown" data-type-id="1" data-val="true" data-val-luhn="Card number not valid" data-val-luhn-allowempty="False" data-val-luhn-allowspaces="False" data-val-required="Card number" id="cardNum" name="PaymentData.cardNum" pattern="[0-9]{13,16}" placeholder="Card number" type="tel" value="">
</iframe>
</iframe>
Which was solved with:
wait = WebDriverWait(driver, 10)
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, 'Intrnl_CO_Container')))
wait.until(EC.frame_to_be_available_and_switch_to_it((By.ID, 'secureWindow')))
checkout = wait.until(EC.presence_of_element_located((By.ID, "cardNum")))
checkout.send_keys("9848432")
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
I have this HTML element code which I am currently struggling to figure out to use it for clicking on the tab that says Problem. As the "Problem" doesnt have a unique classname or element ID, I am unable to figure how to send a Click().
I have tried to check if z-index can be used as index(assumed) and used below line of code
browser.switch_to_frame(a[3])
but it seems I am wrong.
HTML code as below
<div class="TabsViewPort" style="position: relative; overflow: hidden; width: 896px; height: 22px; float: left;">
<div style="overflow: visible; float: left; width: 897px; top: 0px; left: 0px;">
<dl class="OuterOuterTab">
<dd class="OuterTab" artabid="955000038" arwindowid="0" style="top: 1px; z-index: 1; left: 0px; visibility: inherit; display: block;"><span class="TabLeftRounded"> </span>
<span class="Tab">My Profile
</span>
<span class="TabRight"> </span>
</dd>
<dd class="OuterTabSelected" artabid="600000203" arwindowid="0" style="top: 1px; z-index: 3; left: 63px; visibility: inherit; display: block;"><span class="TabLeft"> </span>
<span class="Tab">Approval
</span>
<span class="TabRight"> </span>
</dd>
<dd class="OuterTab" artabid="536870915" arwindowid="0" style="top: 1px; z-index: 1; left: 409px; visibility: inherit; display: block;"><span class="TabLeft"> </span>
<span class="Tab">Problem
</span>
<span class="TabRight"> </span>
</dd>
</dl>
</div>
</div>
The element with text as Problem is a JavaScript enabled element so to click() on the element you have to induce WebDriverWait for the element to be clickable and you can use either of the following solutions:
Using XPATH A:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[#class='TabsViewPort']//dl[#class='OuterOuterTab']//dd[#class='OuterTab']//a[#class='btn f1' and text()='Problem']"))).click()
Using XPATH B (shortened):
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[#class='btn f1' and text()='Problem']"))).click()
Note : You have to add the following imports :
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
If the element present inside an iframe then you need to switch to iframe first to access the element.
You can Use following method to frame_to_be_available_and_switch_to_it()
By locator ID
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.ID,"id of the iframe")))
OR
By locator NAME
WebDriverWait(driver,10).until(EC.frame_to_be_available_and_switch_to_it((By.NAME,"name of the iframe")))
Once You have switched to iframe you can access the element using following xpath
To click on the element Induce WebDriverWait and element_to_be_clickable()
WebDriverWait(driver,10).until(EC.element_to_be_clickable((By.XPATH,"//span[#class='Tab']//a[text()='Problem']"))).click()
You need to import following to execute above code.
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Hope this will help.
please find below xpath to click on third TAB
(//span[#class="Tab"])[3]/a
I would like to test a patched implementation of juptyerlab. I was hoping to use selenium to execute "hello world" in a code cell. So far I can log in and create a new notebook:
from selenium import webdriver
driver = webdriver.Firefox()
# assume jupyterlab is running and serving on localhost at port 8888
driver.get("http://localhost:8888")
elem = driver.find_element_by_id("password_input")
password = ""
elem.send_keys(password)
elem = driver.find_element_by_id("login_submit")
elem.click()
elem = driver.find_element_by_css_selector(".jp-Launcher-cwd+ .jp-Launcher-section .jp-LauncherCard")
elem.click()
This creates a new notebook, but now I'm stuck at the point of entering some code in a cell and running it. If I view the page source I don't see any html elements for the cells. But if I enter print("test") in a cell, then driver.page_source contains this (it's pretty nested in other stuff I've omitted too):
<div class="CodeMirror cm-s-jupyter CodeMirror-wrap jp-mod-readOnly">
<div style="overflow: hidden; position: relative; width: 3px; height: 0px; top: 0px; left: 0px;">
<textarea
style="position: absolute; bottom: -1em; padding: 0px; width: 1px; height: 1em; outline: currentcolor none medium;"
autocorrect="off" autocapitalize="off"
spellcheck="false" tabindex="0"
wrap="off"></textarea></div>
<div class="CodeMirror-vscrollbar" tabindex="-1"
cm-not-content="true"
style="display: block; bottom: 0px;">
<div style="min-width: 1px; height: 33px;"></div>
</div>
<div class="CodeMirror-hscrollbar" tabindex="-1"
cm-not-content="true">
<div style="height: 100%; min-height: 1px; width: 0px;"></div>
</div>
<div class="CodeMirror-scrollbar-filler"
cm-not-content="true"></div>
<div class="CodeMirror-gutter-filler"
cm-not-content="true"></div>
<div class="CodeMirror-scroll" tabindex="-1" draggable="true">
<div class="CodeMirror-sizer"
style="margin-left: 0px; padding-right: 0px; padding-bottom: 0px;">
<div style="position: relative;">
<div class="CodeMirror-lines" role="presentation">
<div style="position: relative; outline: currentcolor none medium;"
role="presentation">
<div class="CodeMirror-measure">
<pre><span>xxxxxxxxxx</span></pre>
</div>
<div class="CodeMirror-measure">
<pre class="CodeMirror-line"
role="presentation"><span
role="presentation"><span
class="cm-builtin">print</span>(<span
class="cm-string">"test"</span>)</span></pre>
</div>
<div style="position: relative; z-index: 1;"></div>
<div class="CodeMirror-cursors"></div>
<div class="CodeMirror-code"
role="presentation"></div>
</div>
</div>
</div>
</div>
<div style="position: absolute; height: 30px; width: 1px; border-bottom: 0px solid transparent;"></div>
<div class="CodeMirror-gutters"
style="display: none;"></div>
</div>
</div>
I can see where the text for print("text") is (i.e. the deepest nested elements in the above html snippet), but I can't figure out which element here I would be able to send text to or send keys to.
I came across robotframework-jupyterlibrary and it has some clues such as this and this . From those links I see
${JLAB CSS ACTIVE INPUT} ${JLAB CSS ACTIVE CELL} .CodeMirror
and
Add and Run JupyterLab Code Cell
[Arguments] ${code}=print("hello world")
[Documentation] Add a ``code`` cell to the currently active notebook and run it.
Click Element css:${JLAB CSS NB TOOLBAR} ${JLAB CSS ICON ADD}
Sleep 0.1s
${cell} = Get WebElement css:${JLAB CSS ACTIVE INPUT}
Click Element ${cell}
Set CodeMirror Value ${JLAB CSS ACTIVE INPUT} ${code}
Run Current JupyterLab Code Cell
Click Element ${cell}
which makes me think if I select the .CodeMirror element, then I just need to figure out what Get WebElement does in that weird language and how to do it in selenium.
Any ideas?
I've also tried (based on https://stackoverflow.com/a/48723135/1011724 and https://stackoverflow.com/a/50279295/1011724):
from selenium.webdriver.common.action_chains import ActionChains
actions = action_chains.ActionChains(driver)
textarea = driver.find_elements_by_css_selector('.CodeMirror textarea')[0] # tried for [0], [1] ,[2] and [3] which is all of them.
actions.move_to_element(textarea).click().send_keys("testing...").perform()
but I keep getting the error
selenium.common.exceptions.WebDriverException: Message: TypeError: rect is undefined
Code below tested with Chrome, Firefox and jupyterlab latest versions:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
wait = WebDriverWait(driver, 10)
driver.get("http://localhost:8888")
token = "0107216930d05db8a7c36ad6a73573dd5349c3dd56fee852"
wait.until(EC.element_to_be_clickable((By.ID, "password_input"))).send_keys(token, Keys.ENTER)
# wait for "Python 3" Notebook menu or CodeMirror element if already launched.
wait.until(EC.presence_of_element_located(
(By.CSS_SELECTOR, "[title='Python 3'][data-category='Notebook'], .jp-Notebook .CodeMirror")))
# if "Python 3" Notebook menu found click to open new Notebook
if len(driver.find_elements_by_css_selector("[title='Python 3'][data-category='Notebook']")) > 0:
driver.find_element_by_css_selector("[title='Python 3'][data-category='Notebook']").click()
# wait for CodeMirror and click to focus
code_mirror = wait.until(EC.presence_of_element_located((By.CSS_SELECTOR, ".jp-Notebook .CodeMirror")))
code_mirror.click()
code_mirror.find_element_by_tag_name("textarea").send_keys("print('Hello World!')")
driver.find_element_by_css_selector("[data-icon='run']").click()
output = wait.until(EC.visibility_of_element_located((By.CSS_SELECTOR, ".jp-OutputArea-output")))
print(output.text)
assert output.text.strip() == "Hello World!"
driver.quit()
To Open Jupyter-Notebook:
Either Open a command window and navigate to the repository folder or Open anaconda-command-prompt window and simply execute
jupyter notebook --NotebookApp.token='' --NotebookApp.password=''
After loading notebook using driver.get("http://localhost:8888"), Here is the most tricky part, that how to select a Dynamically changing object. Visit Reference: Dynamically Changing IDs.
By using find_element_by_xpath
By using find_element_by_css_selector
These both will give you the same selection-point but it would be better to use Xpath which is more convenient technique. You can move forward like this;
import time
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
driver = webdriver.Chrome('chromedriver.exe')
wait = WebDriverWait(driver, 20)
driver.maximize_window()
website_url = "http://localhost:8888/"
driver.get(website_url)
# Using Xpath
# I prefer using xpath, because it is simple to understand
# and if you want to dynamically enter data into fields, it would be an awesome approach
driver.find_element_by_xpath("//div[#id='new-buttons']").click()
if len(driver.find_elements_by_xpath("//div[#id='new-buttons']//li[#id='kernel-python3']")) > 0:
time.sleep(3)
driver.find_element_by_xpath("//div[#id='new-buttons']//li[#id='kernel-python3']").click()
driver.find_element_by_xpath('//div[#class="cell code_cell rendered selected"]').click()
# Using css_selector
#driver.find_element_by_css_selector("#notebook-container > div").click()
command = 'print("Hello World!")'
#a = driver.find_element_by_css_selector("#notebook-container > div > div.input > div.inner_cell >"
# "div.input_area > div > div:nth-child(1) > textarea")
time.sleep(3)
# To select Note-Book text-area and place command in it.
a = driver.find_element_by_xpath('//div[#class="input_area"]//textarea').click().send_keys(command)
# To run the Code in Selected Cell
time.sleep(3)
driver.find_element_by_xpath("//button[#title='Run']").click()
print("Test is done.")
I am using Selenium with Python to download certain files from a website. I am trying to access a option called "View all products >>". From there, you could select the kind of file(csv, xlsx) and then you should download it easily. My problem is that I cannot access that "View all products" area. I've tried in so many ways. I'll attach my code and a screenshot with the structure of website. I cannot post entire HTML because it is a website restrictive only to it's paid users.
.
The five links:
1 and 2. (Here "View All Products" appears twice):
<a class="yui3-c-reportdashboardwidget-reportLink" href="./embedded.html?showGDLogo=false#project=/gdc/projects/e05jwlnny6rlxyt5ib9r6479279crrq8&dashboard=/gdc/md/e05jwlnny6rlxyt5ib9r6479279crrq8/obj/4817&tab=8473392139f7&s=/gdc/projects/e05jwlnny6rlxyt5ib9r6479279crrq8|analysisPage|head|/gdc/md/e05jwlnny6rlxyt5ib9r6479279crrq8/obj/3630" title="Headline - View All Products Link" target="_self">Headline - View All Products Link</a>
3 and 4. (Here "View All Products" appears twice):
<span class="yui3-c-reportdashboardwidget-reportLabel" title="Headline - View All Products Link">Headline - View All Products Link</span>
5:
<div class="number" style="font-size: 16px; color: rgb(0, 61, 76);" id="yui_3_14_1_1_1540109592048_72886"> View all products >></div>
The link that I want to handle is number "5", because I think that is the one I need to click(), so that I can download the report afterwards.
My code for this part:
View All Products Button
#product_button = driver.find_elements_by_xpath("//div[#class='c-oneNumberReport yui3-widget yui3-c-onenumberreport yui3-c-onenumberreport-content yui3-widget-content-expanded drillable']")[-1]
#product_button = driver.find_element_by_xpath(("//div[text()='View all products >>']"))
product_button = driver.find_elements_by_xpath("//a[#class='ember-view reportInfoPanelHandle point-to-top']")[-3]
product_button.click()
#product_button.send_keys(Keys.ENTER)
#####actions = ActionChains(driver)
#actions.move_to_element(product_button).send_keys(Keys.ENTER)
###########actions.move_to_element(product_button)
###########actions.click()
#actions.sendKeys(Keys.Return);
#actions.build().perform()
Update::::
Here is the HTML for "Download As..":
<span class="button-text"><script id="metamorph-39-start" type="text/x-placeholder"></script>Download as...<script id="metamorph-39-end" type="text/x-placeholder"></script></span>
HTML for "CSV ( raw data)":
<ul id="ember2849" class="ember-view reportExportMenu gdc-menu-simple" style="position: absolute; top: 106px; left: 15px; z-index: 3005;"><li id="ember2850" class="ember-view reportExportMenuItem">
<a data-ember-action="17"><script id="metamorph-47-start" type="text/x-placeholder"></script>PDF (Portrait)<script id="metamorph-47-end" type="text/x-placeholder"></script></a>
</li><li id="ember2851" class="ember-view reportExportMenuItem">
<a data-ember-action="18"><script id="metamorph-48-start" type="text/x-placeholder"></script>PDF (Landscape)<script id="metamorph-48-end" type="text/x-placeholder"></script></a>
</li><li id="ember2852" class="ember-view reportExportMenuItem">
<a data-ember-action="19"><script id="metamorph-49-start" type="text/x-placeholder"></script>XLSX...<script id="metamorph-49-end" type="text/x-placeholder"></script></a>
</li><li id="ember2853" class="ember-view reportExportMenuItem">
<a data-ember-action="20"><script id="metamorph-50-start" type="text/x-placeholder"></script>CSV (formatted)<script id="metamorph-50-end" type="text/x-placeholder"></script></a>
</li><li id="ember2854" class="ember-view reportExportMenuItem">
<a data-ember-action="21"><script id="metamorph-51-start" type="text/x-placeholder"></script>CSV (raw data)<script id="metamorph-51-end" type="text/x-placeholder"></script></a>
</li></ul>
Try to use below code to click required element:
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait as wait
link = wait(driver, 10).until(EC.element_to_be_clickable((By.XPATH, '//div[starts-with(normalize-space(), "View all products")]')))
driver.execute_script("arguments[0].scrollIntoView();", link)
link.click()