Develop Reference

Unable to obtain table info through python selenium

I am new bee on python selenium environment. I am trying to get the SQL version table from enter link description here
from selenium.webdriver.common.by import By
from selenium import webdriver
# define the website to scrape and path where the chromediver is located
website = "https://www.sqlserverversions.com"
driver = webdriver.Chrome(executable_path='/Users//Downloads/chromedriver/chromedriver.exe')
# define 'driver' variable
# open Google Chrome with chromedriver
driver.get(website)
matches = driver.find_elements(By.TAG_NAME, 'tr')
for match in matches:
b=match.find_elements(By.XPATH,"./td[1]")
print(b.text)
it says AttributeError: 'list' object has no attribute 'text'. Am i choosing the write syntax and right parameters to grab the data?
Below is the table which i am trying to get data.
enter image description here
Below are the parameters which i am trying to put in code.
enter image description here
Please advise what is required to modify in the code to obtain the data in table format.
Thanks,
Arun

If you need data only from first table:
from selenium.webdriver.common.by import By
from selenium import webdriver
website = "https://www.sqlserverversions.com"
driver = webdriver.Chrome(executable_path='/Users//Downloads/chromedriver/chromedriver.exe')
driver.get(website)
show_service_pack_versions = True
xpath_first_table_sql_rows = "(//table[#class='tbl'])[1]//tr/td/a[starts-with(text(),'SQL Server')]//ancestor::tr"
matches = driver.find_elements(By.XPATH, xpath_first_table_sql_rows)
for match in matches:
sql_server_a_element = match.find_element(By.XPATH, "./td/a[2]")
print(sql_server_a_element.text)
sql_server_rtm_version_a_element = match.find_element(By.XPATH, ".//td[#class='rtm']")
print('RTMs:')
print(sql_server_rtm_version_a_element.text)
if(show_service_pack_versions):
print('SPs:')
sql_server_sp_version_td_elements = match.find_elements(By.XPATH, ".//td[#class='sp']")
for td in sql_server_sp_version_td_elements:
print('---')
print(td.text)
print('----------------------------------')
if you set show_service_pack_versions = False then information regarding service packs will be skipped

There was a part of your code where you were calling b.text after getting the result of find_elements, which returns a list. You can only call b.text on a single WebElement (not a list of them). Here's the updated code:
from selenium.webdriver.common.by import By
from selenium import webdriver
website = "https://www.sqlserverversions.com"
driver = webdriver.Chrome(executable_path='/Users//Downloads/chromedriver/chromedriver.exe')
driver.get(website)
matches = driver.find_elements("css selector", "tr")
for match in matches[1:]:
items = match.find_elements("css selector", "td")
for item in items:
print(item.text)
That will print out A LOT of rows, unless you limit the loop.

If you just need text it's simpler to do it on the browser side:
data = driver.execute_script("""
return [...document.querySelectorAll('tr')].map(tr => [...tr.querySelectorAll('td')].map(td => td.innerText))
""")

Unable to print text from items with 'find_elements_by_xpath'

I'm starting web scraping and followed tutorials. Yet in this code I get a "nameError: name 'avail' is not defined". I guess it's really easy, but how could I fix this ? (Error is probably in the for loop at line 15 in avail = i.text())
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome('/Users/victorfichtner/Downloads/Chromedriver')
driver.get('https://www.myntra.com/smart-watches/boat/boat-unisex-black-storm-m-
smart-watch/13471916/buy')
a = driver.find_elements_by_xpath("//*[#class='pdp-add-to-bag pdp-button pdp-flex
pdp-center']")
for i in a :
avail = i.text()
driver.quit()
print(avail)

Things to be noted.
find_elements return a list, where as find_element return a single web element.
Xpath is brittle.
Use explicit waits for dynamic loading.
It is .text in Python not .text()
Sample code :
driver = webdriver.Chrome('/Users/victorfichtner/Downloads/Chromedriver')
driver.maximize_window()
driver.implicitly_wait(50)
driver.get('https://www.myntra.com/smart-watches/boat/boat-unisex-black-storm-m- smart-watch/13471916/buy')
a = driver.find_elements_by_xpath("//*[contains(#class,'pdp-add-to-bag pdp-button pdp-flex')]")
avail = ""
for i in a :
avail = i.text
driver.quit()
print(avail)
Output :
ADD TO BAG

Selenium (Python) - getting the value inside a button

So I'm trying to get the value inside a button using selenium.
My code is:
getphone = profile.find_element_by_class_name('app-emp-phone-txt').click().find_element_by_tag_name("p")
But I'm getting this error:
AttributeError: 'NoneType' object has no attribute 'find_element_by_tag_name'
What should I write in my code ?
thanks a lot :)

You're trying to call find_element from method (click) call which returns None. Try to update your code:
getphone = profile.find_element_by_class_name('app-emp-phone-txt')
getphone.click()
getphone.find_element_by_tag_name("p")
Update
from selenium import webdriver
driver = webdriver.Chrome()
driver.implicitly_wait(10)
driver.get('https://www.mariages.net/domaine-mariage/la-tour-des-plantes--e129089')
getphone = driver.find_element_by_class_name('app-emp-phone-txt')
getphone.click()
number = driver.find_element_by_css_selector(".app-dropdown-show-phone>.storefrontDrop__tag").text
print(number)

How to fix 'in find_element_by_id return self.find_element(by=By.ID, value=id_)' error in Python Selenium

I'm developing a webscraper via Selenium and Python. When I run it from the command prompt by entering the code per line, it runs fine. But when I run it via the python [program name] method, it returns this error:
The error message
Is my selenium version incompatible?
Thanks :)
Here's my code:
`from selenium import webdriver
from selenium.webdriver.common.keys import Keys
browser = webdriver.Chrome('C://Users/user1/Portable Python 3.7.0 x64/App/Python/Lib/site-packages/chromedriver')
browser.get('https://tweeterid.com/')
bar = browser.find_element_by_name('twitter')
bar.send_keys('865102744809381888')
bar.send_keys(Keys.ENTER)
out = browser.find_element_by_id('0')
temp = out.text
data = temp.split('=>') #split string into a list
list = []
list.append(data[1])
browser.refresh()`

The problem is when you execute the code using python it does not wait for the enter key and finds the find_element_by_id('0')
Add time.sleep(5) after bar.send_keys(Keys.ENTER)
I hope this will solve your problem
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
browser = webdriver.Chrome('chromedriver.exe')
browser.get('https://tweeterid.com/')
bar = browser.find_element_by_name('twitter')
bar.send_keys('865102744809381888')
bar.send_keys(Keys.ENTER)
time.sleep(5)
out = browser.find_element_by_id('0')
temp = out.text
data = temp.split('=>') #split string into a list
list = []
list.append(data[1])

Having trouble in getting data using css selector/xpath in selenium

I'm trying to extract data from the link below using selenium via python:
www.oanda.com
But I'm getting an error that, "Unable to Locate an Element". In browser console i tried using this Css selector:
document.querySelector('div.position.short-position.style-scope.position-ratios-app')
This querySelector returns me the data for short percentage of 1st row in the browser console(for this test), but when i used this selector in the python script below it gives me an error that, "Unable to Locate element" or sometimes empty sctring.
Please suggest me solution if there's any.Will be grateful, thanks :)
# All Imports
import time
from selenium import webdriver
#will return driver
def getDriver():
driver = webdriver.Chrome()
time.sleep(3)
return driver
def getshortPercentages(driver):
shortPercentages = []
shortList = driver.find_elements_by_css_selector('div.position.short-position.style-scope.position-ratios-app')
for elem in shortList:
shortPercentages.append(elem.text)
return shortPercentages
def getData(url):
driver = getDriver()
driver.get(url)
time.sleep(5)
# pagesource = driver.page_source
# print("Page Source: ", pagesource)
shortList = getshortPercentages(driver)
print("Returned source from selector: ", shortList)
if __name__ == '__main__':
url = "https://www.oanda.com/forex-trading/analysis/open-position-ratios"
getData(url)

Required data is located inside an iframe, so you need to switch to iframe before handling elements:
driver.switch_to.frame(driver.find_element_by_class_name('position-ratios-iframe'))
Also note that data inside iframe is dynamic, so make sure that you're using Implicit/Explicit wait (using time.sleep(5) IMHO is not the best solution)