Develop Reference

How to get the href link from nested HTML tags using python and beautifulsoup?

I have used to below code to extract the href content
soup = bs(html, 'html.parser')
pagesize_content = soup.find_all('div',attrs={'class':"pull-right"})
print(pagesize_content)
When I the class "Pull-right"
output as below
[<div class="col-md-4 pull-right">
<button class="btn btn-default closebutton" data-dismiss="modal" type="button">Close</button>
<button class="btn btn-success" id="Name Properties" type="submit">
Submit
<em class="fa fa-check"></em>
</button>
</div>, <div class="pull-right">
<ul class="pagination">
<li class="disabled">
<a class="disabled" href="javascript:void(0)">
First
<em class="fa fa-angle-double-left"></em>
</a>
</li>
<li class="active">
1
</li>
<li class="">
2
</li>
<li class="">
3
</li>
<li class="">
4
</li>
<li class="">
<a href="/Test/country?page=4">
Last
<em class="fa fa-angle-double-right"></em>
</a>
</li>
</ul>
</div>]
From that how to extract the href link.
When I try with
print(pagesize_content.ul.li.a['href'])
error displaying
pagesize_content = soup.find_all('a',attrs={'class':"fa fa-angle-double-right"},href=True)
print(pagesize_content)
empty output

How to pick up specific data with Python and Selenium

I'm Hiro from Japan.
I just started studying Python with Selenium by myself.
I am happy if somebody help me solve following problem.
Number of "data" which is enclosed in "p" tags is always changed so don't know how many data will be shown every time.
Data4 is always appear but order is changed.
Also these data are assigned same class.
For example, In case sample A Data4 is 4th, but sample B is 2nd.
Please help me how pick up "Data4".
Thank you for your help in advance.
(sample A)
----------------------------------------
<div class="elist" id="test">
<ul class="ijkl">
<li class="elRow">
<div class="elRowTitle">
<p>Code1</p>
</div>
<div class="elRowData">
<p>Data1</p>
</div>
</li>
<li class="elRow">
<div class="elRowTitle">
<p>Code2</p>
</div>
<div class="elRowData">
<p>Data2</p>
</div>
</li>
<li class="elRow">
<div class="elRowTitle">
<p>Code3</p>
</div>
<div class="elRowData">
<p>Data3</p>
</div>
</li>
<li class="elRow">
<div class="elRowTitle">
<p>Code4</p>
</div>
<div class="elRowData">
<p>Data4</p>
</div>
</li>
</ul>
</div>
----------------------------------------
(sample B)
----------------------------------------
<div class="elist" id="test">
<ul class="ijkl">
<li class="elRow">
<div class="elRowTitle">
<p>Code1</p>
</div>
<div class="elRowData">
<p>Data1</p>
</div>
</li>
<li class="elRow">
<div class="elRowTitle">
<p>Code4</p>
</div>
<div class="elRowData">
<p>Data4</p>
</div>
</li>
</ul>
</div>
----------------------------------------
I wrote the following script using "find_elements_by_xxxxx" method for both sample A and B but did not work.
from selenium import webdriver
browser = webdriver.Chrome('chromedriver.exe')
elem_ItemCodes = driver.find_elements_by_tag_name('div')
elem_ItemCodes2 = elem_ItemCodes.find_elements_by_class_name('elRowData')
elem_ItemCode = elem_ItemCodes2[3].text
print(elem_ItemCode)

You can use parent as reference,
The xpath equivalent will be :
//div[#id="test"]//div[#class="elRowData"][4]/p
your approach is correct, but make few changes like:
from selenium import webdriver
browser = webdriver.Chrome('chromedriver.exe')
elem_ItemCodes = driver.find_elements_by_id('test')
elem_ItemCodes2 = elem_ItemCodes.find_elements_by_class_name('elRowData')
elem_ItemCode = elem_ItemCodes2[4].find_elemen_by_tag_name("p").text
print(elem_ItemCode)
The text is in P tag and the index in xpath starts from 1 and not 0

How to get child element data using python and selenium?

What I'm trying to do is get the data from the child element in the below code:
<div class="ListContainer">
<ul class="uiList">
...
</ul>
<div class="expandedList">
<ul>
<div id="1012450"><img /></div>
<div id="1012451"><img /></div>
<div id="1012452"><img /></div>
<div id="1012453"><img /></div>
</ul>
</div>
<div class="expandedList">
<ul>
<div id="1012454"><img /></div>
<div id="1012455"><img /></div>
<div id="1012456"><img /></div>
<div id="1012457"><img /></div>
</ul>
</div>
....
....
</div>
I want to get the id of each div inside the expandedList class. I tried using xpath but that not capturing all the expandedList.

div_ids = []
for div_element in driver.find_elements_by_css_selector('div.expandedList div[id]'):
div_ids.append(div_element.get_attribute('id'))

Python-Behave Selenium - Get all child anchor text from parent element

I have the following header that I'm trying to get the multiple text from. I'm only able to get the text of menus Mobile App Guide, Quick Links, SDKs, Protocols & Plugins, and FAQ. But I'm not able to get the sub menu text.
I'm using elem.find_elements_by_css_selector("*") and elem.find_elements_by_xpath(".//*") which creates a list of all child elements under header tag.
I think its because I'm only iterating through to closest child of the parent element where if len(child.text) > 0. Anyone know how I can iterate through all the child elements (including sub menu) of the header tag?
from behave import given, when, then
from selenium.webdriver.common.keys import Keys
# #given('the user is on documentation home page')
# def on_home_page(context):
# context.driver.get("URL removed")
#when('the header is displayed')
def header_display(context):
assert context.driver.find_element_by_xpath('/html/body/header')
#then('the header should have {menu}')
def header_menu(context, menu):
elem = context.driver.find_element_by_tag_name("header")
all_children_by_css = elem.find_elements_by_css_selector("*")
all_children_by_xpath = elem.find_elements_by_xpath(".//*")
children = []
for child in all_children_by_xpath:
# print(len(child.text))
if len(child.text) > 0:
# print(child.text)
if child.text not in children:
children.append(child.text)
print(children)
if menu in children:
assert True
else:
assert False
Source code of page header
<header class="header docs-header">
<nav class="top-bar" data-topbar role="navigation">
<ul class="title-area">
<li class="name">
<h1>
<a title="LaunchKey Home Page" class="home-link" href="https://www.iovation.com/launchkey">
<img alt="LaunchKey Logo" class="header-logo" src="_static/images/logo-name.svg">
</a>
</h1>
</li>
<li class="toggle-topbar menu-icon">
<a href="#" title="menu-toggle">
<span> </span>
</a>
</li>
</ul>
<section class="top-bar-section">
<ul class="left">
<li class="has-dropdown">
<a href="#">Mobile App Guide
<i class="zmdi zmdi-chevron-down margin-left-qtr"></i>
</a>
<ul class="dropdown">
<li class="blue-hover">
Android
</li>
<li class="blue-hover">
iOS
</li>
</ul>
</li>
<li class="has-dropdown">
<a href="#">Quick Links
<i class="zmdi zmdi-chevron-down margin-left-qtr"></i>
</a>
<ul class="dropdown">
<li class="blue-hover">
Getting Started Guide
</li>
<li class="blue-hover">
API Reference
</li>
<li class="blue-hover">
Encryption
</li>
<li class="blue-hover">
Glossary of Terms
</li>
<li class="blue-hover">
Hackers
</li>
<li class="blue-hover">
Designers
</li>
<li class="blue-hover">
Support
</li>
</ul>
</li>
<li class="has-dropdown">
<a href="#">SDKs
<i class="zmdi zmdi-chevron-down margin-left-qtr"></i>
</a>
<ul class="dropdown">
<li class="blue-hover">
Desktop/Web MFA
</li>
<li class="blue-hover">
Native Mobile MFA
</li>
<li class="blue-hover">
Authenticator
</li>
</ul>
</li>
<li class="has-dropdown">
<a href="#">Protocols & Plugins
<i class="zmdi zmdi-chevron-down margin-left-qtr"></i>
</a>
<ul class="dropdown">
<li class="blue-hover">
WordPress
</li>
<li class="blue-hover">
Drupal
</li>
<li class="blue-hover">
OAuth
</li>
<li class="blue-hover">
OpenID
</li>
<li class="blue-hover">
SSH PAM
</li>
</ul>
</li>
<li class="blue-hover">
FAQ
</li>
</ul>
<ul class="right">
<li>
<div role="search">
<form id="rtd-search-form" class="wy-form" action="search.html" method="get">
<input type="text" name="q" placeholder="Search docs" tabindex="1" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</li>
</ul>
</section>
</nav>
</header>

Selenium Web Driver | Find object | Python

In advance I apologize for my English, because I write with an interpreter.
There is a task to click on one of these accounts, suppose AAccount, using Selenium Web Driver (Python, Chrome).
The problem is that it's impossible to do this on XPath and CSS, because the elements are the same.
Here is part of the source code for the page. I also attached a screenshot of the page.
</style>
</head>
<body>
<div class="wrapper">
<div class="gb_ag" ng-non-bindable="" id="gb"><div class="gb_hf gb_fg"><div class="gb_ib gb_fg gb_R gb_eg"><div class="gb_7d gb_R gb_fg gb_7f"><div class="gb_Q gb_R gb_S gb_fg"><span class="gb_P gb_R">TeAOK</span></div></div><div class="gb_sc gb_fg gb_R"><div class="gb_ga" id="gbsfw" style="background-color:#eee;min-width:376px"></div><div class="gb_Ac gb_Wc gb_R gb_Bc"><div class="gb_Dc"><a class="gb_b" aria-haspopup="true" role="button" tabindex="0"><div class="gb_Ec gb_Fc gb_4b"></div></a><div class="gb_lb"></div><div class="gb_kb" style="border-bottom-color:#eee"></div></div><div class="gb_ga" aria-hidden="true" aria-live="assertive"></div></div><div class="gb_fb gb_Wc gb_fg gb_R"><div class="gb_Dc gb_hb gb_fg gb_R"><a class="gb_b gb_db gb_R" href="https://accounts.google.com/SignOutOptions?hl=ru&continue=https://accounts.google.com/b/0/DelegateAccountSelector%3Fcontinue%3Dhttps%253A%252F%252Faccounts.google.com%252Fo%252Foauth2%252Fauth%253Fclient_id%253D702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com%2526redirect_uri%253Dhttps%253A%252F%252Fvkmix.com%252Fmodules%252Fyoutube%252Fbot2.php%2526scope%253Dhttps%253A%252F%252Fwww.googleapis.com%252Fauth%252Fyoutube%2526response_type%253Dcode%2526access_type%253Doffline" title="Аккаунт Google: TeAOK G9T3arz
(teaokg9t3arz#gmail.com)" role="button" tabindex="0"><span class="gb_8a gbii"></span></a><div class="gb_lb"></div><div class="gb_kb"></div></div><div class="gb_mb gb_ga" aria-label="Информация об аккаунте" aria-hidden="true"><div class="gb_pb"><a class="gb_qb gb_tf gb_sb" aria-label="Изменить картинку профиля" href="https://profiles.google.com/?hl=ru"><div class="gb_tb gbip" title="Профиль"></div><span class="gb_jb">Изменить</span></a><div class="gb_rb"><div class="gb_ub gb_vb">TeAOK G9T3arz</div><div class="gb_wb">teaokg9t3arz#gmail.com</div><div class="gb_ob">Конфиденциальность</div><a class="gb_Fa gb_qf gbp1 gb_xb" href="https://myaccount.google.com/?utm_source=OGB&utm_medium=act">Мой аккаунт</a></div></div><div class="gb_Cb"><div class="gb_Db gb_6a" aria-hidden="true"><a class="gb_Eb gb_Mb" href="/b/0/DelegateAccountSelector?continue=https%3A%2F%2Faccounts.google.com%2Fo%2Foauth2%2Fauth%3Fclient_id%3D702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com%26redirect_uri%3Dhttps%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php%26scope%3Dhttps%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube%26response_type%3Dcode%26access_type%3Doffline&authuser=0"><img class="gb_Ob gb_sb" src="data:image/gif;base64,R0lGODlhAQABAIAAAP///////yH5BAEKAAEALAAAAAABAAEAAAICTAEAOw==" alt="Профиль" data-src="https://lh3.googleusercontent.com/-Yg4816hNGF4/AAAAAAAAAAI/AAAAAAAAAAA/AAyYBF6hOAK-q02pBDqts2Bde8yxxNX1wQ/s48-c-mo/photo.jpg"><div class="gb_Hb"><div class="gb_Pb">TeAOK G9T3arz</div><div class="gb_Qb">teaokg9t3arz#gmail.com (по умолчанию)</div></div></a></div><a class="gb_Sb gb_6a" href="https://myaccount.google.com/brandaccounts?authuser=0&continue=https://accounts.google.com/b/0/DelegateAccountSelector%3Fcontinue%3Dhttps%253A%252F%252Faccounts.google.com%252Fo%252Foauth2%252Fauth%253Fclient_id%253D702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com%2526redirect_uri%253Dhttps%253A%252F%252Fvkmix.com%252Fmodules%252Fyoutube%252Fbot2.php%2526scope%253Dhttps%253A%252F%252Fwww.googleapis.com%252Fauth%252Fyoutube%2526response_type%253Dcode%2526access_type%253Doffline&service=/b/0/DelegateAccountSelector%3Fcontinue%3Dhttps%253A%252F%252Faccounts.google.com%252Fo%252Foauth2%252Fauth%253Fclient_id%253D702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com%2526redirect_uri%253Dhttps%253A%252F%252Fvkmix.com%252Fmodules%252Fyoutube%252Fbot2.php%2526scope%253Dhttps%253A%252F%252Fwww.googleapis.com%252Fauth%252Fyoutube%2526response_type%253Dcode%2526access_type%253Doffline%26authuser%3D%24authuser" aria-hidden="true"><span class="gb_Tb gb_4b"></span><div class="gb_Ub">Все аккаунты брендов »</div></a></div><div class="gb_yb"><div><a class="gb_Fa gb_pf gb_xb" href="https://accounts.google.com/AddSession?continue=https%3A%2F%2Faccounts.google.com%2Fb%2F0%2FDelegateAccountSelector%3Fcontinue%3Dhttps%253A%252F%252Faccounts.google.com%252Fo%252Foauth2%252Fauth%253Fclient_id%253D702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com%2526redirect_uri%253Dhttps%253A%252F%252Fvkmix.com%252Fmodules%252Fyoutube%252Fbot2.php%2526scope%253Dhttps%253A%252F%252Fwww.googleapis.com%252Fauth%252Fyoutube%2526response_type%253Dcode%2526access_type%253Doffline">Добавить аккаунт</a></div><div><a class="gb_Fa gb_rf gb_yf gb_xb" id="gb_71" href="https://accounts.google.com/Logout" target="_top">Выйти</a></div></div></div></div></div></div><div class="gb_Zb gb_R gb_3b" id="gbq1" style="max-width:152px;min-width:152px"><div class="gb_0b"><a class="gb_3d gb_2b" href="/" title="Мой аккаунт" data-ved="0EMQuCAQ"><span class="gb_Za gb_4b"></span></a></div></div></div><div id="gbw"></div></div><div class="gb_sd" id="gba"></div>
<div class="delegate-account-selector main content clearfix">
<div class="sign-in">
<div class="signin-box">
<h2 id="signin-action">Выберите аккаунт</h2>
<ol class="accounts" id="account-list">
<li id="account-teaokg9t3arz#gmail.com">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=none">
<span class="account-image" id="account-image-0"></span>
<span class="account-name">TeAOK G9T3arz</span>
<span class="account-email" id="account-email-0">teaokg9t3arz#gmail.com</span>
<span class="select-account text-icon-select" id="account-account-0">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=105773503169388790046">
<span class="account-image" id="account-image-1"></span>
<span class="account-name">EAccount</span>
<span class="select-account text-icon-select" id="account-account-1">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=116942572818460775716">
<span class="account-image" id="account-image-2"></span>
<span class="account-name">FAccount</span>
<span class="select-account text-icon-select" id="account-account-2">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=110025695516281123129">
<span class="account-image" id="account-image-3"></span>
<span class="account-name">GAccount</span>
<span class="select-account text-icon-select" id="account-account-3">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=100327913140968503320">
<span class="account-image" id="account-image-4"></span>
<span class="account-name">CAccount</span>
<span class="select-account text-icon-select" id="account-account-4">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=114686295257876098243">
<span class="account-image" id="account-image-5"></span>
<span class="account-name">DAccount</span>
<span class="select-account text-icon-select" id="account-account-5">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=101169419531313874972">
<span class="account-image" id="account-image-6"></span>
<span class="account-name">iAccount</span>
<span class="select-account text-icon-select" id="account-account-6">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=116191438938972075200">
<span class="account-image" id="account-image-7"></span>
<span class="account-name">AAccount</span>
<span class="select-account text-icon-select" id="account-account-7">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=106861284959818641724">
<span class="account-image" id="account-image-8"></span>
<span class="account-name">BAccount</span>
<span class="select-account text-icon-select" id="account-account-8">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=104823728391082992328">
<span class="account-image" id="account-image-9"></span>
<span class="account-name">IAccount</span>
<span class="select-account text-icon-select" id="account-account-9">Выберите</span>
</a>
</li>
<li id="account-">
<a href="https://accounts.google.com/o/oauth2/auth?client_id=702580890038-5fs7raaanbrl7rssnkg3e62qgprukbni.apps.googleusercontent.com&redirect_uri=https%3A%2F%2Fvkmix.com%2Fmodules%2Fyoutube%2Fbot2.php&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fyoutube&response_type=code&access_type=offline&pageId=102918689899435112584">
<span class="account-image" id="account-image-10"></span>
<span class="account-name">JAccount</span>
<span class="select-account text-icon-select" id="account-account-10">Выберите</span>
</a>
</li>
</ol>
</div>
</div>
<div class="side-content">
<div class="product-headers">
<h1 class="redtext">Аккаунты</h1>
</div>
<div class="select-account">
<p>
Выберите аккаунт Google
</p>
<ul>
<li>
TeAOK G9T3arz
(teaokg9t3arz#gmail.com)
</li>
<li>
EAccount
</li>
<li>
FAccount
</li>
<li>
GAccount
</li>
<li>
CAccount
</li>
<li>
DAccount
</li>
<li>
iAccount
</li>
<li>
AAccount
</li>
<li>
BAccount
</li>
<li>
IAccount
</li>
<li>
JAccount
</li>
</ul>
</div>
</div>
</div>
<div class="google-footer-bar">
<div class="footer content clearfix">
<ul id="footer-list">
<li>Google</li>
<li>
<a href="https://accounts.google.com/TOS?loc=RU&hl=ru" >Конфиденциальность и условия</a>
</li>
<li id="footer-help"><a href="https://support.google.com/accounts?hl=ru" >Справка</a></li>
</ul>
</div>
</div>
<script type="text/javascript">

You can try to locate required element by partial link text:
driver.find_element_by_partial_link_text("AAccount").click()
If it doesn't work, you can try to add some time to wait for required element:
from selenium.webdriver.support.ui import WebDriverWait as wait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
wait(driver, 10).until(EC.element_to_be_clickable((By.PARTIAL_LINK_TEXT, "AAccount"))).click()