I need to check whether a link's disabled attribute is set, in the following code,
<a id="ctl00_ContentPlaceHolder1_lbtnNext" disabled="disabled">Next</a>
However on the last page if I execute,
next_pg=driver.find_element_by_xpath("//a[#id='ctl00_ContentPlaceHolder1_lbtnNext']")
next_pg.click()
print next_pg.is_enabled()
I get True as the output, which should not be the case.
Also, only on the last page is the Next coded as given above, in all other pages it is coded as follows, due to which on checking the is_enabled() tag, an error is produced.
<a id="ctl00_ContentPlaceHolder1_lbtnNext" href="javascript:__doPostBack('ctl00$ContentPlaceHolder1$lbtnNext','')">
How should I solve this?
Use this answer to get the attributes of the tag:
attrs = driver.execute_script('var items = {}; for (index = 0; index < arguments[0].attributes.length; ++index) { items[arguments[0].attributes[index].name] = arguments[0].attributes[index].value }; return items;', next_pg)
and check for the presence of the disabled tag and it's value:
if 'disabled' in attrs and attrs['disabled'] == 'disabled':
# ...
Related
I make a request to my ES in Python but I can only get 10,000 data. I will need to recover many more ( several hundred thousand).
I've modified the "size" variable but it can't go over 10.000
res_cpe = es.search(index=cpe_index, doc_type="entries", body = {
'size' : 10000,
'query': {
'match_all' : {}
}
})
I would like to have all entries in my "res_cpe" variable
You should try to use Scroll API which should help you to retrieve large numbers of results (or even all results, like in your case).
This functionality is similar to cursors from a traditional databases.
All you need to do, is to add scroll param to your request in Python client. Minimum viable example could look like this:
page = es.search(
index = 'yourIndex',
doc_type = 'yourType',
scroll = '2m',
search_type = 'query_then_fetch',
size = 1000,
body = {
//Your query's body
})
sid = page['_scroll_id']
scroll_size = page['hits']['total']
//Start scrolling
while (scroll_size > 0):
print "Scrolling..."
page = es.scroll(scroll_id = sid, scroll = '2m')
//Update the scroll ID
sid = page['_scroll_id']
//Get the number of results that we returned in the last scroll
scroll_size = len(page['hits']['hits'])
print "scroll size: " + str(scroll_size)
//Do something with the obtained page
Example taken from here - https://gist.github.com/drorata/146ce50807d16fd4a6aa
Python client docs reference - https://elasticsearch-py.readthedocs.io/en/master/api.html
When I tried to use urls to scrape web pages, I found that some elements only exists in some pages and other have not. Let's take the code for example
Code:
for urls in article_url_set:
re=requests.get(urls)
soup=BeautifulSoup(re.text.encode('utf-8'), "html.parser")
title_tag = soup.select_one('.page_article_title')
if title_tag=True:
print(title_tag.text)
else:
#do something
if title_tag exits, I want to print them, if it's not, just skip them.
Another thing is that, I need to save other elements and title.tag.text in data.
data={
"Title":title_tag.text,
"Registration":fruit_tag.text,
"Keywords":list2
}
It will have an error cause not all the article have Title, what should I do to skip them when I try to save? 'NoneType' object has no attribute 'text'
Edit: I decide not to skip them and keep them as Null or None.
U code is wrong:
for urls in article_url_set:
re=requests.get(urls)
soup=BeautifulSoup(re.text.encode('utf-8'), "html.parser")
title_tag = soup.select_one('.page_article_title')
if title_tag=True: # wrong
print(title_tag.text)
else:
#do something
your code if title_tag=True,
changed code title_tag == True
It is recommended to create conditional statements as follows.
title_tag == True => True == title_tag
This is a way to make an error when making a mistake.
If Code is True = title_tag, occur error.
You can simply use a truth test to check if the tag is existing, otherwise assign a value like None, then you can insert it in the data container :
title_tag = soup.select_one('.page_article_title')
if title_tag:
print(title_tag.text)
title = title_tag.text
else:
title = None
Or in one line :
title = title_tag.text if title_tag else None
I'm working from the instance id. I can get the tags, but I can't figure out how filter on both the instance id and the tag "Name" or index in and return the value of the tag called "Name"
I ended up grabbing all tags and iterating over them till I find the one I want, which can't possibly be right.
tags = conn.get_all_tags({'resource-id': instance_id})
for tag in tags:
if 'Name' in tag.name:
name = tag.value
You do have a better way:
conn.get_all_tags(filters={'tag-key': 'Name', 'resource-id': instance_id})
I think the way you're doing it is fine. You could always wrap it in a function call as an abstraction:
def get_instance_tag(all_tags, tag_name):
for tag in all_tags:
if tag_name == tag.name:
return tag.value
return None
name = get_instance_tag(conn.get_all_tags({'resource-id': instance_id}), 'Name')
Note that if tag_name == tag.name: is more accurate than if tag_name in tag.name:.
I am usign Flask-SQLAlchemy’s paginate(). Now I need to find what is the page for a specific comment id.
For example, this will work, if I have all comments in the same page:
new_dict['url'] = '/comments#comment_' + str(comment.id)
However in my case I need this structure:
/comments?page=1#comment_73
How can I find what is the page?
From the docs, the Pagination class has .items and .has_next properties and a .next method we can use:
page_number = 0
search = Comment.query.get(15)
query = Comment.query.filter(Comment.id<40)
for num in range(1, query.paginate(1).pages + 1):
if search in query.paginate(num).items:
page_number = num
break
or
page_number = 0
search = Comment.query.get(15)
pag = Comment.query.filter(Comment.id<40).paginate(1)
while pag.has_next:
if search in pag.items:
page_number = num
break
pag.next()
As far as I know, Celeo's answer won't work. For example, what pag.next() does in his code, based on documentations is:
Returns a Pagination object for the next page.
So, basically, it's doing nothing unless you update your variable; and I recommend you to not create a new query since you already have the comment_id so:
comment_id=request.args.get('comment_id')
if comment_id and comment_id.isdigit():
comment_id = int(comment_id )
page_number = -1
index = 1 # page numbers are 1 indexed in Pagination Object
while comments_pagination_object.has_next:
for comment in comments_pagination_object.items:
if comment.id == comment_id :
page_number = index
break
if page_number != -1:
break
index += 1
product_items = product_items.next()
Then, in the URL, you will have something like:
/comments?comment_id=2
and the part product_items.next() is changing the PaginationObject's page till one of it's items (which in this case is a type of class Comment) has the same id as your request args.
In a multi-page survey application I am creating I have a jQuery UI slider bar which is used to provide a rating for an image. This returns a numerical value to my Python/Django view which is stored in a list slider_DV_values
On a later Data Verification survey page the participant is given the opportunity via another jQuery slider bar to update the rating they assigned the image.
My issue is that the jQuery UI slider bar only returnes a numerical value if the participant changes it. Therefore the original rating is getting overwritten, with nothing, if the participant does not update it.
However if they do update their rating the new value is getting stored.
If I try
elif step == 13:
slider_value1 = self.request.POST.get('slider_value1')
print "This is slider_value1", slider_value1
if slider_value1 is not None:
slider_DV_values.pop(0)
slider_DV_values.insert(0, slider_value1)
The original values stored in slider_DV_values are still getting overwritten, with nothing.
I thought the is not None would have prevented an empty value from being used to overwrite the original value? IS this not correct?
Can anyone tell me how to prevent the original values from getting overwritten unless the new value is an updated numerical value?
Thanks, Deepend
EDIT
To see how I am getting my values this is the jQuery slider bar in a page of my SurveyWizardView the value of which is returned via the hidden form element
<div class="DV_image_row">
<div class="DV_image_left">
<img src="{% static "survey/images/pathone/" %}{{first_image}}{{fourth_image}}{{seventh_image}}" height="300" width="250" style="border:1px solid black;" align="middle"/>
<div class="DV_slider_one" id="one"></div>
<script >
$('#submit').click(function() {
var username = $('#hidden').val();
if (username == "") username = 0;
$.post('comment.php', {
hidden: username
}, function(return_data) {
alert(return_data);
});
});
$(".DV_slider_one").slider({
animate: true,
range: "min",
value: {{first_slider}}{{forth_slider}}{{seventh_slider}},
min: -100,
max: +100,
step: 1,
slide: function(event, ui) {
$("#slider-result_left").html((ui.value > 0 ? '+' : '') + ui.value);
if($(this).attr("id") == "one")
$("#hidden2").val((ui.value > 0 ? '+' : '') + ui.value);
}
});
</script>
<div id="slider-result_left">{{first_slider}}{{forth_slider}}{{seventh_slider}}</div>
<input type="hidden" name="slider_value1" id="hidden2"/>
</div>
"" is None is False
ONLY None is None is True
perhaps you just want if not slider_value1 : which is true for ANY falsey value (empty string,empty list, empty tuple, false, 0 , None , etc)
You might want to try the simple if slider_value1 != '' . This is a better option than if not slider_value1 because the latter will block out 0 also which you might not want to do.