I have a small script that returns the title of a webpage
title = BeautifulSoup(urllib2.urlopen(URL)).title.string
But not all websites specify a <title> tag, in which case my script returns
AttributeError: 'NoneType' object has no attribute 'string'
Is there a way to have the title variable equal to the title if there is a title on the webpage?
I am tried
if BeautifulSoup(urllib2.urlopen(url)).title.string.strip():
print BeautifulSoup(urllib2.urlopen(url)).title.string.strip()
else:
print url
But it still raises the AttributeError error
try:
title = BeautifulSoup(urllib2.urlopen(URL)).title.string
except AttributeError:
title = url
You can use inbuilt function getattr to check whether the attribute exists and if not set a default value to it.
In your case it will be like
title = getattr(BeautifulSoup(urllib2.urlopen(URL)).title, 'string', 'default title')
Check documentation for getattr function - https://docs.python.org/2/library/functions.html#getattr
Related
Im writing a google forms submitter and I'm having problems with the textfield-type questions.
Basically I am using:
textfield = question.find_element_by_class_name("quantumWizTextinputPaperinputInput")
to find the textfield and then the problems start coming in. The type of "textfield" is:<selenium.webdriver.firefox.webelement.FirefoxWebElement (session="1b49148e-2a24-4efb-b3a5-e84be92223ae", element="3b437c8b-8d05-4410-8047-bcac9ea81f0f")>
and when I want to call .send_keys(string) on it it says that Exception has occurred: AttributeError 'list' object has no attribute 'send_keys'
So basically it says that the element returned is a list (noenetheless that type() returns a firefoxwebdriver element type).
So if I try to go with textfield[0] or textfield[1] etc... it of course throws an error that a FirefoxWebDriver is not subscribable.
What the frick?
Here's the block of code:
buttons = question.find_elements_by_class_name("appsMaterialWizToggleRadiogroupRadioButtonContainer")
buttons2 = question.find_elements_by_class_name("quantumWizTogglePapercheckboxInnerBox")
try:
textfield = question.find_element_by_class_name("quantumWizTextinputPaperinputInput")
except:
print("not found")
textfield = []
pass
And then below to send keys into it:
if len(buttons) == 0 and len(buttons2) == 0:
print(textfield)
textfield.send_keys("lol spam")
try:
textfield = question.find_element_by_class_name("quantumWizTextinputPaperinputInput")
except:
print("not found")
textfield = []
pass
The problem lies within this snippet. If textfield, or to be more specific the class_name quantumWizTextinputPaperinputInput can't be found, Python continues to evaluate the except block. Within there you stated textfield = [] - that's the reason for your problems:
Exception has occurred: AttributeError 'list' object has no attribute
'send_keys' So basically it says that the element returned is a list
(noenetheless that type() returns a firefoxwebdriver element type). So
if I try to go with textfield[0] or textfield[1] etc... it of course
throws an error that a FirefoxWebDriver is not subscribable.
You can't send send_keys to a list.
List is empty, hence a textfield[0] should throw IndexError.
A solution to this problem is to find the proper class_name. Without a HTML code we can't help you to do that.
Whenever I run the code, the error is this:
'Template' object has no attribute 'replace'
Any help would be nice. Thanks!
The line
template = get_template("user/ticket_print.html")
returns a Template object and not a string containing the code you put in your template (which I think you expect). And the Template object does not have a method called replace() resulting in your error.
To access the string you can use
template.template.source
So I think in your case a slight change in your view could do the trick:
def ticket_print(request, cartitem_id):
item = get_object_or_404(CartItem, object_id=cartitem_id)
template = get_template("user/ticket_print.html")
value = template.template.source
html_result = render_template(value, {"itest": item.cart,},)
return html_to_pdf(html_result, name=f"Ticket_Print{item}")
I want to extract Phone number from this div. This div has style="display:none"
So I can not access the children of this div. Please help me out in getting the phone number from the div.
I guess We need to change that display:none; to visibility:visible. How can I do this in Python Selenuim
Edit
I have tried the code below, as suggested in the first answer but it throws the following error:
email_div = browser.find_element_by_class_name("returnemail")
email_div_contents = browser.execute_script('return arguments[0].innerHTML', email_div)
telephone = email_div_contents.find_element_by_class_name('reply-tel-number').get_attribute('textContent')
AttributeError: 'str' object has no attribute
'find_element_by_class_name
'
As per the documentation execute_script() returns:
The command's JSON response loaded into a dictionary object.
Hence, moving forward when you attempted to invoke find_element_by_class_name() method on the dictionary object as follows:
email_div_contents.find_element_by_class_name('reply-tel-number').get_attribute('textContent')
The following error is raised:
AttributeError: 'str' object has no attribute 'find_element_by_class_name'
To remove the attribute style="display:none" from the desired element and extract the phone number you can use the following solution:
element = driver.find_element_by_xpath("//div[#class='returnemail js-only']")
driver.execute_script("arguments[0].removeAttribute('style')", element)
tel_number = element.find_element_by_xpath("./aside/ul//li//p[#class='reply-tel-number']").get_attribute("innerHTML")
Your code has incorrect place:
email_div = browser.find_element_by_class_name("returnemail")
email_div_contents = browser.execute_script('return arguments[0].innerHTML', email_div)
email_div_contents.find_element_by_class_name()
email_div_contents is a string represents the HTML code of the email_div, not a web element,
You can't call find_element_by_class_name() on a string.
That's why you got error:
'str' object has no attribute 'find_element_by_class_name'
You can always call get_attribute() to fetch attribute value on visible and invisible element.
To get the text content of invisible element you can use get_attribute('innerText').
phone_number = driver.find_element_by_css_selector("div.returnemail .reply-tel-number")
.get_attribute('innerText')
Actually, element.text call element.get_attribute('innerText') inside, but element.text will respect user experience: If user can't see the element from page, element.text will return empty string as user see. (Even element.get_attribute('innerText') return non-empty string)
#property
text:
if ( element is visible ):
return element.get_attribute('innerText')
else:
return ''
I have a nested dictionary like so:
mail = {
'data': { 'from': {'text': '123#example.com'}}
# some other entries...
}
I'm trying to copy from value using following code:
data = mail.get('data')
new_dict['parse']['from'] = data.get('from').get('text')
The second line throws the exception:
AttributeError: 'NoneType' object has no attribute 'get'
The strange thing is, this only happens sometimes. If I add a print statement just before the second line like:
data = mail.get('data')
print(type(data.get('from')))
new_dict['parse']['from'] = data.get('from').get('text')
The error disappears and I get <class 'dict'> as expected. If I remove the print statement, it works sometimes and other times it throws the error. Nothing else changes in the code or data. The reason I'm using get() is to retrieve value safely in case the key is missing.
In the call data.get('from').get('text'), if data does not contain the key 'from', it will return None. None.get('text') raises then the exception you see, because a None object has no get method (of course).
The way around this is to pass in a better default-object than None (the default default-object), which has the get method. That would be an empty dictionary, {}:
data = mail.get('data')
new_dict['parse']['from'] = data.get('from', {}).get('text')
I'm parsing objects returned by the suds client from a Web Services SOAP API
I have a list of attributeObjects, like
(defectStateAttributeValueDataObj){
attributeDefinitionId =
(attributeDefinitionIdDataObj){
name = "Comment"
}
attributeValueId =
(attributeValueIdDataObj){
name = "Owner changed because of default owner assignment specified in the component map"
}
}
and
(defectStateAttributeValueDataObj){
attributeDefinitionId =
(attributeDefinitionIdDataObj){
name = "OwnerName"
}
attributeValueId =
(attributeValueIdDataObj){
name = "Schmoe, Joe"
}
}
I use the following loop to extract key/value pairs:
for defect in myDefectsPage.mergedDefects :
print defect.cid,
for attribute in defect.defectStateAttributeValues:
print attribute
attr= attribute.attributeDefinitionId.name
val=attribute.attributeValueId.name
print attr,'=',val,
print ""
(The above objects are results of the print attribute command)
This will work as expected for EVERY attribute, except the one where attribute.attributeDefinitionId.name == "Comment"
for that one I get the
Traceback (most recent call last):
File , line 63, in
val=attribute.attributeValueId.name
AttributeError: 'Text' object has no attribute 'name'
which is strange, because if I use
val=attribute.attributeValueId #.name
it will print
Commment = (attributeValueIdDataObj){
name = "Owner changed because of default owner assignment specified in the component map"
}
So it looks like it IS an attributeValueIdDataObj and DOES have a name attribute.
I used the suds DEBUG logging and the XML return elements look exactly the same regardless of what the attribute.attributeDefinitionId.name is.
I have no idea how it changes into a 'Text' object when trying to access the name attribute
Any ideas?
On further examination (and printing out the type of the returned object when exception happened) this was a bug in the web services SOAP server.
When a Comment was empty, it returned an
<attributeValueId/>
tag,
instead of the proper
<attributeValueId>
<name/>
</attributeValueId>
object. so it resulted in sax.Text object instead of the suds.attributeValueIdDataObj object
So no python or suds mystery to solve.
Sorry for the false alarm...