I'm trying to use HTML data inside an the text node of an element, but it gets
encoded as if it were meant to not be HTML data.
Here is an MWE:
from xml.etree import ElementTree as ET
data = 'Example data gained from elsewhere.'
p = ET.Element('p')
p.text = data
p = ET.tostring(p, encoding='utf-8', method='html').decode('utf8')
print(p)
The output is...
<p><a href="https://example.com">Example data gained from elsewhere.</a></p>
What I intended is...
<p>Example data gained from elsewhere.</p>
What you are doing is wrong. You are assigning p.text = data, which basically considers the node to be text content. Its quite obvious the text is escaped.
You have to add it as a child. like below:
from xml.etree import ElementTree as ET
data = 'Example data gained from elsewhere.'
d = ET.fromstring(data)
p = ET.Element('p')
p.append(d)
p = ET.tostring(p, encoding='utf-8', method='html').decode('utf8')
print(p)
Giving output
<p>Example data gained from elsewhere.</p>
You can parse the HTML string into an ElementTree object and append it to the DOM:
from xml.etree import ElementTree as ET
data = 'Example data gained from elsewhere.'
p = ET.Element('p')
p.append(ET.fromstring(data))
p = ET.tostring(p, encoding='utf-8', method='html').decode('utf8')
print(p)
Related
I have created this code to substitute some strings in xml file with other text. I used Beautifulsoup for this excersise and as instructed in the documentation i used soup.prettify in the end in order to save changed xml. However prettified xml is not working for me - i get errors when trying to import it back to the CMS.
Is there any other way to save updated xml without changing xml structure and without re-writing the whole code. See my code for reference below. Thanks for advice!
import openpyxl
import sys
#searching for Part Numbers and descriptions in xml
from bs4 import BeautifulSoup
infile = open('name of my file.xml', "r", encoding="utf8")
contents = infile.read()
infile.close()
soup = BeautifulSoup(contents,'xml')
all_Products = soup.find_all('Product')
#gathering all Part Numbers from xml
for i in all_Products:
PN = i.find('Name')
PN_Descr = i.find_all(AttributeID="PartNumberDescription")
PN_Details = i.find_all(AttributeID="PartNumberDetails")
for y in PN_Descr:
PN_Descr_text = y.find("TranslatableText")
try:
string = PN_Descr_text.string
PN_Descr_text.find(text=string).replace_with("New string")
except AttributeError:
print("Attribute error in: PN Description for: ", PN)
continue
for z in PN_Details:
PN_Details_text = z.find("TranslatableText")
try:
string = PN_Details_text.string
PN_Details_text.find(text=string).replace_with("New string")
except AttributeError:
print("Attribute error in: PN Details for: ", PN)
continue
xml = soup.prettify("utf-8")
with open('name of my file.xml', "wb") as file:
file.write(xml)
After reading from an existing file with 'ugly' XML and doing some modifications, pretty printing doesn't work. I've tried etree.write(FILE_NAME, pretty_print=True).
I have the following XML:
<testsuites tests="14" failures="0" disabled="0" errors="0" time="0.306" name="AllTests">
<testsuite name="AIR" tests="14" failures="0" disabled="0" errors="0" time="0.306">
....
And I use it like this:
tree = etree.parse('original.xml')
root = tree.getroot()
...
# modifications
...
with open(FILE_NAME, "w") as f:
tree.write(f, pretty_print=True)
For me, this issue was not solved until I noticed this little tidbit here:
http://lxml.de/FAQ.html#why-doesn-t-the-pretty-print-option-reformat-my-xml-output
Short version:
Read in the file with this command:
>>> parser = etree.XMLParser(remove_blank_text=True)
>>> tree = etree.parse(filename, parser)
That will "reset" the already existing indentation, allowing the output to generate it's own indentation correctly. Then pretty_print as normal:
>>> tree.write(<output_file_name>, pretty_print=True)
Well, according to the API docs, there is no method "write" in the lxml etree module. You've got a couple of options in regards to getting a pretty printed xml string into a file. You can use the tostring method like so:
f = open('doc.xml', 'w')
f.write(etree.tostring(root, pretty_print=True))
f.close()
Or, if your input source is less than perfect and/or you want more knobs and buttons to configure your out put you could use one of the python wrappers for the tidy lib.
http://utidylib.berlios.de/
import tidy
f.write(tidy.parseString(your_xml_str, **{'output_xml':1, 'indent':1, 'input_xml':1}))
http://countergram.com/open-source/pytidylib
from tidylib import tidy_document
document, errors = tidy_document(your_xml_str, options={'output_xml':1, 'indent':1, 'input_xml':1})
f.write(document)
fp = file('out.txt', 'w')
print(e.tree.tostring(...), file=fp)
fp.close()
Here is an answer that is fixed to work with Python 3:
from lxml import etree
from sys import stdout
from io import BytesIO
parser = etree.XMLParser(remove_blank_text = True)
file_obj = BytesIO(text)
tree = etree.parse(file_obj, parser)
tree.write(stdout.buffer, pretty_print = True)
where text is the xml code as a sequence of bytes.
I am not sure why other answers did not mention this. If you want to obtain the root of the xml there is a method called getroot(). I hope I answered your question (though a little late).
tree = et.parse(xmlFile)
root = tree.getroot()
Of course - pretty print of lxml.etree is possible.
In my case, the old trick with remove_blank_text=True and pretty_print=True was not working as I expected (was too delicate), so I decided to write it by myself.
Here is it - a modern, forcible, native pythonic way to correct lxml.etee.Element tree indentation.
This gives a nicely prettified XML string:
from typing import Optional
import lxml.etree
def indent_lxml(element: lxml.etree.Element, level: int = 0, is_last_child: bool = True) -> None:
space = " "
indent_str = "\n" + level * space
element.text = strip_or_null(element.text)
if element.text:
element.text = f"{indent_str}{space}{element.text}"
num_children = len(element)
if num_children:
element.text = f"{element.text or ''}{indent_str}{space}"
for index, child in enumerate(element.iterchildren()):
is_last = index == num_children - 1
indent_lxml(child, level + 1, is_last)
elif element.text:
element.text += indent_str
tail_level = max(0, level - 1) if is_last_child else level
tail_indent = "\n" + tail_level * space
tail = strip_or_null(element.tail)
element.tail = f"{indent_str}{tail}{tail_indent}" if tail else tail_indent
def strip_or_null(text: Optional[str]) -> Optional[str]:
if text is not None:
return text.strip() or None
It's decent fast, because it doesn't allocate any additional structures in memory and also traversing the tree - it visits each node only once, giving the best possible - O x N computational complexity.
It rearranges all the existing indentation "in place" in the tree (the DOM) by correcting contents of Element.text and Element.tail attributes (affects white-spaces only).
Naturally, it also can be used with HTML parsed by lxml.
In order to use it, do something like that:
root = lxml.etree.parse("path/to/the_file.xml").getroot()
# or
root = lxml.etree.fromstring("<xml><body><leaf1/><leaf2/></body></xml>")
indent_lxml(root) # corrects indentation "in place"
result = lxml.etree.tostring(root, encoding="unicode")
print(result)
Which prints:
<xml>
<body>
<leaf1/>
<leaf2/>
</body>
</xml>
I have some code that is parsing an xml file and saving it as a csv. I can do this two ways, one by manually downloading the xml file and then parsing it, the other by taking the xml feed directly using ET.fromstring and then parsing. When I go directly I get data errors it appears to be an integrity issue. I am trying to include the xml download in to the code, but I am not quite sure the best way to approach this.
import xml.etree.ElementTree as ET
import csv
import urllib
url = 'http://www.capitalbikeshare.com/data/stations/bikeStations.xml'
connection = urllib.urlopen(url)
data = connection.read()
#I need code here!!!
tree = ET.parse('bikeStations.xml')
root = tree.getroot()
#for child in root:
#print child.tag, child.attrib
locations = []
for station in root.findall('station'):
name = station.find('name').text
bikes = station.find('nbBikes').text
docks = station.find('nbEmptyDocks').text
time = station.find('latestUpdateTime').text
sublist = [name, bikes, docks, time]
locations.append(sublist)
#print 'Station:', name, 'has', bikes, 'bikes and' ,docks, 'docks'
#print locations
s = open('statuslog.csv', 'wb')
w = csv.writer(s)
w.writerows(locations)
s.close()
f = open('filelog.csv', 'ab')
w = csv.writer(f)
w.writerows(locations)
f.close()
What you need is:
root = ET.fromstring(data)
and omit the line of: tree = ET.parse('bikeStations.xml')
As the response from connection.read() returns String, you can directly read the XML string by using fromstring method, you can read more from HERE.
I want to insert this string:
No, on the 5<Font Script="super">th</Font>
as a Text Node in XML by xml.dom.minidom createTextNode(), however, after I writexml() to a file, the signs:
< > "
turns to:
No, on the 5<Font Script="super">th</Font>
How can I avoid this? Thanks.
A part of my code:
impl = minidom.getDOMImplementation()
dom = impl.createDocument(None, None, None)
TextTextNode = dom.createTextNode(text.decode("utf-8"))
Text = dom.createElement("Text")
Text.appendChild(TextTextNode)
fileToWrite = codecs.open(output, 'w', encoding='utf-8')
dom.writexml(fileToWrite, indent=" ", addindent=" ", newl="\n", encoding='utf-8')
fileToWrite.close()
There is a sample for this by the cinecanvase specification:
<Text HAlign=”left” HPosition=”10.2” VAlign=”bottom” VPosition=”10.0”> This <Font Script=”super”>word </Font>is superscript </Text >
I need insert the <Font>..</Font> into another element, the .
I'm not familiar with that format, but that thing looks like an XML node. Try this:
from xml.dom import minidom
import codecs
output = "test.xml"
text="No, on the 5"
impl = minidom.getDOMImplementation()
dom = impl.createDocument(None, None, None)
FontNode = dom.createElement("Font")
FontNode.setAttribute('Script', 'super')
FontNode.appendChild(dom.createTextNode('th'))
Text = dom.createElement("Text")
TextTextNode = dom.createTextNode(text.decode("utf-8"))
Text.appendChild(TextTextNode)
Text.appendChild(FontNode)
fileToWrite = codecs.open(output, 'w', encoding='utf-8')
Text.writexml(fileToWrite, indent=" ", addindent=" ", newl="\n")
fileToWrite.close()
That outputs:
<Text>
No, on the 5
<Font Script="super">th</Font>
</Text>
Be aware that what you want to write a tree in a file (when you call writexml) you need to call the writexml method with your XML's tree root (you were calling it with dom, not with your root node)
I'm thinking of Python code to create a dynamic xml ETREE subElement.
I have a hierarchical header to describe a peace of book as the following:
<Books>
<Booktype List= "Story > Fiction > Young">
#here the rest of book text
</Booktype>
<Booktype List= "Science > Math > Young">
#here the rest of book text
</Booktype>
</Books>
How to get a hierarchical xml tag like this :
<Books>
<Booktype>
<Story>
<Fiction>
<Young>
#here the rest of book text
</Young>
</Fiction>
</Story>
</Booktype>
</Books>
This is my code:
import re
import xml.etree.ElementTree as ET
from xml.etree import ElementTree
List= "Story>Fiction>Young"
List = List.split('>')
root = ET.Element('Books')
Booktype =ET.SubElement(root,'Booktype')
for l in List:
ND = ET.SubElement(Booktype,str(l))
Booktype.append(ND)
tree = ET.ElementTree(root)
ElementTree.tostring(root,'utf-8')
I got this bad result:
'<Books><Booktype><Story /><Story /><Story /><Fiction /><Fiction /><Young /><Young /><Story /><Story /><Fiction /><Fiction /><Young /><Young /></Booktype></Books>'
If you want to nest the list elements you have to keep the reference to the previous one so you can add the child element to it, and not to the Booktype element. See the variable currrent in the examples.
from xml.etree import ElementTree as ET
xml_string = '''<Books>
<Booktype List= "Story > Fiction > Young">
#here the rest of book text
</Booktype>
<Booktype List= "Science > Math > Young">
#here the rest of book text 2
</Booktype>
</Books>
'''
xml = ET.fromstring(xml_string)
for booktype in xml.findall('Booktype'):
types = map(lambda x: x.strip(), booktype.get('List').split('>'))
current = booktype
for t in types:
current = ET.SubElement(current, t)
current.text = booktype.text
booktype.text = ''
del booktype.attrib['List']
print ET.tostring(xml,'utf-8')
Gives me the result:
<Books>
<Booktype><Story><Fiction><Young>
#here the rest of book text
</Young></Fiction></Story></Booktype>
<Booktype><Science><Math><Young>
#here the rest of book text 2
</Young></Math></Science></Booktype>
</Books>
And if you want to create a completely new structure you can do:
xml = ET.fromstring(xml_string)
root = ET.Element('Books')
for booktype in xml.findall('Booktype'):
current = ET.SubElement(root, 'Booktype')
for t in map(lambda x: x.strip(), booktype.get('List').split('>')):
current = ET.SubElement(current, t)
current.text = booktype.text
print ET.tostring(root, 'utf-8')