Develop Reference

Getting the content of any specific tag anywhere in an XML file with Python

I have an XML file like this:
<file>
<dict>
<element>
<s>text</s>
<i>56</i>
<s>foobar</s> <--- strange formatting issues cause it to be below the root tag
... more tags like these
</element>
...more <element>s like above
</dict>
</file>
How can I get the content between all <s> and </s> tags in the entire file, no matter where the tags are? I'm running Python 3.10.4 and I prefer using the builtin xml module. (If I need to use a different one, I'd be OK with that!)
(Also, assume I don't want to fix the <s>foobar<\s> issue.)
Output I want:
text
foobar
...more content from between <s></s> tags

How to output elements in ElementTree PIParser?

If you want to preserve comments in an XML file with ElementTree, you can use the PIParser from http://effbot.org/zone/element-pi.htm
So if I have a file containing
<?xml version="1.0" encoding="iso-8859-1"?>
<!-- This is a comment -->
<root>
<foo>Hello World</foo>
</root>
<!-- That's all, folks -->
then the two comments will be preserved.
PIParser wraps the xml in another <document> node, so there is something to contain any comments that come outside the root node. The two comments are just two more elements contained in <document>.
But then how should I output the xml? I use code like this to output the contents of <document> without outputting the <document> tag itself:
file.write('<?xml version="1.0" encoding="%s"?>\n' % encoding)
doc = tree.getroot()
for child in doc:
file.write(ET.tostring(child, encoding, method))
file.write("\n")
Now, that seems to work if encoding is "utf-8". The tostring() method does not output one of those <?xml...?> lines, presumably because utf-8 is the default. That's why I wrote the <?xml...?> explicitly with file.write() above. But if encoding is "iso-8859-1" then tostring() puts a <?xml version="1.0" encoding="iso-8859-1"?> at the start of every element for which it is called! So I get one in front of every comment outside the root node, and another in front of the root node itself. I don't want that, I only want one at the top of the file. But instead of getting the xml as shown above, I get
<?xml version="1.0" encoding="iso-8859-1"?>
<?xml version='1.0' encoding='iso-8859-1'?>
<!-- This is a comment -->
<?xml version='1.0' encoding='iso-8859-1'?>
<root>
<foo>Hello World</foo>
</root>
<?xml version='1.0' encoding='iso-8859-1'?>
<!-- That's all, folks -->
How do I control whether tostring() outputs a <?xml...?>? Or, should I be doing this another way?

copying input xml file and write exactly with Python

Input xml file:
<?xml version="1.0"?>
<res:testcases xmlns:res="urn:testcases" id="a1e4bfdb-40a2-485c-a1ac-54d220056dd5" type="MODEL">
<mode>PRESSURE_CONTROL</mode>
<category>ADULT</category>
<testcase id="1" type="UNIQUE">
<parameter id="PEEP" value="1.0">true</parameter>
<parameter id="CMV_FREQ" value="4.0">true</parameter>
<parameter id="PRESS_ABOVE_PEEP" value="0.0">true</parameter>
<parameter id="I_E_RATIO" value="0.1">false</parameter>
</testcase>
</res:testcases>
Python Code:
import xml.etree.ElementTree as ET
tree = ET.parse('/home/AlAhAb65/Desktop/input.xml')
root = tree.getroot()
root.attrib['type'] = 'AVA'
tree.write('/home/AlAhAb65/Desktop/output1.xml')
Output xml file:
<ns0:testcases id="a1e4bfdb-40a2-485c-a1ac-54d220056dd5" type="AVA" xmlns:ns0="urn:testcases">
<mode>PRESSURE_CONTROL</mode>
<category>ADULT</category>
<testcase id="1" type="UNIQUE">
<parameter id="PEEP" value="1.0">true</parameter>
<parameter id="CMV_FREQ" value="4.0">true</parameter>
<parameter id="PRESS_ABOVE_PEEP" value="0.0">true</parameter>
<parameter id="I_E_RATIO" value="0.1">false</parameter>
</testcase>
</ns0:testcases>
The problem is when I am copying and writing the output xml file 3 unexpected things happen. They are given below:
1. The first line from the input xml file is removed automatically
2. In second line (in input), the text 'res' is replaced with 'ns0'. Same happens while closing the tag
3. The order of the attribute (of the second line of input) is changed.
But I want to write (as output) the exact copy of xml file that I got as an input. Please help me in this regard.

W3 has defined a Canonical XML standard. Documents written in this format can be faithfully round-tripped by any C14N-compliant toolchain.
In the case of lxml.etree (a more capable implementation of the ElementTree API with C14N support), this means that you need to do two things:
Convert your original input document into C14N form.
Use the ElementTree.write_c14n() call to generate your output document.
A C14N-form version of your input file will look like so (generated by the xmlstarlet c14n command):
<res:testcases xmlns:res="urn:testcases" id="a1e4bfdb-40a2-485c-a1ac-54d220056dd5" type="MODEL">
<mode>PRESSURE_CONTROL</mode>
<category>ADULT</category>
<testcase id="1" type="UNIQUE">
<parameter id="PEEP" value="1.0">true</parameter>
<parameter id="CMV_FREQ" value="4.0">true</parameter>
<parameter id="PRESS_ABOVE_PEEP" value="0.0">true</parameter>
<parameter id="I_E_RATIO" value="0.1">false</parameter>
</testcase>
</res:testcases>
...and an appropriately modified version of your code:
#!/usr/bin/env python
import lxml.etree
tree = lxml.etree.parse('input.xml')
root = tree.getroot()
root.attrib['type'] = 'AVA'
tree.write_c14n('output1.xml')
If you add an XML declaration (the <?xml version="1.0"?> line), you will be noncomplaint with the C14N standard. As such, this is something you absolutely should not do. If you really, really want to do this wrongheaded thing...
Don't.
But if you must, you'd do it like so:
outfile = open('output1.xml', 'w')
outfile.write('<?xml version="1.0"?>\n')
tree.write_c14n(outfile)
outfile.close()

From the documentation page, the XML declaration can be added like this:
tree.write('/home/AlAhAb65/Desktop/output1.xml', xml_declaration=True)
You should also add the encoding because the default one is us-ascii:
tree.write('/home/AlAhAb65/Desktop/output1.xml', encoding='utf-8', xml_declaration=True)
Or you can retrieve the encoding from the original file, but in any case you will get a different XML declaration, probably something like this:
<?xml version="1.0" encoding="UTF-8"?>
Or you can manually add the XML declaration. Anyway a slight declaration mismatch should not be a problem for any robust XML parser as long as the declared encoding is coherent with the real encoding.
Attribute order is not significant in XML, so the information is probably lost when the file is parsed within the API. There is probably no simple way to make this work when processing the file through the standard ElementTree API. You would probably better have to go with lxml C14N support if you want to do minor changes to the file.
The namespace prefixes are changed by default in ElementTree. To prevent this behavior, you can switch to lxml which seems to preserve namespace prefixes by default:
Because etree is built on top of libxml2, which is namespace prefix aware, etree preserves namespaces declarations and prefixes while ElementTree tends to come up with its own prefixes (ns0, ns1, etc). When no namespace prefix is given, however, etree creates ElementTree style prefixes as well.
Switching to lxml is a good idea in any case, but the changes you observe should not be a problem if the program reading the file at the other end is XML compliant enough. Unfortunately a lot of XPath processors have issues with namespace prefixes changes...

Get some unexpected changes in xml file when use python/elementtree

Here is the original xml file:
<?xml version="1.0" encoding="UTF-8"?>
<TVAMain xml:lang="en-GB" xmlns="urn:tva:metadata:2010" xmlns:tva2="urn:tva:metadata:extended:2010" xmlns:yv="http://refdata.youview.com/schemas/Metadata/2012-10-16" xmlns:mpeg7="urn:tva:mpeg7:2008" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://refdata.youview.com/schemas/Metadata/2012-09-26 ../schemas/youview_metadata_2012-09-26.xsd">
<!-- -->
<ProgramDescription> .............................
I changes some of the content of the xml(but not the one I post here, those codes should be unchanged), then write to a new xml file, but the new xml file content become like this:
<?xml version='1.0' encoding='UTF-8'?>
<TVAMain xmlns="urn:tva:metadata:2010" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://refdata.youview.com/schemas/Metadata/2012-09-26 ../schemas/youview_metadata_2012-09-26.xsd" xml:lang="en-GB">
<ProgramDescription>....................
you can see that the some contents are lost, and the order is also changed, what should I do in order to avoid any changes to xml?

Attributes on XML tags do not have a fixed order, changing their ordering doesn't change their meaning.
ElementTree will only write out namespace qualifiers for namespaces actually in use. Your example is very brief, but I suspect it doesn't make use of the yv and mpeg7 namespaces at all.

Issues when writing an xml file using xml.dom.minidom python

I have an xml file and a python script is used for adding a new node to that xml file.I used xml.dom.minidom module for processing the xml file.My xml file after processing with the python module is given below
<?xml version="1.0" ?><Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PostBuildEvent>
<Command>xcopy "SourceLoc" "DestLoc"</Command>
</PostBuildEvent>
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<Import Project="project.targets"/></Project>
What i actually needed is as given below .The changes are a newline character after the first line and before the last line and also '&quot' is converted to "
<?xml version="1.0" ?>
<Project DefaultTargets="Build" ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
<PostBuildEvent>
<Command>xcopy "SourceLoc" "DestLoc"</Command>
</PostBuildEvent>
<ImportGroup Label="ExtensionTargets">
</ImportGroup>
<Import Project="project.targets"/>
</Project>
The python code i used is given below
xmltree=xml.dom.minidom.parse(xmlFile)
for Import in Project.getElementsByTagName("Import"):
newImport = xml.dom.minidom.Element("Import")
newImport.setAttribute("Project", "project.targets")
vcxprojxmltree.writexml(open(VcxProjFile, 'w'))
What should i update in my code to get the xml in correct format
Thanks,

From docs of minidom:
Node.toprettyxml([indent=""[, newl=""[, encoding=""]]])
Return a pretty-printed version of the document. indent specifies the indentation string and defaults to a tabulator; newl specifies the string emitted at the end of each line and defaults to \n.
That's all customisation you get from minidom.
Tried inserting a Text node as a root sibling for newline. Hope dies last.
I recommend using regular expressions from re module and inserting it manually.
As for removing SGML entities, there's apparently an undocumented function for that in python standard library:
import HTMLParser
h = HTMLParser.HTMLParser()
unicode_string = h.unescape(string_with_entities)
Alternatively, you can do this manually, again using re, as all named entity names and corresponding codepoints are inside the htmlentitydefs module.