Develop Reference

Parsing a deeply nested xml file using a for loop

How could I efficiently pull data from the nested xml?
By efficiently, I mean for example using a for loop.
Would I need to make use a of new data structure?
Parsing function:
import xml.etree.ElementTree as ET
it = ET.iterparse('OTA_AirSeatMapRS.xml')
# This for loop removes the namespaces
for _, el in it:
_, _, el.tag = el.tag.rpartition('}')
root = it.root
# I am not able to select data with this loop
for x in element.find(Service):
print(x)
This is part of the XML file:
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope
xmlns:soapenc="http://schemas.xmlsoap.org/soap/encoding/"
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:xsd="http://www.w3.org/2001/XMLSchema"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<soapenv:Body>
<ns:OTA_AirSeatMapRS Version="1"
xmlns:ns="http://www.opentravel.org/OTA/2003/05/common/">
<ns:Success/>
<ns:SeatMapResponses>
<ns:SeatMapResponse>
<ns:FlightSegmentInfo DepartureDateTime="2020-11-22T15:30:00" FlightNumber="1179">
<ns:DepartureAirport LocationCode="LAS"/>
<ns:ArrivalAirport LocationCode="IAH"/>
<ns:Equipment AirEquipType="739"/>
</ns:FlightSegmentInfo>
<ns:SeatMapDetails>
<ns:CabinClass Layout="AB EF" UpperDeckInd="false">
<ns:RowInfo CabinType="First" OperableInd="true" RowNumber="1">
<ns:SeatInfo BlockedInd="false" BulkheadInd="false" ColumnNumber="1" ExitRowInd="false" GalleyInd="false" GridNumber="1" PlaneSection="Left">
<ns:Summary AvailableInd="false" InoperativeInd="false" OccupiedInd="false" SeatNumber="1A"/>
<ns:Features>Window</ns:Features>
</ns:SeatInfo>
My eventual goal is to use the parsed data to store in a JSON.

Take a look at the following instruction in your code:
for x in element.find(Service):
The first flaw in your code sample is that:
Service is a variable (not a string literal),
probably you initialized this variable to some string, but failed
to put this instruction in your code sample.
The source of another flaw is that find finds the first element
matching the given path, so you should not use it in a loop.
Maybe you should also check whether find returned some not-None
content, but this is another detail.
The third reason why you got the empty outptut is that print(x)
prints actually only the text of the element in question.
So to have a more general example, run:
Service = 'Summary'
x = root.find(f'.//{Service}')
print(f'{x.tag}, {x.text}, {x.attrib}')
The first instruction sets the tag name.
The second instruction invokes find, but note that I added './/'
to the XPath, to look at any depth of the source XML tree.
And the last instruction prints not only text of the element found,
but also the tag name and attributes.
The result I got (for your input XML) is:
Summary, None, {'AvailableInd': 'false', 'InoperativeInd': 'false', 'OccupiedInd': 'false', 'SeatNumber': '1A'}
(text is just None, so you didn't see any result in your original
output).

Iterparse object in Python is not returning iter object

I'm working with the XML file in this link (Downloadable file of 40MB). In this file, I'm expecting data from 2 types of tags.
Those are: OpportunityForecastDetail_1_0 and OpportunitySynopsisDetail_1_0.
I wrote the following code for that:
ARTICLE_TAGS = ['OpportunitySynopsisDetail_1_0', 'OpportunityForecastDetail_1_0']
for _tag in ARTICLE_TAGS:
f = open(xml_f)
context = etree.iterparse(f, tag = _tag)
for _, e in context:
_id = e.xpath('.//OpportunityID/text()')
text = e.xpath('.//OpportunityTitle/text()')
f.close()
Then etree.iterparse(f, tag = _tag) is returning an object which is not iterable. I think this occurs when the tag is not found in the XML file.
So, I added name spaces to the iterable tag like this.
context = etree.iterparse(f, tag='{http://apply.grants.gov/system/OpportunityDetail-V1.0}'+_tag)
Now, it is creating an iterable object. But, I'm not getting any text. I tried other namespaces in that file. But, not working.
Please tell me the solution to this problem. This is a sample snippet of the XML file. OpportunityForecastDetail_1_0 and OpportunitySynopsisDetail_1_0 tags are repeated n number of times in the XML file.
<?xml version="1.0" encoding="UTF-8"?>
<Grants xsi:schemaLocation="http://apply.grants.gov/system/OpportunityDetail-V1.0 https://apply07.grants.gov/apply/system/schemas/OppotunityDetail-V1.0.xsd" xmlns="http://apply.grants.gov/system/OpportunityDetail-V1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instace">
<OpportunitySynopsisDetail_1_0>
<OpportunityID>262148</OpportunityID>
<OpportunityTitle>Establishment of the Edmund S. Muskie Graduate Internship Program</OpportunityTitle>
</OpportunitySynopsisDetail_1_0>
<OpportunityForecastDetail_1_0>
<OpportunityID>284765</OpportunityID>
<OpportunityTitle>PPHF 2015: Immunization Grants-CDC Partnership: Strengthening Public Health Laboratories-financed in part by 2015 Prevention and Public Health Funds</OpportunityTitle>
</OpportunityForecastDetail_1_0>
</Grants>

First, when you are parsing XML that contains namespaces, you must use those namespaces when looking at tag names.
Second, iterparse doesn't take an argument named tag, so I don't see how your code could have worked as posted.
Finally, the elements returned from iterparse don't have a member function called xpath, so that can't have worked either.
Here is an example of how to parse XML using iterparse:
NS='{http://apply.grants.gov/system/OpportunityDetail-V1.0}'
ARTICLE_TAGS = [NS+'OpportunitySynopsisDetail_1_0', NS+'OpportunityForecastDetail_1_0']
with open(xml_f, 'r') as f:
context = etree.iterparse(f)
for _, e in context:
if e.tag in ARTICLE_TAGS:
_id = e.find(NS+'OpportunityID')
text = e.find(NS+'OpportunityTitle')
print(_id.text, text.text)
As I said in my comment, the Python documentation is helpful, as is the Effbot page on ElementTree. There are lots of other resources available; put xml.etree.elementtree into Google and start reading!

Creating Properly-Nested XML Output in Python

I'm attempting to save data from several lists in XML format, but I cannot understand how to make the XML display properly. An example of my code right now is as follows:
from lxml import etree
#Create XML Root
articles = etree.Element('root')
#Create Lists & Data
t_list = ['title1', 'title2', 'title3', 'title4', 'title5']
c_list = ['content1', 'content2', 'content3', 'content4', 'content5']
sum_list = ['summary1', 'summary2', 'summary3', 'summary4', 'summary5']
s_list = ['source1', 'source2', 'source3', 'source4', 'source5']
i = 0
for t in t_list:
for i in range(len(t_list)):
#Create SubElements of XML Root
article = etree.SubElement(articles, 'Article')
titles = etree.SubElement(article, 'Title')
summary = etree.SubElement(article, 'Summary')
source = etree.SubElement(article, 'Source')
content = etree.SubElement(article, 'Content')
#Add List Data to SubElements
titles.text = t_list[i]
summary.text = sum_list[i]
source.text = s_list[i]
content.text = c_list[i]
print(etree.tostring(articles, pretty_print=True))
My Current Output is written in one very jumbled fashion, all on a single line as follows:
b'<root>\n <Article>\n <Title>title1</Title>\n <Summary>summary1</Summary>\n <Source>source1</Source>\n <Content>content1</Content>\n </Article>\n
It looks like the pretty_print function within lxml is adding proper indentation, as well as \n breaks as I would want, but it doesn't seem to be getting interpreted correctly during output; it write on a single line.
The output I'm trying to get is as follows:
<root>
<Article>
<Title>title1</Title>
<Summary>summary1</Summary>
<Source>source1</Source>
<Content>content1</Content>
</Article>
Ideally, I'd like for my output to be viewed as a valid XML document, and display in proper nested format.

Your "Current Output" is the representation (internal python representation) of the bytestring generated by etree.tostring(), and seems that in Python3 print(somebytestring) prints the representation instead of the actual string.
Hopefully the solution is quite simple: just pass the desired encoding to etree.tostring(), ie:
xml = etree.tostring(articles, encoding="unicode", pretty_print=True)
print(xml)

I've only used the base ET module in Python and can't find an lxml download for python 3.5 (which I'm on) in order to test it, but the b before the line indicates bytes and a quick glance at the documentation indicates that tostring() has an encoding keyword, so you should just need to set that to unicode or utf-8.
I'll also mention that you don't need to set "i" before your for-loop (python will create the "i" it needs for the for-loop), though I- personally- would zip the lists and iterate the items in the lists themselves (though that's not going to have any real impact on the code in this situation).

How can I open multiple XML documents stored in a single list?

This python list contains multiple XML files in a single LIST.
list_xml = ['hello.xml,'hi.xml','08333.xml']
I want to parse Hello.xml and store the data into another list. Once this is completed, I want to read hi.xml.
How can I do this?

See the pattern of the following code. This should help you.
Also, I'd advise you to go through some fundamentals of programming.
list_xml = ['a.xml', 'b.xml', ... ]
out = []
for xml in list_xml:
# Parse XML here
# more code
out.append(parsed_data)
return

Xml creation using ElementTree

I am new to python. I want to create a xml tree with one parent, several childs and several subchilds. I've stored child tags are in list 'TAG' and Subchild tags are in list 'SUB'
And i have came up with following code but i am not able to achieve the desired result !
def make_xml(tag,sub):
'''
Takes in two lists and Returns a XML object.
The first list has to contain all the tag objects
The Second list has to contain child data's
'''
from xml.etree.ElementTree import Element, SubElement, Comment, tostring
top = Element("Grand Parent")
comment = Comment('This is the ccode parse tree')
top.append(comment)
i=0
try:
for ee in tag:
child = SubElement(top, 'Tag'+str(i))
child.text = str(tag[i]).encode('utf-8',errors = 'ignore')
subchild = SubElement(child, 'Content'+str(i))
subchild.text = str(sub[i]).encode('utf-8',errors = 'ignore')
i = i+1;
except UnicodeDecodeError:
print 'oops'
return top
EDIT:
I have two lists like these:
TAG = ['HAPPY','GO','LUCKY']
SUB = ['ED','EDD','EDDY']
What i want is:
<G_parent>
<parent1>
HAPPY
<child1>
ED
<\child1>
<\parent1>
<parent2>
GO
<child2>
EDD
<\child2>
<\parent2>
<parent3>
LUCKY
<child3>
EDDY
<\child3
<\parent3>
<\G_parent>
The actual list has many more contents than this. I want to achieve using a for loop or so.
EDIT:
OOP's. My bad !
The code works as expected when i pass the example list. But in my real application the list is long. The list contains text fragments extracted from a pdf file. Somewhere in that text i get UnicodeDecodeError(reason: pdf extracted text messy. Proof: 'oops' get printed once ) and the returned xml object is incomplete.
So I need to figure out a way that even on UnicodeDecodeErrors my complete list is parsed. Is that possible ! I'm using .decode('utf-8',errors='ignore') even then the parsing does not complete !