Develop Reference

read the text of a file between 2 words in python

I am trying to open, read and extract the content (fragment) that is between 2 words (which are opening and closing profile, also included) of an .xml locating the fragment by means of a keyword that I introduce and write only that fragment (between 2 tags) in another new .xml that I generate.
Currently the python script that I have allows me to open, read the source .xml file, search for the keyword that I introduce in the text and return those complete lines where the keyword is found by writing them in a new .xml file that I generate as follows:
keyword = 'Georgia'
occurrences = []
with open('test_input.xml') as lines:
for line in lines:
if keyword in line:
occurrences.append(line)
archi1=open("test_output.xml","w")
archi1.write(''.join(occurrences))
archi1.close()
The result I get is a "test_output.xml" file that contains the following:
<id>Georgia-1</id>
<profile>Georgia-p1</profile>
<id>Georgia-2</id>
<profile>Georgia-p2</profile>
And the problem is that I not only need it to return the complete lines that contain the keyword (in this case 'Georgia') but also the entire fragment that contains those two words and that is delimited between the opening and the closing of the word or tag 'profile', that is, I need it to return the following result:
<profile>
<id>Georgia-1</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>Georgia-p1</profile>
<showtitle>Georgia_s1</showtitle>
<ip>000.000.0.3</ip>
<port>00003</port>
<persistencePort>00033</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_3</webstart.server.name>
<codebaseProtocolServer>T3</codebaseProtocolServer>
</properties>
</profile>
<profile>
<id>Georgia-2</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>Georgia-p2</profile>
<showtitle>Georgia_s2</showtitle>
<ip>000.000.0.4</ip>
<port>00004</port>
<persistencePort>00044</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_4</webstart.server.name>
<codebaseProtocolServer>T4</codebaseProtocolServer>
</properties>
</profile>
The full source .xml I am using is as follows:
<project>
<profile>
<id>Azerbaiyan-1</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>Azerbaiyan-p1</profile>
<showtitle>Azerbaiyan_s1</showtitle>
<ip>000.000.0.1</ip>
<port>00001</port>
<persistencePort>00011</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_1</webstart.server.name>
<codebaseProtocolServer>T1</codebaseProtocolServer>
</properties>
</profile>
<profile>
<id>Azerbaiyan-2</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>Azerbaiyan-p2</profile>
<showtitle>Azerbaiyan_s2</showtitle>
<ip>000.000.0.2</ip>
<port>00002</port>
<persistencePort>00022</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_2</webstart.server.name>
<codebaseProtocolServer>T2</codebaseProtocolServer>
</properties>
</profile>
<profile>
<id>Georgia-1</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>Georgia-p1</profile>
<showtitle>Georgia_s1</showtitle>
<ip>000.000.0.3</ip>
<port>00003</port>
<persistencePort>00033</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_3</webstart.server.name>
<codebaseProtocolServer>T3</codebaseProtocolServer>
</properties>
</profile>
<profile>
<id>Georgia-2</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>Georgia-p2</profile>
<showtitle>Georgia_s2</showtitle>
<ip>000.000.0.4</ip>
<port>00004</port>
<persistencePort>00044</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_4</webstart.server.name>
<codebaseProtocolServer>T4</codebaseProtocolServer>
</properties>
</profile>
<profile>
<id>USA-1</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>USA-p1</profile>
<showtitle>USA1_s1</showtitle>
<ip>000.000.0.5</ip>
<port>00005</port>
<persistencePort>00055</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_5</webstart.server.name>
<codebaseProtocolServer>T5</codebaseProtocolServer>
</properties>
</profile>
<profile>
<id>USA-2</id>
<activation>
<activeByDefault>false</activeByDefault>
</activation>
<properties>
<profile>USA-p2</profile>
<showtitle>USA1_s2</showtitle>
<ip>000.000.0.6</ip>
<port>00006</port>
<persistencePort>00066</persistencePort>
<defaultLocale>en_GB</defaultLocale>
<webstart.server.name>host_6</webstart.server.name>
<codebaseProtocolServer>T6</codebaseProtocolServer>
</properties>
</profile>

Parse the input as XML and capture the profile elements that have an id child element whose text value contains the string "Georgia".
The following program uses the ElementTree standard library and outputs the wanted result:
import xml.etree.ElementTree as ET
tree = ET.parse("input.xml")
# Iterate over all 'profile' elements
for profile in tree.findall("profile"):
id = profile.find("id").text
if "Georgia" in id:
print(ET.tostring(profile).decode())

Python lxml does not support xpath syntax 'starts-with'?

str = """<ROOT>
<ITEM>
<REVENUE_YEAR>2554-02</REVENUE_YEAR>
<REGION>Central</REGION>
</ITEM>
<ITEM>
<REVENUE_YEAR>2552-02</REVENUE_YEAR>
<REGION>Central</REGION>
</ITEM>
<ITEM>
<REVENUE_YEAR>2552-03</REVENUE_YEAR>
<REGION>Central</REGION>
</ITEM>
</ROOT>"""
xml = etree.fromstring(str)
xpath_str = ".//ITEM[starts-with(REVENUE_YEAR,'2554')]"
result = xml.find(xpath_str)
print(result)
Hi, the code above raised SyntaxError: invalid predicate, does it mean lxml do not support starts-with? Any other way to locate the REVENUE_YEAR element(2554-02) by xpath with lxml? Thanks!

It supports xpath but you need to use xpath:
str = """<ROOT>
<ITEM>
<REVENUE_YEAR>2554-02</REVENUE_YEAR>
<REGION>Central</REGION>
</ITEM>
<ITEM>
<REVENUE_YEAR>2552-02</REVENUE_YEAR>
<REGION>Central</REGION>
</ITEM>
<ITEM>
<REVENUE_YEAR>2552-03</REVENUE_YEAR>
<REGION>Central</REGION>
</ITEM>
</ROOT>"""
xml = etree.fromstring(str)
xpath_str = ".//ITEM[starts-with(REVENUE_YEAR,'2554')]"
result = xml.xpath(xpath_str)
print(result) # which is a list containing only one element

Quick way to Upper every value in xml?

I have the following xml:
<Item>
<Platform>itunes</Platform>
<PlatformID>102224185</PlatformID>
<Info>
<LanguageOfMetadata>EN</LanguageOfMetadata>
<Name>Commander In Chief</Name>
<Studio>abc</Studio>
</Info>
<Type>TVSeries</Type>
</Item>
What would be the quickest way to UPPER all the values? For example:
<Item>
<Platform>ITUNES</Platform>
<PlatformID>102224185</PlatformID>
<Info>
<LanguageOfMetadata>EN</LanguageOfMetadata>
<Name>COMMANDER IN CHIEF</Name>
<Studio>ABC</Studio>
</Info>
<Type>TVSERIES</Type>
</Item>

You can find all elements and call upper() on each element's text:
import lxml.etree as ET
data = """<Item>
<Platform>itunes</Platform>
<PlatformID>102224185</PlatformID>
<Info>
<LanguageOfMetadata>EN</LanguageOfMetadata>
<Name>Commander In Chief</Name>
<Studio>abc</Studio>
</Info>
<Type>TVSeries</Type>
</Item>
"""
root = ET.fromstring(data)
for elm in root.xpath("//*"): # //* would find all elements recursively
elm.text = elm.text.upper()
print(ET.tostring(root))
Prints:
<Item>
<Platform>ITUNES</Platform>
<PlatformID>102224185</PlatformID>
<Info>
<LanguageOfMetadata>EN</LanguageOfMetadata>
<Name>COMMANDER IN CHIEF</Name>
<Studio>ABC</Studio>
</Info>
<Type>TVSERIES</Type>
</Item>
This though does not cover cases when you, for example, have a tail of an element - e.g. have <Studio>ABC</Studio>test instead of just <Studio>ABC</Studio>. To support that as well, put the following under the for loop as well:
elm.tail = elm.tail.upper() if elm.tail else None

Here is a way to upper everything, though note that this will include the tags as well:
node = etree.fromstring(etree.tostring(item).upper())
print etree.tostring(node, pretty_print=True)
<ITEM>
<PLATFORM>ITUNES</PLATFORM>
<PLATFORMID>102224185</PLATFORMID>
<INFO>
<LANGUAGEOFMETADATA>EN</LANGUAGEOFMETADATA>
<NAME>COMMANDER IN CHIEF</NAME>
<STUDIO>ABC</STUDIO>
</INFO>
<TYPE>TVSERIES</TYPE>
</ITEM>

Assuming you can parse the XML file you can just rewrite the contents using the .upper() function that is built into python for strings. You can call it like that:
"mystring".upper().

Large XML parsing in Python

I am a novice in python and have the following task on hand.
I have a large xml file like the one below:
<Configuration>
<Parameters>
<Component Name='ABC'>
<Group Name='DEF'>
<Parameter Name='GHI'>
<Description>
Some Text
</Description>
<Type>Integer</Type>
<Restriction>
<Level>5</Level>
</Restriction>
<Value>
<Item Value='5'/>
</Value>
</Parameter>
<Parameter Name='JKL'>
<Description>
Some Text
</Description>
<Type>Integer</Type>
<Restriction>
<Level>5</Level>
</Restriction>
<Value>
<Item Value='5'/>
</Value>
</Parameter>
</Group>
<Group Name='MNO'>
<Parameter Name='PQR'>
<Description>
Some Text
</Description>
<Type>Integer</Type>
<Restriction>
<Level>5</Level>
</Restriction>
<Value>
<Item Value='5'/>
</Value>
</Parameter>
<Parameter Name='TUV'>
<Description>
Some Text
</Description>
<Type>Integer</Type>
<Restriction>
<Level>5</Level>
</Restriction>
<Value>
<Item Value='5'/>
</Value>
</Parameter>
</Group>
</Component>
</Parameters>
</Configuration>
In this xml file I have to parse through the component "ABC" go to group "MNO" and then to the parameter "TUV" and under this I have to change the item value to 10.
I have tried using xml.etree.cElementTree but to no use. And lxml dosent support on the server as its running a very old version of python. And I have no permissions to upgrade the version
I have been using the following code to parse and edit a relatively small xml:
def fnXMLModification(ArgStr):
argList = ArgStr.split()
strXMLPath = argList[0]
if not os.path.exists(strXMLPath):
fnlogs("XML File: " + strXMLPath + " does not exist.\n")
return False
try:
import xml.etree.cElementTree as ET
except ImportError:
import xml.etree.ElementTree as ET
f=open(strXMLPath, 'rt')
tree = ET.parse(f)
ValueSetFlag = False
AttrSetFlag = False
for strXPath in argList[1:]:
strXPathList = strXPath.split("[")
sxPath = strXPathList[0]
if len(strXPathList)==3:
# both present
AttrSetFlag = True
ValueSetFlag = True
valToBeSet = strXPathList[1].strip("]")
sAttr = strXPathList[2].strip("]")
attrList = sAttr.split(",")
elif len(strXPathList) == 2:
#anyone present
if "=" in strXPathList[1]:
AttrSetFlag = True
sAttr = strXPathList[1].strip("]")
attrList = sAttr.split(",")
else:
ValueSetFlag = True
valToBeSet = strXPathList[1].strip("]")
node = tree.find(sxPath)
if AttrSetFlag:
for att in attrList:
slist = att.split("=")
node.set(slist[0].strip(),slist[1].strip())
if ValueSetFlag:
node.text = valToBeSet
tree.write(strXMLPath)
fnlogs("XML File: " + strXMLPath + " has been modified successfully.\n")
return True
Using this function I am not able to traverse the current xml as it has lot of children attributes or sub groups.

import statement
import xml.etree.cElementTree as ET
Parse content by fromstring method.
root = ET.fromstring(data)
Iterate according our requirement and get target Item tag and change value of Value attribute
for component_tag in root.iter("Component"):
if "Name" in component_tag.attrib and component_tag.attrib['Name']=='ABC':
for group_tag in component_tag.iter("Group"):
if "Name" in group_tag.attrib and group_tag.attrib['Name']=='MNO':
#for value_tag in group_tag.iter("Value"):
for item_tag in group_tag.findall("Parameter[#Name='TUV']/Value/Item"):
item_tag.attrib["Value"] = "10"
We can use Xpath to get target Item tag
for item_tag in root.findall("Parameters/Component[#Name='ABC']/Group[#Name='MNO']/Parameter[#Name='TUV']/Value/Item"):
item_tag.attrib["Value"] = "10"
Use tostring method to get content.
data = ET.tostring(root)

Modify XML file using ElementTree

I am trying to do the folowing with Python:
get "price" value and change it
find "price_qty" and insert new line with new tier and different price based on the "price".
so far I could only find the price and change it and insert line in about correct place but I can't find a way how to get there "item" and "qty" and "price" attributes, nothing has worked so far...
this is my original xml:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<body start="20.04.2014 10:02:60">
<pricelist>
<item>
<name>LEO - red pen</name>
<price>31,4</price>
<price_snc>0</price_snc>
<price_ao>0</price_ao>
<price_qty>
<item qty="150" price="28.20" />
<item qty="750" price="26.80" />
<item qty="1500" price="25.60" />
</price_qty>
<stock>50</stock>
</item>
</pricelist>
the new xml should look this way:
<pricelist>
<item>
<name>LEO - red pen</name>
<price>31,4</price>
<price_snc>0</price_snc>
<price_ao>0</price_ao>
<price_qty>
<item qty="10" price="31.20" /> **-this is the new line**
<item qty="150" price="28.20" />
<item qty="750" price="26.80" />
<item qty="1500" price="25.60" />
</price_qty>
<stock>50</stock>
</item>
</pricelist>
my code so far:
import xml.etree.cElementTree as ET
from xml.etree.ElementTree import Element, SubElement
tree = ET.ElementTree(file='pricelist.xml')
root = tree.getroot()
pos=0
# price - raise the main price and insert new tier
for elem in tree.iterfind('pricelist/item/price'):
price = elem.text
newprice = (float(price.replace(",", ".")))*1.2
newtier = "NEW TIER"
SubElement(root[0][pos][5], newtier)
pos+=1
tree.write('pricelist.xml', "UTF-8")
result:
...
<price_qty>
<item price="28.20" qty="150" />
<item price="26.80" qty="750" />
<item price="25.60" qty="1500" />
<NEW TIER /></price_qty>
thank you for any help.

Don't use fixed indexing. You already have the item element, so why don't use it?
tree = ET.ElementTree(file='pricelist.xml')
root = tree.getroot()
for elem in tree.iterfind('pricelist/item'):
price = elem.findtext('price')
newprice = float(price.replace(",", ".")) * 1.2
newtier = ET.Element("item", qty="10", price="%.2f" % newprice)
elem.find('price_qty').insert(0, newtier)
tree.write('pricelist.xml', "UTF-8")