I know the question is redundant but I tried all the Python code that I found and modified for my file but they did not work. I need to find a way to convert my file myData.csv in to a XML format file which can be read by a navigator.
I just started to learn Python this month so I'm a beginner. This is my code:
#! usr/bin/python
# -*- coding: utf-8 -*-
import csv, sys, os
from lxml import etree
csvFile = 'myData.csv' # création de la variable pour le fichier csv
reader= csv.reader(open(csvFile), delimiter=';', quoting=csv.QUOTE_NONE) # création d'une variable reader à qui on renvoie le tableau csv
print "<data>"
for record in reader:
if reader.line_num == 1:
header = record
else:
innerXml = ""
dontShow = False
type = ""
for i, field in enumerate(record):
innerXml += "<%s>" % header[i].lower() + field + "</%s>" % header[i].lower()
if i == 1 and field == "0":
type = "Next"
elif type == "" and i == 3 and field == "0":
type = "Next"
elif type == "" and i == 3 and field != "0":
type = "film"
if i == 1 and field == "X":
dontShow = True
if dontShow == False:
xml = "<%s>" % type
xml += innerXml
xml += "</%s>" % type
print xml
print "</data>"
Consider building your XML with dedicated DOM objects and not a concatenation of strings which you can do with the lxml module. Using methods such as Element(), SubElement(), etc. you can iteratively build XML tree from reading CSV data:
import csv
import lxml.etree as ET
headers = ['Titre', 'Realisateur', 'Date_Debut_Evenement', 'Date_Fin_Evenement', 'Cadre',
'Lieu', 'Adresse', 'Arrondissement', 'Adresse_complète', 'Geo_Coordinates']
# INITIALIZING XML FILE
root = ET.Element('root')
# READING CSV FILE AND BUILD TREE
with open('myData.csv') as f:
next(f) # SKIP HEADER
csvreader = csv.reader(f)
for row in csvreader:
data = ET.SubElement(root, "data")
for col in range(len(headers)):
node = ET.SubElement(data, headers[col]).text = str(row[col])
# SAVE XML TO FILE
tree_out = (ET.tostring(root, pretty_print=True, xml_declaration=True, encoding="UTF-8"))
# OUTPUTTING XML CONTENT TO FILE
with open('Output.xml', 'wb') as f:
f.write(tree_out)
Output
<?xml version='1.0' encoding='UTF-8'?>
<root>
<data>
<Titre>1</Titre>
<Realisateur>BUS PALLADIUM</Realisateur>
<Date_Debut_Evenement>CHRISTOPHER THOMPSON</Date_Debut_Evenement>
<Date_Fin_Evenement>21 mai 2009</Date_Fin_Evenement>
<Cadre>21 mai 2009</Cadre>
<Lieu>EXTERIEUR</Lieu>
<Adresse>PLACE</Adresse>
<Arrondissement>PIGALLE</Arrondissement>
<Adresse_complète>75018</Adresse_complète>
<Geo_Coordinates>PLACE PIGALLE 75018 Paris France</Geo_Coordinates>
</data>
<data>
<Titre>2</Titre>
<Realisateur>LES INVITES DE MON PERE</Realisateur>
<Date_Debut_Evenement>ANNE LE NY</Date_Debut_Evenement>
<Date_Fin_Evenement>20 mai 2009</Date_Fin_Evenement>
<Cadre>20 mai 2009</Cadre>
<Lieu>DOMAINE PUBLIC</Lieu>
<Adresse>SQUARE</Adresse>
<Arrondissement>DU CLIGNANCOURT</Arrondissement>
<Adresse_complète>75018</Adresse_complète>
<Geo_Coordinates>SQUARE DU CLIGNANCOURT 75018 Paris France</Geo_Coordinates>
</data>
<data>
<Titre>3</Titre>
<Realisateur>DEMAIN, A L'AUBE</Realisateur>
<Date_Debut_Evenement>GAEL CABOUAT</Date_Debut_Evenement>
<Date_Fin_Evenement>17 avril 2009</Date_Fin_Evenement>
<Cadre>17 avril 2009</Cadre>
<Lieu>EXTERIEUR</Lieu>
<Adresse>RUE</Adresse>
<Arrondissement>QUINCAMPOIX</Arrondissement>
<Adresse_complète>75004</Adresse_complète>
<Geo_Coordinates>RUE QUINCAMPOIX 75004 Paris France</Geo_Coordinates>
</data>
...
(posted as an answer so I can show a code block)
There are a lot of picky details when writing XML. In Python, you should probably use some version of ElementTree to help with that. One good tutorial is Creating XML Documents. Quoting from there:
from xml.etree.ElementTree import Element, SubElement, Comment, tostring
top = Element('top')
comment = Comment('Generated for PyMOTW')
top.append(comment)
child = SubElement(top, 'child')
child.text = 'This child contains text.'
child_with_tail = SubElement(top, 'child_with_tail')
child_with_tail.text = 'This child has regular text.'
child_with_tail.tail = 'And "tail" text.'
child_with_entity_ref = SubElement(top, 'child_with_entity_ref')
child_with_entity_ref.text = 'This & that'
print(tostring(top))
If you use this as an example of how to create a tree of XML elements, you should be able to translate your code into the XML structure you need.
Importing pandas and saving file name:
import pandas as pd
csvFile = 'myData.csv'
The following will read CSV into a pandas data frame, then convert to XML.
df = pd.read_csv(path)
df_xml = df.to_xml()
The below code will create a new file and then save the XML data to a file named "csv2xml"
f = open("csv2xml.xml", "w")
f.write(df_xml)
f.close()
Related
I wrote a script, based on some of the existing StackOverflow questions, but no one perfectly fit my issues.
The user use xpath to find a XML tag from a given XML file, and update the tag text based on user inputs.
Below is my script using Python 3 (The most difficult part for me is around the namespaces):
import xml.etree.ElementTree as ET
import sys
# user inputs and variables
filename = 'actors.xml'
xpath = 'actor/name'
value = 'test name'
temp_namespace = 'temp_namespace'
# get all namespaces
all_namespaces = dict([node for _, node in ET.iterparse(filename, events=['start-ns'])])
# register namespace
for key in all_namespaces.keys():
ET.register_namespace(key, all_namespaces[key])
# remove all namespace from elements first
# and temp save it to tag attribute
# The below logic is copied from other Stackoverflow answers
# From **Python 3.8**, we can add the parser to insert comments
it = ET.iterparse(filename, parser=ET.XMLParser(target=ET.TreeBuilder(insert_comments=True)))
for _, el in it:
prefix, has_namespace, postfix = el.tag.partition('}')
if has_namespace:
el.tag = postfix
el.set(temp_namespace, prefix + has_namespace)
# find and update
root = it.root
for el in root.findall(xpath):
el.text = str(value)
# get xml comments before root level
doc_comments = []
with open(filename, 'r') as f:
lines = f.readlines()
for line in lines:
if line.startswith('<?xml'):
continue
if line.startswith('<' + root.tag):
break
else:
doc_comments.append(line)
def add_tag_namespace(el):
for sub_el in el:
if temp_namespace in sub_el.attrib.keys():
sub_el.tag = sub_el.attrib[temp_namespace] + sub_el.tag
del sub_el.attrib[temp_namespace]
add_tag_namespace(sub_el)
if temp_namespace in el.attrib.keys():
el.tag = el.attrib[temp_namespace] + el.tag
del el.attrib[temp_namespace]
# add all namespace back
# and delete the temp namespace attribute
add_tag_namespace(root)
# write back to xml file
tree = ET.ElementTree(root)
tree.write(filename, encoding='unicode', xml_declaration=True)
if len(doc_comments) == 0:
sys.exit()
# write xml comments before root back
lines = []
# first read all lines
with open(filename, 'r') as f:
lines = f.readlines()
# second, insert xml comments back into memory
for i, line in enumerate(lines):
if line.startswith('<?xml'):
insert_at = i + 1
for comment in doc_comments:
lines.insert(insert_at, comment)
insert_at += 1
break
# finally, write all contents to file
with open(filename, 'w') as f:
for line in lines:
f.write(line)
actors.xml:
<?xml version="1.0"?>
<actors xmlns:fictional="http://characters.example.com"
xmlns="http://people.example.com">
<actor>
<name>John Cleese</name>
<fictional:character>Lancelot</fictional:character>
<fictional:character>Archie Leach</fictional:character>
</actor>
<actor>
<name>Eric Idle</name>
<fictional:character>Sir Robin</fictional:character>
<fictional:character>Gunther</fictional:character>
<fictional:character>Commander Clement</fictional:character>
</actor>
</actors>
My input json file looks something like this
{"id":1,"author":"abc","title":"xyz"}
{"id":2,"author":"def","title":"mno"}
I want a python script to create a xml file that looks like this
<sequence>
<id>1</id>
<author>abc</author>
<title>xyz</title>
</sequence>
<sequence>
<id>2</id>
<author>def</author>
<title>mno</title>
</sequence>
Right now this is the code I'm using
import json as j
with open("test.json") as json_format_file:
d = j.load(json_format_file)
import xml.etree.cElementTree as e
r = e.Element("sequence")
e.SubElement(r,"id").text = d["id"]
e.SubElement(r,"submitter").text = d["submitter"]
e.SubElement(r,"authors").text = str(d["authors"])
e.SubElement(r,"title").text = str(d["title"])
a = e.ElementTree(r)
a.write("json_to_xml.xml")
Problem is that it only works for 1 entry, and if i have more than 1 entry in the JSON file, it throws an error. How can I make this run for multiple entries and write it all into the xml file?
EDIT:
Have changed my JSON file to look like this
[{"id":1,"author":"abc","title":"xyz"},
{"id":2,"author":"def","title":"mno"}]
json2xml is one of the powerful python library. Try this one.
json.json
[{"id":1,"author":"abc","title":"xyz"}, {"id":2,"author":"def","title":"mno"}]
Python
from json2xml import json2xml
import lxml.etree as ET
import json
with open('json.json') as data_file:
data = json.load(data_file)
json2xml = json2xml.Json2xml(data, wrapper="sequence", pretty=True, attr_type=False).to_xml()
with open('json2xml.xml', 'w') as f:
f.write(json2xml)
## modifie xml data
tree = ET.parse('json2xml.xml')
root = tree.getroot()
for item in root.findall('item'):
for child in item:
root.append(child)
root.remove(item)
new_xml = ET.tostring(root, pretty_print=True, xml_declaration=True, encoding="UTF-8")
with open('file.xml', 'wb' ) as final:
final.write(new_xml)
Can you modify the json data, make it into an array?
{
"updated": "2020-07-09",
"list": [
{"id":1,"author":"abc","title":"xyz"},
{"id":2,"author":"def","title":"mno"}
]
}
import json as j
with open('data.json') as data_file:
data = json.load(data_file)
for obj in data['list']:
r = e.Element("sequence")
e.SubElement(r,"id").text = obj["list"]["id"]
e.SubElement(r,"submitter").text = obj["list"]["submitter"]
e.SubElement(r,"authors").text = str(obj["list"]["authors"])
e.SubElement(r,"title").text = str(obj["list"]["title"])
a = e.ElementTree(r)
a.write("json_to_xml.xml")
I'm using python to make a program which will have to write data in a XML tag of a specific file.
The line of data I'm willing to write is the following.
<Stream>XXXX-XXXX-XXXX-XXXX?p=0</Stream><URL>rtmp://a.rtmp.youtube.com/live2</URL>
But what I get in my XML file after writing is pretty different.
<Stream>XXXX-XXXX-XXXX-XXXX?p=0</Stream><URL>rtmp://a.rtmp.youtube.com/live2</URL>
The < and > are here for purpose, and are NOT < and >. I need to keep this formatting but when I use the export as xml file, it replaces all the & by &
I use this code to write data in the xml file:
from lxml import etree as ET
Name_with_single_quote= """IF [Calculation_1] = 'Day-1' THEN [begintime] + 1
ELSEIF[Calculation_1] < 'Day-2' THEN [begintime] + 2
ELSEIF [Calculation_1] > "Day-3" THEN [begintime] + 3
ELSE [begintime]
END"""
Name_with_single_quote = Name_with_single_quote.replace("\n", "
").replace("<", "<").replace("'", "'").replace(">",">").replace("\"", """)
Name_with_single_quote = str(Name_with_single_quote)
xml = """<?xml version="1.0"?>
<column role="dimension" type="nominal" name="[Calculation_1]" datatype="boolean" caption="">
<calculation formula=""/>
</column>"""
tree = ET.fromstring(xml)
formula = tree.find('.//calculation')
formula.set('formula', Name_with_single_quote)
from xml.dom import minidom
xmlstr = minidom.parseString(ET.tostring(tree)).toprettyxml()
xmlstr = '\n'.join(list(filter(lambda x: len(x.strip()), xmlstr.split('\n'))))
with open('test_for_esc_result.xml', "w") as f:
f.write(xmlstr)
I have an XML file like that and trying to convert it to CSV with xml2csv python library. But there is a < images > image tag that brokes everything. I want to get all < img_item > tags on different column. How can I achieve that?
Thanks,
<products>
<product>
<code>722</code>
<ws_code>B515C16CRU</ws_code>
<supplier_code>B515C16CRU</supplier_code>
<images>
<img_item type_name="">
https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3378-72-B.jpg
</img_item>
<img_item type_name="">
https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3379-72-B.jpg
</img_item>
<img_item type_name="">
https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3380-72-B.jpg
</img_item>
</images>
</product>
....
</products>
As you might have guessed, the problem is because each product node has multiple img_item tags which xml2csv does not know how to handle (and, going over its documentation, does not seem to have an option to let it know how to handle these nodes).
You can, however, do this quite easily using the builtin csv module. You just need to decide how you want to delimit the different images' urls. In the example below I've decided to use ; (obviously you can't use ,, unless you use another delimiter for the columns).
Also note that I hardcoded the headers. This can be (quite) easily changed so that the headers are dynamically detected from the product node's sub-elements.
import csv
import xml.etree.ElementTree as ET
string = '''<products>
<product>
<code>722</code>
<ws_code>B515C16CRU</ws_code>
<supplier_code>B515C16CRU</supplier_code>
<images>
<img_item type_name="">https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3378-72-B.jpg</img_item>
<img_item type_name="">https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3379-72-B.jpg</img_item>
<img_item type_name="">https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3380-72-B.jpg</img_item>
</images>
</product>
</products>'''
root = ET.fromstring(string)
headers = ('code', 'ws_code', 'supplier_code', 'images')
with open('test.csv', 'w', newline='') as f:
writer = csv.DictWriter(f, fieldnames=headers)
writer.writeheader()
for product in root.iter('product'):
writer.writerow({'code': product.find('code').text,
'ws_code': product.find('ws_code').text,
'supplier_code': product.find('supplier_code').text,
'images': ';'.join(img.text for img in product.iter('img_item'))})
Which produces the below CSV:
code,ws_code,supplier_code,images
722,B515C16CRU,B515C16CRU,https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3378-72-B.jpg;https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3379-72-B.jpg;https://www.apparel.com.tr/stance-corap-cruker-grey-orap-stance-ankle-bters-3380-72-B.jpg
import xml.etree.ElementTree as ET
import csv
import re
class xml_to_csv:
def do(self):
#self.xml_file_location = input("Enter full path of XML file(Eg = D:\programs\ResidentData.xml) : ")
self.tree = ET.parse("urunler-fotolu.xml")
self.root = self.tree.getroot()
self.csv_file_location = input("Enter full path to store CSV file(Eg = D:\programs\csv_file.csv ) : ")
self.csv_data = open(self.csv_file_location, 'w')
self.csv_writer = csv.writer(self.csv_data)
self.find_records(self.root)
def find_attributes(self,record):
temp = []
dont_do = 0
for j in record:
temp = temp + self.find_attributes(j)
dont_do = 1
if(dont_do == 0):
return [record.text]
return temp
def find_records(self,root1):
for i in root1:
csv_record = self.find_attributes(i)
sz = len(csv_record)
i=0
while (i<sz):
if csv_record[i][0] == '\n':
csv_record[i] = csv_record[i][1:len(csv_record[i])-1]
i = i+1;
print(csv_record)
self.csv_writer.writerow(csv_record)
if __name__ == "__main__":
obj = xml_to_csv()
obj.do()
Input:
For this = """
<State>
<Resident Id="100">
<Name>Sample Name</Name>
<PhoneNumber>1234567891</PhoneNumber>
<EmailAddress>sample_name#example.com</EmailAddress
<Address>
<StreetLine1>Street Line1</StreetLine1>
<City>City Name</City>
<StateCode>AE</StateCode>
<PostalCode>12345</PostalCode>
</Address>
</Resident>
</State>
"""
Output :
['Sample Name', '1234567891', 'sample_name#example.com', 'Street Line1', 'City Name', 'AE', '12345']
I need a python script for extract data from xml file
I have a xml file as shoen below:
<software>
<name>Update Image</name>
<Build>22.02</Build>
<description>Firmware for Delta-M Series </description>
<CommonImages> </CommonImages>
<ModelBasedImages>
<ULT>
<CNTRL_0>
<file type="UI_APP" ver="2.35" crc="1234"/>
<file type="MainFW" ver="5.01" crc="5678"/>
<SIZE300>
<file type="ParamTableDB" ver="1.1.4" crc="9101"/>
</SIZE300>
</CNTRL_0>
<CNTRL_2>
<file type="UI_APP" ver="2.35" crc="1234"/>
<file type="MainFW" ver="5.01" crc="9158"/>
</CNTRL_2>
</ULT>
</ModelBasedImages>
</software>
I want the data in table format like:
type ver crc
UI_APP 2.35 1234
MainFW 5.01 5678
ParamTableDB 1.1.4 9101
UI_APP 2.35 1234
MainFW 5.01 9158
Extract into any type of file csv/doc....
I tried this code:
import xml.etree.ElementTree as ET
import csv
tree = ET.parse("Build_40.01 (copy).xml")
root = tree.getroot()
# open a file for writing
Resident_data = open('ResidentData.csv', 'w')
# create the csv writer object
csvwriter = csv.writer(Resident_data)
resident_head = []
count = 0
for member in root.findall('file'):
resident = []
address_list = []
if count == 0:
name = member.find('type').tag
resident_head.append(name)
ver = member.find('ver').tag
resident_head.append(ver)
crc = member.find('crc').tag
resident_head.append(crc)
csvwriter.writerow(resident_head)
count = count + 1
name = member.find('type').text
resident.append(name)
ver = member.find('ver').text
resident.append(ver)
crc = member.find('crc').text
resident.append(crc)
csvwriter.writerow(resident)
Resident_data.close()
Thanks in advance
edited:xml code updated.
Use the xpath expression .//file to find all <file> elements in the XML document, and then use each element's attributes to populate the CSV file through a csv.DictWriter:
import csv
import xml.etree.ElementTree as ET
tree = ET.parse("Build_40.01 (copy).xml")
root = tree.getroot()
with open('ResidentData.csv', 'w') as f:
w = csv.DictWriter(f, fieldnames=('type', 'ver', 'crc'))
w.writerheader()
w.writerows(e.attrib for e in root.findall('.//file'))
For your sample input the output CSV file will look like this:
type,ver,crc
UI_APP,2.35,1234
MainFW,5.01,5678
ParamTableDB,1.1.4,9101
UI_APP,2.35,1234
MainFW,5.01,9158
which uses the default delimiter (comma) for a CSV file. You can change the delimiter using the delimiter=' ' option to DictWriter(), however, you will not be able to obtain the same formatting as your sample output, which appears to use fixed width fields (but you might get away with using tab as the delimiter).