Python parse JSON object and ask for selection of values - python

I am new to programming in general. I am trying to make a Python script that helps with making part numbers. Here, for an example, computer memory modules.
I have a python script that needs to read the bmt object from a JSON file. Then ask the user to select from it then append the value to a string.
{"common": {
"bmt": {
"DR1": "DDR",
"DR2": "DDR2",
"DR3": "DDR3",
"DR4": "DDR4",
"DR5": "DDR5",
"DR6": "DDR6",
"FER": "FeRAM",
"GD1": "GDDR",
"GD2": "GDDR2",
"GD3": "GDDR3",
"GD4": "GDDR4",
"GD5": "GDDR5",
"GX5": "GDDR5X",
"GD6": "GDDR6",
"GX6": "GDDR6X",
"LP1": "LPDDR",
"LP2": "LPDDR2",
"LP3": "LPDDR3",
"LP4": "LPDDR4",
"LX4": "LPDDR4X",
"LP5": "LPDDR5",
"MLP": "mDDR",
"OPX": "Intel Optane/Micron 3D XPoint",
"SRM": "SRAM",
"WRM": "WRAM"
},
"modtype": {
"CM": "Custom Module",
"DD": "DIMM",
"MD": "MicroDIMM",
"SD": "SODIMM",
"SM": "SIMM",
"SP": "SIPP",
"UD": "UniDIMM"
}
}}
Example: There is a string called "code" that already has the value "MMD". The script asks the user what to select from the listed values, (e.g. "DR1"). If a selection is made (user enters "DR1", it appends that value to the string, the new value would be "MMDDR1".
This code is to print the JSON. This is how far I have gotten
def enc(code):
memdjson = json.loads("memd.json")
print(memdjson)
How do I do this?
Repo with the rest of the code is here: https://github.com/CrazyblocksTechnologies/PyCIPN


Try:
import json
def enc(code):
memdjson = json.load(open("memd.json"))["common"]["bmt"]
selected = input("Select a value from the following: \n{}\n\n".format(' '.join(memdjson.keys())))
return code+memdjson[selected]


import json
import inquirer
with open(path_to_json_file) as f:
file = json.load(f)
code = "MMD"
questions = [
inquirer.List('input',
message="Select one of the following:",
choices=list(file["common"]["bmt"].keys()),
),
]
answer = inquirer.prompt(questions)
code += answer

Related

Python - search dict value from list of strings

I have a list of strings that represent specific error codes my code needs to filter out.
DMS_WARNING_STOP_REASONS = ["Stop Reason RECOVERABLE_ERROR",
"Stop Reason FATAL_ERROR",
"Stop Reason STOPPED_DUE_TO_LOW_MEMORY",
"Stop Reason STOPPED_DUE_TO_LOW_DISK",
"Stop Reason FAILED"
]
StopReason field from api's response sometimes contains them nested within longer string.
dict_response = {'name': 'job_name',
'Status': 'Stopped',
'StopReason': "terminated [reptask/replicationtask.c:2868] [1020101]\n Stop Reason FATAL_ERROR Error Level FATAL"}
I tried using find and other methods but don't know how to achieve following behaviour:
Check if any value from DMS_WARNING_STOP_REASONS exists/is part of StopReason field.
Line that should perform this search is: if x.get("StopReason") in self.DMS_WARNING_STOP_REASONS
def process_tasks(self, all_tasks: list) -> dict:
result = {}
for env in self.envs:
tasks = [x for x in all_tasks if env in self.__get_task_id(x)]
warnings = [
{"name": self.__get_task_id(x), "Status": x["Status"],"StopReason": x.get("StopReason", "Stop Reason FAILED")}
for x in tasks
if x.get("StopReason") in self.DMS_WARNING_STOP_REASONS
]
working = [...
]
result[env] = {...
}
return result

I had to re-use.get() as it modifies the dict in-fly without inserting the key into the dict.
Working solution based of #ILS code:
if (any(stop_reason in x.get("StopReason", "Stop Reason FAILED")

Python dict overwrite value from a copy variable

I'm new on Python. I tried to write a script to read data from excel and post data as json to an API. Python is really different with Java. I have some questions can not understand.
The problem is I create a variable that is a copy from another one. When I give the value to the variable, the original variable also changed. Can anyone tell me why? And how to fix that. Thank you.
I have a APIContent.py as below:
APIContent = {
"partnerIdentifier":"",
"test":True,
"key":"",
"interchange":{
"dateCreated":"",
#invoices
"transactionSet":[],
"controlNumber":""
}
}
ContentInvoice = {
"detail":{
"shipToAddress":{
"zipCode":"",
"nameSecondary":"",
"city":"",
"streetPrimary":"",
"streetSecondary":"",
"countryCode":"",
"stateCode":"",
"namePrimary":""
},
"termsDiscountPercent":0,
"miscellaneousFees":0,
"freight":0,
"creditMemo":False,
"tax":0,
"termsDiscountAmount":0,
"subTotal":0,
"invoiceDate":"",
"invoiceDueDate":"",
"billToAddress":{
"zipCode":"",
"nameSecondary":"",
"city":"",
"streetPrimary":"",
"streetSecondary":"",
"countryCode":"",
"stateCode":"",
"namePrimary":""
},
"total":0,
"invoiceNumber":"",
"poNumber":"",
#items
"items":[]
},
"controlNumber":""
}
InvoiceItem = {
"unitPrice":0,
"vendorItemNumber":"",
"total":0,
"quantity":0,
"quantityMeasurement":"Case",
"discount":0,
"itemDescription":""
}
In a APIPoster.py. I created a copy var(contentInvoice) from APIContent.ContentInvoice. When I give values to contentInvoice. The APIContent.ContentInvoice also changed. So the next loop will create a contentInvoice with some values. The debug info please see image.
contentInvoice = APIContent.ContentInvoice.copy()
invoiceItem = APIContent.InvoiceItem.copy()
contentInvoice['detail']['invoiceNumber'] = row[0].value
contentInvoice['detail']['poNumber'] = row[1].value
Debug

Python How to add nested fields to Yaml file

I need to modify a YAML file and add several fields.I am using the ruamel.yaml package.
First I load the YAML file:
data = yaml.load(file_name)
I can easily add new simple fields, like-
data['prop1'] = "value1"
The problem I face is that I need to add a nested dictionary incorporate with array:
prop2:
prop3:
- prop4:
prop5: "Some title"
prop6: "Some more data"
I tried to define-
record_to_add = dict(prop2 = dict(prop3 = ['prop4']))
This is working, but when I try to add beneath it prop5 it fails-
record_to_add = dict(prop2 = dict(prop3 = ['prop4'= dict(prop5 = "Value")]))
I get
SyntaxError: expression cannot contain assignment, perhaps you meant "=="?
What am I doing wrong?

The problem has little to do with ruamel.yaml. This:
['prop4'= dict(prop5 = "Value")]
is invalid Python as a list ([ ]) expects comma separated values. You would need to use something like:
record_to_add = dict(prop2 = dict(prop3 = dict(prop4= [dict(prop5 = "Some title"), dict(prop6='Some more data'),])))
As your program is incomplete I am not sure if you are using the old API or not. Make sure to use
import ruamel.yaml
yaml = ruamel.yaml.YAML()
and not
import ruamel.yaml as yaml

Its because of having ['prop4'= <> ].Instead record_to_add = dict(prop2 = dict(prop3 = [dict(prop4 = dict(prop5 = "Value"))])) should work.
Another alternate would be,
import yaml
data = {
"prop1": {
"prop3":
[{ "prop4":
{
"prop5": "some title",
"prop6": "some more data"
}
}]
}
}
with open(filename, 'w') as outfile:
yaml.dump(data, outfile, default_flow_style=False)

Custom parser for file similar to json python

I'm attempting to create a parser to translate a "custom" file into JSON so I can more easily manipulate its contents (For argument's sake, call the "custom" formate a .qwerty).
I've already created a Lexer which breaks down the file into individual lexemes (tokens) which structure is [token_type, token_value]. Now I am struggling to parse the lexemes into their correct dictionaries, as it is difficult to insert data into a sub-sub-dictionary since Keys aren't constant. As well as insert data into arrays stored in dictionaries.
It should be noted I am attempting to sequentially parse tokens into an actual python json object then dump the json object.
An example of the file can be seen below, along with what the end result is meant to resemble.
FILE: ABC.querty
Dict_abc_1{
Dict_abc_2{
HeaderGUID="";
Version_TPI="999";
EncryptionType="0";
}
Dict_abc_3{
FamilyName="John Doe";
}
Dict_abc_4{
Array_abc{
{TimeStamp="2018-11-07 01:00:00"; otherinfo="";}
{TimeStamp="2018-11-07 01:00:00"; otherinfo="";}
{TimeStamp="2018-11-07 01:00:00"; otherinfo="";}
{TimeStamp="2018-11-07 02:53:57"; otherinfo="";}
{TimeStamp="2018-11-07 02:53:57"; otherinfo="";}
}
Dict_abc_5{
LastContact="2018-11-08 01:00:00";
BatteryStatus=99;
BUStatus=PowerOn;
LastCallTime="2018-11-08 01:12:46";
LastSuccessPoll="2018-11-08 01:12:46";
CallResult=Successful;
}
}
}
Code=999999;
FILE: ABC.json
{
"Dict_abc_1":{
"Dict_abc_2":{
"HeaderGUID":"",
"Version_TPI":"999",
"EncryptionType":"0"
},
"Dict_abc_3":{
"FamilyName":"John Doe"
},
"Dict_abc_4":{
"Array_abc":[
{"TimeStamp":"2018-11-07 01:00:00", "otherinfo":""},
{"TimeStamp":"2018-11-07 01:00:00", "otherinfo":""},
{"TimeStamp":"2018-11-07 01:00:00", "otherinfo":""},
{"TimeStamp":"2018-11-07 02:53:57", "otherinfo":""},
{"TimeStamp":"2018-11-07 02:53:57", "otherinfo":""}
],
"Dict_abc_5":{
"LastContact":"2018-11-08 01:00:00",
"BatteryStatus":99,
"BUStatus":"PowerOn",
"LastCallTime":"2018-11-08 01:12:46",
"LastSuccessPoll":"2018-11-08 01:12:46",
"CallResult":"Successful"
}
}
},
"Code":999999
}
Additional token information,
Token types can either be (with possible values)
IDENTIFIER contain the name of the variable identifier
VARIABLE containing actual data belonging to the parent IDENTIFIER
OPERATOR equal "="
OPEN_BRACKET equal "{"
CLOSE_BRACKET equal "}"
An example of ABC.querty's lexemes can be seen HERE
fundamental logical extract of main.py
def main():
content = open_file(file_name) ## read file
lexer = Lexer(content) ## create lexer class
tokens = lexer.tokenize() ## create lexems as seen in pastebin
parser = Parser(tokens).parse() ## create parser class given tokens
print(json.dumps(parser, sort_keys=True,indent=4, separators=(',', ': ')))
parser.py
import re
class Parser(object):
def __init__(self, tokens):
self.tokens = tokens
self.token_index = 0
self.json_object = {}
self.current_object = {}
self.path = [self.json_object]
def parse(self):
while self.token_index < len(self.tokens):
token = self.getToken()
token_type = token[0]
token_value = token[1]
print("%s \t %s" % (token_type, token_value))
if token_type in "IDENTIFIER":
self.increment()
identifier_type = self.getToken()
if identifier_type[0] in "OPEN_BRACKET":
identifier_two_type = self.getToken(1)
if identifier_two_type[0] in ["OPERATOR","IDENTIFIER"]:
## make dict in current dict
pass
elif identifier_two_type[0] in "OPEN_BRACKET":
## make array in current dict
pass
elif identifier_type[0] in "OPERATOR":
## insert data into current dict
pass
if token_type in "CLOSE_BRACKET":
identifier_type = self.getToken()
if "OPEN_BRACKET" in identifier_type[0]:
#still in array of current dict
pass
elif "IDENTIFIER" in identifier_type[0]:
self.changeDirectory()
else:
#end script
pass
self.increment()
print(self.path)
return self.json_object
def changeDirectory(self):
if len(self.path) > 0:
self.path = self.path.pop()
self.current_object = -1
def increment(self):
if self.token_index < len(self.tokens):
self.token_index+=1
def getToken(self, x=0):
return self.tokens[self.token_index+x]
Additional parse information,
Currently, I was trying to store the current dictionary in a path array to allow me to insert into dictionaries and arrays within dictionaries.
Any suggestions or solutions are very much appreciated,
Thanks.

Last time I solved this problem I find out that finite-state machine is very helpful. I want to recommend the way after you have tokens but I don't know how it's called in english. The principle is: you go through tokens and add one by one on stack. After adding on stack you are checking stack for some rules. Like you combine primitive tokens into expressions that might be a part of more complex expressions.
For example "FamilyName":"John Doe". Tokens are "FamilyName", : and "John Doe".
You add first token on stack.
stack = ["FamilyName"].
Rule 1: str_obj -> E. So you create Expression(type='str', value="FamilyName") and stack is now stack = [Expression].
Then you add next token.
stack = [Expression, ':']. No rules for ':'. Go next.
stack = [Expression, ':', "FamilyName"]. Again we meet rule 1. So stack becomes stack = [Expression, ':', Expression]. Then we see another rule. Rule 2: E:E -> E. Use it like Expression(type='kv_pair, value=(Expression, Expression)). And stack becomes stack=[Expression].
And if you describes all the rules it will work like that. Hope it helps.

python json to csv converting script?

Let me start by stating that I am new to python. I wrote a script that will convert a .json file to csv format. I managed to write a script to do the job, however I don't think that my script will work if the format of the json file was to change. My script assumes that the json file will be in the same format at all times.
<json file example>
{
"Order":
{
"order_id":"8251662",
"order_date":"2012-08-20 13:17:37",
"order_date_shipped":"0000-00-00 00:00:00",
"order_status":"fraudreview",
"order_ship_firstname":"pam",
"order_ship_lastname":"Gregorio",
"order_ship_address1":"1533 E. Dexter St",
"order_ship_address2":"",
"order_ship_city":"Covina",
"order_ship_state":"CA",
"order_ship_zip":"91746",
"order_ship_country":"US United States",
"order_ship_phone":"6268936923",
"order_ship_email":"pgregorio#brighton.com",
"order_bill_firstname":"pam",
"order_bill_lastname":"Gregorio",
"order_bill_address1":"1533 E. Dexter St",
"order_bill_address2":"",
"order_bill_city":"Covina",
"order_bill_state":"CA",
"order_bill_zip":"91746",
"order_bill_country":"US United States",
"order_bill_phone":"6268936923",
"order_bill_email":"pgregorio#brighton.com",
"order_gift_message":"",
"order_giftwrap":"0",
"order_gift_charge":"0",
"order_shipping":"Standard (Within 5-10 Business Days)",
"order_tax_charge":"62.83",
"order_tax_shipping":"0",
"order_tax_rate":"0.0875",
"order_shipping_charge":"7.5",
"order_total":"788.33",
"order_item_count":"12",
"order_tracking":"",
"order_carrier":"1"
},
"Items":
[
{
"item_id":"25379",
"item_date_shipped":"",
"item_code":"17345-J3553-J35532",
"item_quantity":"2","item_taxable":"YES",
"item_unit_price":"32","item_shipping":"0.67",
"item_addcharge_price":"0",
"item_description":" ABC Slide Bracelet: : Size: OS: Silver Sku: J35532",
"item_quantity_returned":"0",
"item_quantity_shipped":"0",
"item_quantity_canceled":"0",
"item_status":"pending",
"item_product_id":"17345",
"item_product_kit_id":"0",
"item_product_sku":"J35532",
"item_product_barcode":"881934310775",
"item_tracking":"",
"item_carrier":"0",
"item_source_orderid":""
},
{
"item_id":"25382",
"item_date_shipped":"",
"item_code":"17608-J3809-J3809C",
"item_quantity":"1",
"item_taxable":"YES",
"item_unit_price":"22",
"item_shipping":"0.23",
"item_addcharge_price":"0",
"item_description":" \"ABC Starter Bracelet 7 1\/4\"\"\": : Size: OS: Silver Sku: J3809C",
"item_quantity_returned":"0",
"item_quantity_shipped":"0",
"item_quantity_canceled":"0",
"item_status":"pending",
"item_product_id":"17608",
"item_product_kit_id":"0",
"item_product_sku":"J3809C",
"item_product_barcode":"881934594175",
"item_tracking":"",
"item_carrier":"0",
"item_source_orderid":""
},
{
"item_id":"25385",
"item_date_shipped":"",
"item_code":"17687-J9200-J92000",
"item_quantity":"2",
"item_taxable":"YES",
"item_unit_price":"12",
"item_shipping":"0.25",
"item_addcharge_price":"0",
"item_description":" ABC Cathedral Bead: : Size: OS: Silver Sku: J92000",
"item_quantity_returned":"0",
"item_quantity_shipped":"0",
"item_quantity_canceled":"0",
"item_status":"pending",
"item_product_id":"17687",
"item_product_kit_id":"0",
"item_product_sku":"J92000",
"item_product_barcode":"881934602832",
"item_tracking":"",
"item_carrier":"0",
"item_source_orderid":""
},
{
"item_id":"25388",
"item_date_shipped":"",
"item_code":"17766-J9240-J92402",
"item_quantity":"2",
"item_taxable":"YES",
"item_unit_price":"22",
"item_shipping":"0.46",
"item_addcharge_price":"0",
"item_description":" ABC Ice Diva Bead: : Size: OS: Silver Sku: J92402",
"item_quantity_returned":"0",
"item_quantity_shipped":"0",
"item_quantity_canceled":"0",
"item_status":"pending",
"item_product_id":"17766",
"item_product_kit_id":"0",
"item_product_sku":"J92402",
"item_product_barcode":"881934655838",
"item_tracking":"",
"item_carrier":"0",
"item_source_orderid":""
},
],
"FraudReasons":
[
{
"order_id":"11957",
"fraud_reason":"order total exceeds max amount"
},
{
"order_id":"11957",
"fraud_reason":"order exceeds max item count"
}
]
}
My script currently works fine with this json file but It wont work if there is only one item or one fraudreason. Here is the code to my script.
<script code>
#!/usr/bin/python
import simplejson as json
import optparse
import pycurl
import sys
import csv
json_data = open(file)
data = json.load(json_data)
json_data.close()
csv_file = '/tmp/' + str(options.orderId) + '.csv'
orders = data['Order']
items = data['Items']
frauds = data['FraudReasons']
o = csv.writer(open(csv_file, 'w'), lineterminator=',')
o.writerow([orders['order_id'],orders['order_date'],orders['order_date_shipped'],orders['order_status'],orders['order_ship_firstname'],orders['order_ship_lastname'],orders['order_ship_address1'],orders['order_ship_address2'],orders['order_ship_city'],orders['order_ship_state'],orders['order_ship_zip'],orders['order_ship_country'],orders['order_ship_phone'],orders['order_ship_email'],orders['order_bill_firstname'],orders['order_bill_lastname'],orders['order_bill_address1'],orders['order_bill_address2'],orders['order_bill_city'],orders['order_bill_state'],orders['order_bill_zip'],orders['order_bill_country'],orders['order_bill_phone'],orders['order_bill_email'],orders['order_gift_message'],orders['order_giftwrap'],orders['order_gift_charge'],orders['order_shipping'],orders['order_tax_charge'],orders['order_tax_shipping'],orders['order_tax_rate'],orders['order_shipping_charge'],orders['order_total'],orders['order_item_count'],orders['order_tracking'],orders['order_carrier']])
for item in items:
o.writerow([item['item_id'],item['item_date_shipped'],item['item_code'],item['item_quantity'],item['item_taxable'],item['item_unit_price'],item['item_shipping'],item['item_addcharge_price'],item['item_description'],item['item_quantity_returned'],item['item_quantity_shipped'],item['item_quantity_canceled'],item['item_status'],item['item_product_id'],item['item_product_kit_id'],item['item_product_sku'],item['item_product_barcode'],item['item_tracking'],item['item_carrier'],item['item_source_orderid']])
for fraud in frauds:
o.writerow([fraud['fraud_reason']],)
I also have not been able to figure out how not to use the labels I hope someone can help me with this
thanks in advance.

You may want to use csv.DictWriter:
# It's considered best to stash the main logic of your script
# in a main() function like this.
def main(filename, options):
with open(filename) as fi:
data = json.load(fi)
csv_file = '/tmp/' + str(options.orderId) + '.csv'
order = data['Order']
items = data['Items']
frauds = data['FraudReasons']
# Here's one way to keep this maintainable if the JSON
# format changes, and you don't care too much about the
# order of the fields...
orders_fields = sorted(orders.keys())
item_fields = sorted(items[0].keys()) if items else ()
fraud_fields = sorted(fraud[0].keys()) if fraud else ()
csv_options = dict(lineterminator=',')
with open(csv_file, 'w') as fo:
o = csv.DictWriter(fo, order_fields, **csv_options)
o.writeheader()
o.writerow(orders)
fo.write('\n') # Optional, if you want to keep them separated.
o = csv.DictWriter(fo, item_fields, **csv_options)
o.writeheader()
o.writerows(items)
fo.write('\n') # Optional, if you want to keep them separated.
o = csv.DictWriter(fo, fraud_fields, **csv_options)
o.writeheader()
o.writerows(frauds)
# If this script is run from the command line, just run
# main(). Here's the place to use `optparse`.
if __name__ == '__main__':
main(...) # You'll need to fill in the main() arguments...
If you need to specify the order of fields, assign them to a tuple like this:
orders_fields = (
'order_id',
'order_date',
'order_date_shipped',
# ... etc.
)

You should ask the json-generated object (data) for the names of the fields. To retain the input order, tell json to use collections.OrderedDict instead of plain dict (requires python 2.7):
import json
from collections import OrderedDict as ordereddict
data = json.loads(open('mydata.json', object_pairs_hook=ordereddict)
orders = data['Order']
print orders.keys() # Will print the keys in the order they were read
You can then use orders.keys() instead of your hard-coded list, either with writerow or (simpler) with csv.DictWriter.
Note that this uses the default json, not simplejson, and requires python 2.7 for the ordered_pairs_hook argument and the OrderedDict type.
Edit: Yeah, I see from the comments that you're stuck with 2.4. You can download an ordereddict from PyPi, and you can extend the JSONDecoder class and pass it with the cls argument (see here), instead of object_pairs_hook, but that's uglier and more work...

Categories

Resources