In my application I am using json as the main API. It stores a set of commands to process files. This works great except part of my requirements is that I need to embed python.
I have a clunky solution, but am looking for a better one.
Here is an example of my approach:
{ "file key" :
{ "instruction1" :
{ "commandList" : [
{
"conditionalPythonBlock" : [
"myval = int(dict['count'])",
"retval = myval > 0"
]
}
]}
}
}
I wrote a routine to collapse the "conditionalPythonBlock" into one multi line string which can then be executed. The multi line approach is necessary for correct indentation.
The 'dict' variable in 'dict['mycount']' is local context in the parser of this file.
The retval is also a local variable in the context of the parser.
Anyone have a more elegant solution?
Thanks!
I think one very good answer here is to use YAML instead of JSON.
PyYaml documentation
The above example becomes:
file key :
instruction1 :
- commands :
conditionalPythonBlock : |
myval = int(dict['count'])
retval = myval > 0
Much more readable, and the script section can just be copied in.
Related
I use xmltodict to convert Python dictionaries to XML. I would like to include the namespaces. What I've come up with is this:
xml_dict = {
"http://namespace.org:workflow":
{"action-list": "..."}
}
namespaces = {"http://namespace.org": "ws"}
xml = xmltodict.unparse(xml_dict, namespaces=namespaces,
pretty=True, short_empty_elements=True, full_document=False)
That gives me the result:
<ws:workflow>
<action-list>...</action-list>
</ws:workflow>
How can I include the namespaces in the result? I would like something like this:
<ws:workflow xmlns:ws="http://namespace.org">
Is there a simple solution using xmltodict?
Note: I've checked all keyword arguments in the source code of unparse and _emit functions. I guess the preprocessor argument will be the key to a better solution. But it's not documented.
Not nice, and not a general solution to add all the namespaces, but works.
import xmltodict
xml_dict = {
"http://namespace.org:workflow":
{
"#REPLACE_THIS": "http://namespace.org",
"action-list": "..."
}
}
namespaces = {"http://namespace.org": "ws"}
xml = xmltodict.unparse(xml_dict, namespaces=namespaces, pretty=True, short_empty_elements=True, full_document=False)
xml = xml.replace("REPLACE_THIS", "xmlns:ws", 1)
print(xml)
That gives me exactly what I want:
<ws:workflow xmlns:ws="http://namespace.org">
<action-list>...</action-list>
</ws:workflow>
The third argument in the replace function makes it sure that not other occurrences of "REPLACE_THIS" will be replaced, just the first one.
This question already has an answer here:
Syntax Error on elif statement in Python
(1 answer)
Closed 6 years ago.
I have used Python occasionally for several months, I know we can use # and """ or ''' to comment. But when i wanted to comment some items of a dictionary, with comment words ('''), i failed.
testItems = {
'TestOne':
{
"NameId":101
"Score":99
},
'''
'TestTwo':
{
"NameId":101
"Score":99
}
'''
}
then i get ther error of SyntaxError: invalid syntax pointing to the last ''' position.
I also know there are some indent rule of python language. But i tried so many indent possibility, still fail.
You can only use ''' or """ to comment where strings are allowed as these don't create comments, but just strings.
In the situation you describe, you are not allowed to put a string. Either move the closing } upwards or uncomment your unwanted code part line by line.
Doing
test_items_1 = {
"NameId":101,
"Score":99
}
test_items_2 = {
"NameId":101,
"Score":99
}
testItems = {
'TestOne': test_items_1,
# 'TestTwo': test_items_2,
}
would work as well.
Values in between ''' or """ within dictionary will be considered as another item, not comment.
In your case the the content between ''' is treated as another item's key in that dictionary.
You have to use # to comment the unwanted codes.
Ex:
testItems = {
'TestOne':
{
"NameId":101,
"Score":99
},
# 'TestTwo':
# {
# "NameId":101
# "Score":99
# }
}
As Ilja Everilä mentioned, there is no multi-line comment in python. Although when I copied your code to my pycharm templet, It didn't gave me any error.
Still, in your case, I'd recommend you to use single line commenting method (#). Still, '''...''' or """..."""(convert that part to string) could be used but it will just increase your line of code.
Coming to your question, you are getting an error because:
Your code could be rewritten as :
testItems = {'TestOne': {"NameId":101, "Score":99} ''' 'TestTwo':{
"NameId":101"Score":99 } ''' }
That's how python reads it,
As you want to keep the part in bold, and comment the italics part. You just can't use string. As you can see that python is taking the whole part in braces (Bold + Italics) as single entity. So either use #, or take out that part from there.
You could rewrite as:
testItems = {
'TestOne':
{
"NameId":101,
"Score":99
}
# 'TestTwo':
# {
# "NameId":101,
# "Score":99
# }
}
I'm fairly new to javascript and such so I don't know if this will be worded correctly, but I'm trying to parse a JSON object that I read from a database. I send the html page the variable from a python script using Django where the variable looks like this:
{
"data":{
"nodes":[
{
"id":"n0",
"label":"Redditor(user_name='awesomeasianguy')"
},
...
]
}
}
Currently, the response looks like:
"{u'data': {u'nodes': [{u'id': u'n0', u'label': u"Redditor(user_name='awesomeasianguy')"}, ...
I tried to take out the characters like u' with a replaceAll type statement as seen below. This however is not that easy of a solution and it seems like there has got to be a better way to escape those characters.
var networ_json = JSON.parse("{{ networ_json }}".replace(/u'/g, '"').replace(/'/g, '"').replace(/u"/g, '"').replace(/"/g, '"'));
If there are any suggestions on a method I'm not using or even a tool to use for this, it would be greatly appreciated.
Use the template filter "|safe" to disable escaping, like,
var networ_json = JSON.parse("{{ networ_json|safe }}";
Read up on it here: https://docs.djangoproject.com/en/dev/ref/templates/builtins/#safe
This is the way reading from a .json file on ubuntu terminal:
python -c "import json;print json.loads(open('json_file.json', 'r').read())['foo']['bar']"
What I'd like to do is altering the JSON file, adding new objects and arrays. So how to do this in python?
json_file.json:
{
"data1" :
[
{
"unit" : "Unit_1",
"value" : "20"
},
{
"unit" : "Unit_2",
"value" : "10"
}
]
}
First of all, create a new python file.
import json
data = json.loads(open('json_file.json', 'r').read())
The data is then just a bunch of nested dictionaries and lists.
You can modify it the same way you would modify any python dictionary and list; it shouldn't be hard to find a resource on this as it is one of the most basic python functionalities. You can find a complete reference at the official python documentation, and if you are familiar with arrays/lists and associative arrays/hashes in any language, this should be enough to get you going. If it's not, you can probably find a tutorial and if that doesn't help, if you are able to create a well-formed specific question then you could ask it here.
once you are done, you can put everything back into json:
print json.dumps(data)
For more information on how to customize the output, and about the json module overall, see the documentation.
I have a file containing lots of data put in a form similar to this:
Group1 {
Entry1 {
Title1 [{Data1:Member1, Data2:Member2}]
Title2 [{Data3:Member3, Data4:Member4}]
}
Entry2 {
...
}
}
Group2 {
DifferentEntry1 {
DiffTitle1 {
...
}
}
}
Thing is, I don't know how many layers of parentheses there are, and how the data is structured. I need modify the data, and delete entire 'Entry's depending on conditions involving data members before writing everything to a new file. What's the best way of reading in a file like this? Thanks!
The data structure basically seems to be a dict where they keys are strings and the value is either a string or another dict of the same type, so I'd recommend maybe pulling it into that sort of python structure,
eg:
{'group1': {'Entry2': {}, 'Entry1': {'Title1':{'Data4': 'Member4',
'Data1': 'Member1','Data3': 'Member3', 'Data2': 'Member2'},
'Title2': {}}}
At the top level of the file you would create a blank dict, and then for each line you read, you use the identifier as a key, and then when you see a { you create the value for that key as a dict. When you see Key:Value, then instead of creating that key as a dict, you just insert the value normally. When you see a } you have to 'go back up' to the previous dict you were working on and go back to filling that in.
I'd think this whole parser to put the file into a python structure like this could be done in one fairly short recursive function that just called itself to fill in each sub-dict when it saw a { and then returned to its caller upon seeing }
Here is a grammar.
dict_content : NAME ':' NAME [ ',' dict_content ]?
| NAME '{' [ dict_content ]? '}' [ dict_content ]?
| NAME '[' [ list_content ]? ']' [ dict_content ]?
;
list_content : NAME [ ',' list_content ]?
| '{' [ dict_content ]? '}' [ ',' list_content ]?
| '[' [ list_content ]? ']' [ ',' list_content ]?
;
Top level is dict_content.
I'm a little unsure about the comma after dicts and lists embedded in a list, as you didn't provide any example of that.
If you have the grammar for the structure of your data file, or you can create it yourself, you could use a parser generator for Python, like YAPPS: link text.
I have something similar but written in java. It parses a file with the same basic structure with a little different syntax (no '{' and '}' only indentation like in python). It is a very simple script language.
Basically it works like this: It uses a stack to keep track of the inner most block of instructions (or in your case data) and appends every new instruction to the block on the top. If it parses an instruction which expects a new block it is pushed to the stack. If a block ends it pops one element from the stack.
I do not want to post the entire source because it is big and it is available on google code (lizzard-entertainment, revision 405). There is a few things you need to know.
Instruction is an abstract class and it has a block_expected method to indicate wether the concrete instruction needs a block (like loops, etc) In your case this is unnecessary you only need to check for '{'.
Block extends Instruction. It contains a list of instructions and has an add method to add more.
indent_level return how many spaces are preceding the instruction text. This is also unneccessary with '{}' singns.
placeholder
BufferedReader input = null;
try {
input = new BufferedReader(new FileReader(inputFileName));
// Stack of instruction blocks
Stack<Block> stack = new Stack<Block>();
// Push the root block
stack.push(this.topLevelBlock);
String line = null;
Instruction prev = new Noop();
while ((line = input.readLine()) != null) {
// Difference between the indentation of the previous and this line
// You do not need this you will be using {} to specify block boundaries
int level = indent_level(line) - stack.size();
// Parse the line (returns an instruction object)
Instruction inst = Instruction.parse(line.trim().split(" +"));
// If the previous instruction expects a block (for example repeat)
if (prev.block_expected()) {
if (level != 1) {
// TODO handle error
continue;
}
// Push the previous instruction and add the current instruction
stack.push((Block)(prev));
stack.peek().add(inst);
} else {
if (level > 0) {
// TODO handle error
continue;
} else if (level < 0) {
// Pop the stack at the end of blocks
for (int i = 0; i < -level; ++i)
stack.pop();
}
stack.peek().add(inst);
}
prev = inst;
}
} finally {
if (input != null)
input.close();
}
That depends on how the data is structured, and what kind of changes you need to do.
One option might be to parse that into a Python data structure, it seems similar, except that you don't have quotes around the strings. That makes complex manipulation easy.
On the other hand, if all you need to do is make changes that modify some entries to other entries, you can do it with search and replace.
So you need to understand the issue better before you can know what the best way is.
This is a pretty similar problem to XML processing, and there's a lot of Python code to do that. So if you could somehow convert the file to XML, you could just run it through a parser from the standard library. An XML version of your example would be something like this:
<group id="Group1">
<entry id="Entry1">
<title id="Title1"><data id="Data1">Member1</data> <data id="Data2">Member2</data></title>
<title id="Title2"><data id="Data3">Member3</data> <data id="Data4">Member4</data></title>
</entry>
<entry id="Entry2">
...
</entry>
</group>
Of course, converting to XML probably isn't the most straightforward thing to do. But your job is pretty similar to what's already been done with the XML parsers, you just have a different syntax to deal with. So you could take a look at some XML parsing code and write a little Python parser for your data file based on that. (Depending on how the XML parser is implemented, you might even be able to copy the code, just change a few regular expressions, and run it for your file)