Ansible uses YAML syntax mainly has key-value pairs, where every value can be
a simple value (number or string)
or
a list
or
a key-value pair(nested)
Anchoring a value, Type conversion in YAML is just a pre-processing option.
1)
From the data structure aspect,
Is YAML syntax a dictionary of dictionary?
2)
For command: ansible -m shell 'hostname' all, Is ansible a wrapper of python interpreter? taking multiple command line options...
From the data structure aspect,
Is YAML syntax a dictionary of dictionary?
No. YAML syntax models a directed graph. Your assumptions on YAML given initially are wrong. In YAML, a value is one of three things:
A scalar (number, string, date, …)
A sequence (list of values)
A mapping (list of key-value pairs where both keys and values are any kind of value)
Since any non-scalar value can contain other non-scalar values, YAML can represent a tree of arbitrary depth – so it's not necessarily a dictionary of dictionaries.
Now, YAML also allows to have an anchor on any value, and reference that value later via alias:
anchored value: &anchor My value
alias: *anchor
Here, *alias references the anchored scalar value My value. This can be used to define cyclic graphs:
--- &root # this annotates the root sequence;
- one
- two # simple sequence items
- three
- *root # reference to the sequence, meaning that the sequence contains itself
Mind that both sequence and mappings are usually started implicitly in YAML syntax. If children are key/value pairs, it's a mapping (first example); if children are list items, it's a sequence (second example). --- starts the document and is usually omitted.
For command: ansible -m shell 'hostname' all, Is ansible a wrapper of python interpreter? taking multiple command line options...
See the man page of the ansible command. You are probably looking for the -a ARGS option. I am unsure what you would consider a wrapper of the Python interpreter and you may want to clarify what you actually want to do. Generally, the answer to that is no.
Related
I am running topic modeling using Gensim. Before creating the document-term matrix, one needs to create a dictionary of tokens.
dictionary = corpora.Dictionary(tokenized_reviews)
doc_term_matrix = [dictionary.doc2bow(rev) for rev in tokenized_reviews]
But, I don't understand what kind of object "dictionary" is.
So, when I type:
type(dictionary)
I get
gensim.corpora.dictionary.Dictionary
Is this a dictionary ( a kind of data structure)? If so, why can't I see the content (I am just curious)?
When I type
dictionary
I get:
<gensim.corpora.dictionary.Dictionary at 0x1bac985ebe0>
The same issue exists with some of the objects in NLTK.
If this is a dictionary (as a data structure), why I am not able to see the keys and values like any other Python dictionary?
Thanks,
Navid
This is a specific Dictionary class implemented by the Gensim project.
It will be very similar in interface to the standard Python dict (and other various Dictionary/HashMap/etc types you may have used elsewhere).
However, to see exactly what it can do, you should consult the class-specific documentation:
https://radimrehurek.com/gensim/corpora/dictionary.html
Like a dict, you can do typical operations:
len(dictionary) # gets number of entries
dictionary[key] # gets the value at a certain key (word)
dictionary.keys() # gets all stored keys
The reason you see a generic <gensim.corpora.dictionary.Dictionary at 0x1bac985ebe0> when you try to display the value of the dictionary itself is that it hasn't defined any convenience display-string with more info - so you're seeing the default for any random Python object. (Such dictionaries are usually far too large to usefull dump their full contents whenever asked, generically, to "show yourself".
I have googled without success so I would like what does isinstance means combined with yaml ScalarNode, SequenceNode and MappingNode?
(I know already what isinstance is)
For example
if isinstance(v,yaml.ScalarNode):
#do something
elif isinstance(v,yaml.SequenceNode):
#something else
elif isinstance(v, yaml.MappingNode):
#another thing
Node types are a part of YAML's Representation data structure. YAML defines its (de)serialization pipeline as follows:
(source: yaml.org)
The Representation is a, potentially cyclic, graph of nodes. In it, anchors and aliases have been resolved. In PyYAML, you typically use subgraphs of this data structure to implement custom constructors and representers that generate your native objects, as indicated by the arrows in the diagram.
A ScalarNode is a node representing a single scalar in the YAML source or output. A single scalar can be a plain scalar (e.g. foo), a quoted scalar ('foo' or "foo"), or a block scalar (starting with | or >). The scalar content, with escape sequences, newlines, indentation already processed, is available in the field .value as string. This is even true for values that are by default constructed into non-strings. For example, true will by default generate a boolean value, but as ScalarNode, it contains the value "true" as string.
SequenceNode is a node representing a sequence. The value field of a SequenceNode contains a list of nodes that correspond to the items in the sequence.
MappingNode is a node representing a mapping. The value field of a MappingNode contains a list of tuples, where each tuple consists of the key node and the value node.
All nodes have a field tag that contains the resolved tag of the node. i.e. a ScalarNode with value true would typically have the tag yaml.org,2002:bool. The resolved tag depends on the loader you use, for example if you use PyYAML's BaseLoader, true will resolve to a normal string, which is yaml.org,2002:str. In any case, if there was an explicit tag on the node (e.g. !!str, that tag will be in the tag field.
Coming back to the question, this kind of code is typically used in custom constructors. They get a node as input and are to produce a native value and return it. Usually, a custom constructor expects a specific kind of node but if you want to do proper error reporting, you still want to check whether you actually got the kind of node you need. For this, you use the code you posted.
TLDR: I am making a python wrapper around something for LabVIEW to use and I want to pass a dict (or even kwargs) [i.e. key/value pairs] to a python script so I can have more dynamic function arguments.
LabVIEW 2018 implemented a Python Node which allows LabVIEW to interact with python scripts by calling, passing, and getting returned variables.
The issue is it doesn't appear to have native support for the dict type:
Python Node Details Supported Data Types
The Python Node supports a large number of data types. You can use
this node to call the following data types:
Numerics Arrays, including multi-dimensional arrays Strings Clusters
Calling Conventions
This node converts integers and strings to the corresponding data
types in Python, converts arrays to lists, and converts clusters to
tuples.
Of course python is built around dictionaries but it appears LabVIEW does not support any way to pass a dictionary object.
Does anyone know of a way I can pass a cluster of named elements (or any other dictionary type) to a python script as a dict object?
There is no direct way to do it.
The simplest way on both sides would be to use JSON strings.
From LabVIEW to Python
LabVIEW Clusters can be flattened to JSON (Strings > Flatten/unflatten):
The resulting string can be converted to a dict in just one line (plus an import) python:
>>> import json
>>> myDict=json.loads('{"MyString":"FooBar","MySubCluster":{"MyInt":42,"MyFloat":3.1410000000000000142},"myIntArray":[1,2,3]}')
>>> myDict
{u'MyString': u'FooBar', u'MySubCluster': {u'MyInt': 42, u'MyFloat': 3.141}, u'myIntArray': [1, 2, 3]}
>>> myDict['MySubCluster']['MyFloat']
3.141
From Python to LabVIEW
The Python side is easy again:
>>> MyJson = json.dumps(myDict)
In LabVIEW, unflatten JSON from string, and wire a cluster of the expected structure with default values:
This of course requires that the structure of the dict is fixed.
If it is not, you can still access single elements by giving the path to them as array:
Limitations:
While this works like a charm (did you even notice that my locale uses comma as decimal sign?), not all datatypes are supported. For example, JSON itself does not have a time datatype, nor a dedicated path datatype, and so, the JSON VIs refuse to handle them. Use a numerical or string datatype, and convert it within LabVIEW.
Excourse: A dict-ish datatype in LabVIEW
If you ever need a dynamic datatype in LabVIEW, have a look at attributes of variants.
These are pairs of keys (string) and values (any datatype!), which can be added and reads about as simple as in Python. But there is no (builtin, simple) way to use this to interchange data with Python.
I have a dictionary whose keys are tuples like (int, str, int, str, int), and the corresponding values are lists of floats of the same size.
I pickled the dictionary twice by the same script:
import pickle
with open(name, 'wb') as source:
pickle.dump(the_dict, source)
For the two resulting binary files test_1 and test_2, I run
diff test_1 test_2
in a terminal (I'm using macOS) to see whether I can use diff to tell the difference. However, I received
Binary files test_1 and test_2 differ
Why? Was the same dictionary being pickled in different ways? Does it mean I cannot use diff to tell whether two dictionaries are identical?
Depending on what version of Python you are using, Python versions before v3.6 do not remember the order of insertion. Python v3.6 made this an implementation detail and v3.7 made it a language feature.
For backwards compatibility, you shouldn't depend on the dictionary remembering the order of inserted keys. Instead, you can use OrderedDict from the Collections module.
Also, using diff on pickled dict data may show differences in the data even though the actual dictionaries are equivalent -- since dicts, unlike lists, generally make no assurances on order state (see above for when that is not the case).
In Python, there are dictionaries:
residents = {'Puffin' : 104, 'Sloth' : 105, 'Burmese Python' : 106}
In Ruby, there are Hashes:
residents = {'Puffin' => 104, 'Sloth' => 105, 'Burmese Python' => 106}
The only difference is the : versus => syntax. (Note that if the example were using variables instead of strings, then there would be no syntax difference.)
In Python, you call a dictionary's value via a key:
residents['Puffin']
# => 104
In Ruby, you grab a Hash's value via a key as well:
residents['Puffin']
# => 104
They appear to be the same.
What is the difference between a Hash in Ruby and a dictionary in Python?
Both Ruby's Hash and Python's dictionary represent a Map Abstract Data Type (ADT)
.. an associative array, map, symbol table, or dictionary is an abstract data type composed of a collection of (key, value) pairs, such that each possible key appears at most once in the collection.
Furthermore, both Hash and dictionary are implemented as Hash Tables which require that keys are hashable and equatable. Generally speaking, insert and delete and fetch operations on a hash table are O(1) amortized or "fast, independent of hash/dict size".
[A hash table] is a data structure used to implement an associative array, a structure that can map keys to values. A hash table uses a hash function to compute an index into an array of buckets or slots, from which the correct value can be found.
(Map implementations that use Trees, as opposed to Hash Tables, are found in persisted and functional programming contexts.)
Of course, there are also differences between Ruby and Python design choices and the specific/default Map implementations provided:
Default behavior on missing key lookup: nil in Hash, exception in dict1
Insertion-ordering guarantees: guaranteed in Hash (since Ruby 2.0), no guarantee in dict (until Python 3.6)1
Being able to specify a default value generator: Hash only1
Ability to use core mutable types (eg. lists) as keys: Hash only2
Syntax used for Hash/dict Literals, etc..
The [] syntax support is common insofar as both languages provide syntactic sugar for an overloaded index operator, but is implemented differently underneath and has different semantics in the case of missing keys.
1 Python offers defaultdict and OrderedDict implementations as well which have different behavior/functionality from the standard dict. These implementation allow default value generators, missing-key handling, and additional ordering guarantees that are not found in the standard dict type.
2 Certain core types in Python (eg. list and dict) explicitly reject being hashable and thus they cannot be used as keys in a dictionary that is based on hashing. This is not strictly a difference of dict itself and one can still use mutable custom types as keys, although such is discouraged in most cases.
They (dictionary in Python, hash in Ruby) are identical for all practical purposes, and implement a general Dictionary / Hashtable (a key - value store) where you typically store an entry given a unique key, and get fast lookup for it's value.
Now ruby also supports following sysntax:
residents = {'Puffin': 104, 'Sloth': 105, 'Burmese Python': 106}
But then we should access values by the symbol notation:
residents[:Puffin]