I am looking to create a new folder for each item under the folders key as below.
Yaml file is as below
---
software:
dir: C:/software
folders:
- bob:
ip:
- "192.168.1.5"
- "192.168.1.6"
password:
- kdhkfjhkjdkjfjsikd
- alice:
ip:
- "192.168.1.3"
password:
- hfsdkljfdhkjsfkjfsd
Python Code
from sys import path
import requests
import os
import yaml
with open('config.yaml') as f:
config = yaml.load(f, yaml.loader.FullLoader)+
for i in config["folders"]:
for f in i.values():
print(i)
I am just using print as its easier to understand the data.
This returns
{'bob': {'ip': ['192.168.103.5', '192.168.103.6'], 'password': ['kdhkfjhkjdkjfjsikd']}}
{'alice': {'ip': ['192.168.105.3'], 'password': ['hfsdkljfdhkjsfkjfsd']}}
now what I want to happen is just return the folder names, e.g. bob, alice.
I have tried the below with no luck, thank you.
for i in config["folders"]:
for f in i.values():
print(i[0])
which returns
print(i[0])
KeyError: 0
You can get the folder names with:
import yaml
with open('config.yaml') as f:
config = yaml.load(f, yaml.loader.FullLoader)
folder_names = [list(folder)[0] for folder in config["folders"]]
You should always be aware of what type the variables are. You can check this using the type() function.
Some examples:
type(config) should give you <class 'dict'>
type(config["folders"]) should give you <class 'list'>, as it is a list of dictionaries.
When you do
for i in config["folders"]:
print(i[0])
every i in the loop corresponds to an item of list config["folders"], which itself is a dictionary. Therefore, with i[0], you try to access the dictionary item with key 0, which does not exist.
Instead, you probably want to print the first key of the dictionary with:
for i in config["folders"]:
print(list(i.keys())[0])
try this:
import yaml
with open('yamer.yaml') as f:
data = yaml.load(f, Loader=yaml.FullLoader)
data = data.get('folders')
for x in data:
for i in x.keys():
print(i)
It basically "gets" the dictionary with the key "folders" from data dict. Then we can iterate through it as shown above and print the folder names. You can create a list and start appending the names (strings) to it if needed.
Thank you!
Related
I have a few small dictionaries which I would like to include in one yaml file and access each one of them separately with PyYAML.
After having some trouble finding out how to write the YAML file, I ended up with this version, where --- is supposed to distinguish the two dictionaries, which are named as elements and as parameters. This was inspired by this post and answer
--- !elements
n: 'N'
p: 'P'
k: 'K'
--- !parameters
ph: 'pH'
org_mat : 'organic matter'
To continue, I created a variable with the name of the path of the file: yaml_fpath = r"\Users\user1\Desktop\yaml_file" and I tried several methods to access the dictionaries, such as:
for item in yaml.safe_load_all(yaml_fpath):
print(item)
or
yaml.safe_load(open(yaml_fpath, 'r', encoding = 'utf-8'))
but none does what I need. In fact, I would like to load the file and be able to call each dictionary by each name when I need to use it.
What am I missing from the documentation of PyYAML?
Assume you have multiple YAML files in a directory that consist of a root level mapping that is tagged:
!elements
n: 'N'
p: 'P'
k: 'K'
If these files have the recommended suffix for YAML files you can combine them using:
import sys
import ruamel.yaml
from pathlib import Path
file_out = Path('out.yaml')
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
yaml.explicit_start = True
data = []
for file_name in Path('.').glob('*.yaml'):
if file_name.name == file_out.name:
continue
print('appending', file_name)
data.append(yaml.load(file_name))
yaml.dump_all(data, file_out)
print(file_out.read_text())
which gives:
appending file1.yaml
appending file2.yaml
--- !elements
n: 'N'
p: 'P'
k: 'K'
--- !parameters
ph: 'pH'
org_mat: 'organic matter'
There is no need to register any classes that can handle the tag if you use the (default)
roundtrip mode. You do have to set explicit_start to get the leading directives end indicator
(---, often incorrectly called document separator, although it doesn't have to
appear at the beginning of a document). The other directives end indicators are a
result of using dump_all() instead of dump().
If you want to access some value, assuming you don't know where it is in out.yaml, but knowing
the tag and key, you can do:
import sys
import ruamel.yaml
from pathlib import Path
file_in = Path('out.yaml')
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
data = yaml.load_all(file_in)
my_tag = 'parameters'
my_key = 'org_mat'
for d in data:
if d.tag.value == '!' + my_tag:
if my_key in d:
print('found ->', d[my_key])
which gives:
found -> organic matter
I have dictionary with modules and their paths
modulesDict that looks like this:
'Module1': ['folder1/folder2/Module1'], 'Module2': ['folder1/folder2/Module2'] , etc
Now I need to make a script in python that will browse each of the paths and collect all files with the .txt extension inside and store them in a unique place for that one specific module.
My current code is:
for j in modulesDict[path]:
if j != 0:
continue
for file in os.listdir(j):
if file.endswith(".txt"):
pathList.append(file)
when I run it I get an error: KeyError: X for the last item in the dict.
Short example:
So I have a dict that looks like 'Module1': ['folder1/folder2/Module1'], 'Module2': ['folder1/folder2/Module2'] etc..
and I need to make a script that will browse each of the paths, take all files with the .txt extension from there, and store them in a unique list. Example: path ['folder1/folder2/Module1'] contains inside module1.txt, module1.1.txt, module1.2.txt, and I need to store them in a list called list_module1 = [module1.txt, module1.1.txt, module1.2.txt]
please advice.
You can use two nested list comprehensions to produce a list of list of files:
from os import listdir
result = [[f for file in pathList\
for f in listdir(file) if f.endswith(".txt")]\
for pathList in modulesDict.values()]
Similarly, you can use a list comprehension nested inside a dictionary comprehension to produce a dictionary with module names as keys and list of files as values:
from os import listdir
result = {name: [f for file in pathList\
for f in listdir(file) if f.endswith(".txt")]\
for name, pathList in modulesDict.items()}
... or file paths as values:
from os import listdir
# As list:
result_list = [[os.path.join(file, f) for file in pathList\
for f in listdir(file) if f.endswith(".txt")]\
for pathList in modulesDict.values()]
# As dictionary:
result_dict = {name: [os.path.join(file, f) for file in pathList\
for f in listdir(file) if f.endswith(".txt")]\
for name, pathList in modulesDict.items()}
You can try something like this:
from pathlib import Path # a python standard library
modulesDict = {'Module1': ['folder1/folder2/Module1'], 'Module2': ['folder1/folder2/Module2']}
output = {}
for module, folder_paths in modulesDict.items():
for folder_path in folder_paths:
output[module] = list(map(str, list(Path(folder_path).glob("*.txt"))))
The above code outputs something like this:
{
'Module1': ['path/to/text/file1', 'path/to/text/file/2', etc..],
'Module2': ['path/to/text/file1', 'path/to/text/file/2', etc..],
.
.
}
import glob
# Input Dictionary
input_ = {
"path1": "path/to/folder1",
"path2": "path/to/folder2"
}
output_ = {}
for path_ in input_.keys():
# uncomment the below line to see which keys you have in the input_ dictionary
# print(path_)
output_[path] = glob.glob(input_[path] + "/*.txt"),
Output -
{
"path1": ["path/to/folder1/1.txt", "path/to/folder1/1.txt"]
"path2": [] # Empty list if there are no txt files in the module.
}
Brief explanation -
glob library can be used to find files in paths.
We can store the new paths in another dictionary with the same keys to avoid confusion. This way we can get unique lists for each path in the input_ dictionary inside a single object.
If there are not txt files in a module, then glob.glob returns an empty list.
Regarding your error, it seems like you are trying to access a key which is not in the dictionary. You can try to debug by adding a print(path_) in the for loop
So I have a project I'm working on for fun but it requires me to append to a dictionary from another python file. In file1.py it will look like
Name: Eric <-- user input
Age: 27 <-- user input
and file2.py,
information = {'Eric':27}
I know that I can temporarily append to a dictionary while running the code, but it seems to reset after I close the program. Like recently I've seen this on a StackOverflow question
d = {'key': 'value'}
print(d)
# {'key': 'value'}
d['mynewkey'] = 'mynewvalue'
print(d)
# {'key': 'value', 'mynewkey': 'mynewvalue'}
But this too, resets after every run so I thought that the only way to save the dictionary is to write it to another file. Is there any way that I can achieve this or maybe a better alternative?
You can use JSON to save data to a file.
This will save the data, that is stored in your dictionary, in a file.
import json
my_dict = {"key": "value", "key2": "value2"}
with open("output_file.txt", "w") as file:
json.dump(my_dict, file, indent=4)
To use that data again, you can load that file.
import json
with open("output_file.txt") as file:
my_dict = json.load(file)
print(my_dict) # Will print {"key": "value", "key2": "value2"}
JSON stands for JavaScriptObjectNotation, and it's a way to save data in a string format (a file)
So JSON can convert a string into data, if it is valid JSON:
import json
string_data = '{"key": "value"}'
dictionary = json.loads(string_data)
print(type(string_data)) # <class 'str'>
print(type(dictionary)) # <class 'dict'>
I have the following txt... (I've saved as a dictionary)
"{'03/01/20': ['luiana','macarena']}\n"
"{'03/01/21': ['juana','roberta','mariana']}\n"
"{'03/01/24': ['pedro','jose','mario','luis']}\n"
"{'03/01/24': ['pedro','jose','mario','luis']}\n"
"{'03/01/22': ['emanuel']}\n"
the problem is that I want to open it as a dictionary, but I don't know how I can do it. I've tried with:
f = open ('usuarios.txt','r')
lines=f.readlines()
whip=eval(str(lines))
but it's not working... my idea is for example just take the dictionaries that have as a value the next day 03/01/24
if you want to to have only one dict with all the saved dictionaries you can use:
import ast
my_dict = {}
with open('your_file.txt', 'r') as fp:
for line in fp.readlines():
new_dict = ast.literal_eval(line)
for key, value in new_dict.items():
if key in my_dict:
my_dict[key].extend(value)
else:
my_dict[key] = value
print(my_dict)
output:
{'03/01/20': ['luiana', 'macarena'], '03/01/21': ['juana', 'roberta', 'mariana'], '03/01/24': ['pedro', 'jose', 'mario', 'luis', 'pedro', 'jose', 'mario', 'luis'], '03/01/22': ['emanuel']}
if yo could change the format you are saving the strings from
"{'03/01/20': ['luiana','macarena']}\n"
to
'{"03/01/20": ["luiana","macarena"]}\n'
Then you could just do the following:
import json
line = '{"03/01/20": ["luiana","macarena"]}\n'
d = json.loads('{"03/01/20": ["luiana","macarena"]}\n')
The result would be a dictionary d with dates as keys:
>>> {'03/01/20': ['luiana', 'macarena']}
Them, it would be just a mater of looping over the lines of your file and adding them to your dictionary.
An alternative approach would be to use pickle to save your dictionary instead of the .txt, them use it to load from the disk.
This is my YAML file (input.yaml):
team_member:
name: Max
hobbies:
- Reading
team_leader:
name: Stuart
hobbies:
- dancing
I want to edit this YAML file to add more values in key 'hobbies', example:
team_member:
name: Max
hobbies:
- Reading
- Painting
team_leader:
name: Stuart
hobbies:
- Dancing
- Fishing
I tried to implement the code Anthon to fit my situation but it didn't helped at all, because the indention level of that YAML file is different from mine.
Example:
import sys
import ruamel.yaml
yaml = ruamel.yaml.YAML()
# yaml.preserve_quotes = True
with open('input.yaml') as fp:
data = yaml.load(fp)
for elem in data:
if elem['name'] == 'Stuart':
elem['hobbies'] = ['Fishing']
break # no need to iterate further
yaml.dump(data, sys.stdout)
I get error "TypeError('string indices must be integers',)", I know this code might be completely wrong, but I am new to ruamel.yaml.
How to code this?
The thing missing form the error message displayed is the line number (I assume that it is 9). That points to the line
if elem['name'] == 'Stuart':
And if that doesn't give you a clue, the approach that I recommend in such cases is starting to add some print functions, so that you know what you are working on. The for loop looks like:
for elem in data:
print('elem', elem)
if elem['name'] == 'Stuart':
print('elem->hobbies', elem['hobbies'])
elem['hobbies'] = ['Fishing']
this prints
elem team_member
before the exception is thrown, and I hope that will make you realize your are not iterating over the elements (items) of a list, but over the keys of a dict (constructed from the root level mapping in your YAML). And the value associated with the key is the object having a key name and a key hobbies.
So change the variable elem to key to make clear what you're handling and then proceed to work with value, the value associated with that key instead of elem within that loop¹:
for key in data:
value = data[key]
if value['name'] == 'Stuart':
print('value->hobbies', value['hobbies'])
value['hobbies'] = ['Fishing']
This gives:
value->hobbies ['dancing']
team_member:
name: Max
hobbies:
- Reading
team_leader:
name: Stuart
hobbies:
- Fishing
So we got rid of the exception, but the result is not exactly what you want. The element dancing for the key 'hobbies' is gone, because you assign a new (list) value to that key, whereas what you should do is append a single item to the list. We can also get rid of the print function by now:
for key in data:
value = data[key]
if value['name'] == 'Stuart':
value['hobbies'].append('Fishing')
This will get you two items in the final sequence in the file. There is a few more things to address:
the capitalization of dancing incorrect. To correct that, add a line handling the list if there is only one element
the code for the name Max, needs to be added (and that is why you need to get rid of the break in your code)
the empty line, is considered a comment on the last element of the first sequence, that comment needs to be moved
your indentation of sequences is non-default
The final code would be like:
from pathlib import Path
import ruamel.yaml
path = Path('input.yaml')
yaml = ruamel.yaml.YAML()
yaml.indent(sequence=4, offset=2) # for the non-default indentation of sequences
data = yaml.load(path)
for key in data:
value = data[key]
if value['name'] == 'Stuart':
if len(value['hobbies']) == 1:
value['hobbies'][0] = value['hobbies'][0].capitalize()
value['hobbies'].append('Fishing')
elif value['name'] == 'Max':
last_item_index = len(value['hobbies']) - 1
value['hobbies'].append('Painting')
comments = value['hobbies'].ca
if not comments.items[last_item_index][0].value.strip():
# move empty comment lines from previous last item to new last item
comments.items[last_item_index + 1] = comments.items.pop(last_item_index)
yaml.dump(data, path)
Which gives something quite close to what you wanted to get
team_member:
name: Max
hobbies:
- Reading
- Painting
team_leader:
name: Stuart
hobbies:
- Dancing
- Fishing
¹Alternative for the first two lines: for key, value in data.items()
Thanks Anthon your code worked I have to edit this code as follows:
import sys
import ruamel.yaml
from pathlib import Path
yaml = ruamel.yaml.YAML()
path = Path('input.yaml')
yaml.indent(sequence=4, offset=2) # for the non-default indentation of sequences
with open(path) as fp:
data = yaml.load(fp)
for key in data:
value = data[key]
if value['name'] == 'Stuart':
if len(value['hobbies']) == 1:
value['hobbies'][0] = value['hobbies'][0].capitalize()
value['hobbies'].append('Fishing')
elif value['name'] == 'Max':
last_item_index = len(value['hobbies']) - 1
value['hobbies'].append('Painting')
comments = value['hobbies'].ca
if not comments.items[last_item_index][0].value.strip():
# move empty comment lines from previous last item to new last item
comments.items[last_item_index + 1] = comments.items.pop(last_item_index)
yaml.dump(data, path)