Develop Reference

How to combine YAML files in python?

I got some Kubernetes YAML files which I need to combine.
For that, I tried using Python.
The second file, sample.yaml, should be merged to the first file, source.yaml.
The source.yaml file has one section sample:, where the complete sample.yaml should be added.
I tried using the below code:
#pip install pyyaml
import yaml
def yaml_loader(filepath):
#Loads a yaml file
with open(filepath,'r')as file_descriptor:
data = yaml.load(file_descriptor)
return data
def yaml_dump(filepath,data):
with open(filepath,"w") as file_descriptor:
yaml.dump(data, file_descriptor)
if __name__ == "__main__":
file_path1 = "source"
data1 = yaml_loader(file_path1)
file_path2 = "sample.yaml"
with open(file_path2, 'r') as file2:
sample_yaml = file2.read()
data1['data']['sample'] = sample_yml
yaml_dump("temp.yml", data1)
This is creating a new file temp.yml but instead of line breaks, it is saving \n as strings:
How to fix this?

Your original YAML may have issues. If you use VS Code, format your YAML file. Click on the bottom of vscode(if using the same) [Spaces]
and select convert indentation to spaces
also, you can check if YAML module has any indentation property to be configured ,when loading the file

creating and using a preferences file in python

Brand new to stack and python; hopefully someone wiser than myself can help. I have searched up and down and can't seem to find an actual answer to this, apologies if there is an exact answer and I've missed it :( (the few that I've found are either old or don't seem to work).
Closest I've found is
Best way to retrieve variable values from a text file?
Alas, imp seems to be depreciated and tried figuring out importlib but little above my current brain to figure out how to adapt it as errors throw up left and right on me.
This is very close to what I want and could potentially work if someone can help update with new methods, alas still doesn't have how to overwrite the old variable.
= - - Scenario - - =
I would like to create a preferences file (let's call it settings.txt or settings.py: doesn't need to be cross-compatible with other languages, but some reason I'd prefer txt - any preference/standards coders can impart would be appreciated?).
\\\ settings.txt\
water_type = "Fresh"\
measurement = "Metric"\
colour = "Blue"\
location = "Bottom"\
...
I am creating a script main_menu.py which will read variables in settings.txt and write to this file if changes are 'saved'
ie.
"select water type:"
Fresh
Salt
if water_type is the same as settings.txt, do nothing,
if water_type different, overwrite the variable in the settings.txt file
Other scripts down the line will also read and write to this settings file.
I've seen:
from settings import *
Which seems to work for reading the file if I go the settings.py path but still leaves me on how do I overwrite this.
also open to any better/standard/ideas you guys can think of.
Appreciate any help on this!

Here are some suggestions that may help you:
Use a json file:
settings.json
{
"water_type": "Fresh",
"measurement": "Metric",
"colour": "Blue",
"location": "Bottom",
...
}
then in python:
import json
# Load data from the json file
with open("settings.json", "r") as f:
x = json.load(f) # x is a python dictionary in this case
# Change water_type in x
x["water_type"] = "Salt"
# Save changes
with open("settings.json", "w") as f:
json.dump(x, f, indent=4)
Use a yaml file: (edit: you will need to install pyyaml)
settings.yaml
water_type: Fresh
measurement: Metric
colour: Blue
location: Bottom
...
then in python:
import yaml
# Load data from the yaml file
with open("settings.yaml", "r") as f:
x = yaml.load(f, Loader=yaml.FullLoader) # x is a python dictionary in this case
# Change water_type in x
x["water_type"] = "Salt"
# Save changes
with open("settings.yaml", "w") as f:
yaml.dump(x, f)
Use a INI file:
settings.ini
[Preferences]
water_type=Fresh
measurement=Metric
colour=Blue
location=Bottom
...
then in python:
import configparser
# Load data from the ini file
config = configparser.ConfigParser()
config.read('settings.ini')
# Change water_type in config
config["Preferences"]["water_type"] = "Salt"
# Save changes
with open("settings.ini", "w") as f:
config.write(f)

For .py config files, it's usually static options or settings.
Ex.
# config.py
STRINGTOWRITE01 = "Hello, "
STRINGTOWRITE02 = "World!"
LINEENDING = "\n"
It would be hard to save changes made to the settings in such a format.
I'd recommend a JSON file.
Ex. settings.json
{
"MainSettings": {
"StringToWrite": "Hello, World!"
}
}
To read the settings from this file into a Python Dictionary, you can use this bit of code.
import json # Import pythons JSON library
JSON_FILE = open('settings.json','r').read() # Open the file with read permissions, then read it.
JSON_DATA = json.loads(JSON_FILE) # load the raw text from the file into a json object or dictionary
print(JSON_DATA["MainSettings"]["StringToWrite"]) # Access the 'StringToWrite' variable, just as you would with a dictionary.
To write to the settings.json file you can use this bit of code
import json # import pythons json lib
JSON_FILE = open('settings.json','r').read() # Open the file with read permissions, then read it.
JSON_DATA = json.loads(JSON_FILE) # load the data into a json object or dictionary
print(JSON_DATA["MainSettings"]["StringToWrite"]) # Print out the StringToWrite "variable"
JSON_DATA["MainSettings"]["StringToWrite"] = "Goodnight!" # Change the StringToWrite
JSON_DUMP = json.dumps(JSON_DATA) # Turn the json object or dictionary back into a regular string
JSON_FILE = open('settings.json','w') # Reopen the file, this time with read and write permissions
JSON_FILE.write(JSON_DUMP) # Update our settings file, by overwriting our previous settings
Now, I've written this so that it is as easy as possible to understand what's going on. There are better ways to do this with Python Functions.

You guys are fast! I'm away from the computer for the weekend but had to log in just to say thanks.
I'll look into these more next week when I'm back at it and have some time to give it the attention needed. A quick glance could be a bit of fun to implement and learn a bit more.
Had to answer as adding comment only is on one of your guys solutions and wanted to give a blanket thanks to all!
Cheers

Here's a python library if you choose to do it this way.
If not this is also a good resource.
Creating a preferences file example
Writing preferences to file from python file
import json
# Data to be written
dictionary ={
"name" : "sathiyajith",
"rollno" : 56,
"cgpa" : 8.6,
"phonenumber" : "9976770500"
}
# Serializing json
json_object = json.dumps(dictionary, indent = 4)
# Writing to sample.json
with open("sample.json", "w") as outfile:
outfile.write(json_object)
Reading preferences from .json file in Python
import json
# open and read file content
with open('sample.json') as json_file:
data = json.load(json_file)
# print json file
print(data)

Reading config file that does not begin with section

I have config file that looks something like:
#some text
host=abc
context=/abc
user=abc
pw=abc
#some text
[dev]
host=abc
context=/abc
user=abc
pw=abc
[acc]
host=abc
context=/abc
user=abc
pw=abc
I would like to parse the cfg file with ConfigParser in Python 2.7. The problem is that the cfg file does not start with the section. I cannot delete the text lines before the sections. Is there any workaround for that?

Inject a section header of your choice.
import ConfigParser
import StringIO
def add_header(cfg, hdr="DEFAULT"):
s = StringIO.StringIO()
s.write("[{}]\n".format(hdr))
for line in cfg:
s.write(line)
s.seek(0)
return s
parser = ConfigParser.ConfigParser()
with open('file.cfg') as cfg:
parser.readfp(add_header(cfg, "foo"))

How do I load data from a text file in python?

I'm basically writing text to a file for example
data = ("save.data", "a+")
data.write(u"name = 'zrman'")
I wan't to be able to load that file and allow me to do this in python
print name
Any help would be great
-Thx

Use ConfigParser Python module. It's exactely what you're looking for.
Write :
import ConfigParser
config = ConfigParser.RawConfigParser()
config.set('main', 'name', 'zrman')
with open('conf.ini', 'wb') as configfile:
config.write(configfile)
Read :
from ConfigParser import ConfigParser
config = ConfigParser()
config.read('conf.ini')
print config.sections()
# ['main']
print config.items('main')
# [('name', 'zrman')]

Take a look at the docs here, this will walk you through the process of reading and writing files in python.

You could read in the lines and then exec the code:
f = open('workfile', 'w')
for line in f:
exec(line)
print name

Change value in ini file using ConfigParser Python

So, I have this settings.ini :
[SETTINGS]
value = 1
And this python script
from ConfigParser import SafeConfigParser
parser = SafeConfigParser()
parser.read('settings.ini')
print parser.get('SETTINGS', 'value')
As you can see, I want to read and then replace the value "1" by another one. All I was able to do so far is to read it. I searched on the net how to replace it but I didn't find.

As from the examples of the documentation:
https://docs.python.org/2/library/configparser.html
parser.set('SETTINGS', 'value', '15')
# Writing our configuration file to 'example.ini'
with open('example.ini', 'wb') as configfile:
parser.write(configfile)

Python's official docs on configparser illustrate how to read, modify and write a config-file.
import configparser
config = configparser.ConfigParser()
config.read('settings.ini')
config.set('SETTINGS', 'value','15')
with open('settings.ini', 'w') as configfile:
config.write(configfile)

I had an issue with:with open
Other way:
import configparser
def set_value_in_property_file(file_path, section, key, value):
config = configparser.RawConfigParser()
config.read(file_path)
config.set(section,key,value)
cfgfile = open(file_path,'w')
config.write(cfgfile, space_around_delimiters=False) # use flag in case case you need to avoid white space.
cfgfile.close()
It can be used for modifying java properties file: file.properties

Below example will help change the value in the ini file:
PROJECT_HOME="/test/"
parser = ConfigParser()
parser.read("{}/conf/cdc_config.ini".format(PROJECT_HOME))
parser.set("default","project_home",str(PROJECT_HOME))
with open('{}/conf/cdc_config.ini'.format(PROJECT_HOME), 'w') as configfile:
parser.write(configfile)
[default]
project_home = /Mypath/