I have a module that needs to update new variable values from the web, about once a week. I could place those variable values in a file & load those values on startup. Or, a simpler solution would be to simply auto-update the code.
Is this possible in Python?
Something like this...
def self_updating_module_template():
dynamic_var1 = {'dynamic var1'} # some kind of place holder tag
dynamic_var2 = {'dynamic var2'} # some kind of place holder tag
return
def self_updating_module():
dynamic_var1 = 'old data'
dynamic_var2 = 'old data'
return
def updater():
new_data_from_web = ''
new_dynamic_var1 = new_data_from_web # Makes API call. gets values.
new_dynamic_var2 = new_data_from_web
# loads self_updating_module_template
dynamic_var1 = new_dynamic_var1
dynamic_var2 = new_dynamic_var2
# replace module place holders with new values.
# overwrite self_updating_module.py.
return
I would recommend that you use configparser and a set of default values located in an ini-style file.
The ConfigParser class implements a basic configuration file parser
language which provides a structure similar to what you would find on
Microsoft Windows INI files. You can use this to write Python programs
which can be customized by end users easily.
Whenever the configuration values are updated from the web api endpoint, configparser also lets us write those back out to the configuration file. That said, be careful! The reason that most people recommend that configuration files be included at build/deploy time and not at run time is for security/stability. You have to lock down the endpoint that allows updates to your running configuration in production and have some way to verify any configuration value updates before they are retrieved by your application:
import configparser
filename = 'config.ini'
def load_config():
config = configparser.ConfigParser()
config.read(filename)
if 'WEB_DATA' not in config:
config['WEB_DATA'] = {'dynamic_var1': 'dynamic var1', # some kind of place holder tag
'dynamic_var2': 'dynamic var2'} # some kind of place holder tag
return config
def update_config(config):
new_data_from_web = ''
new_dynamic_var1 = new_data_from_web # Makes API call. gets values.
new_dynamic_var2 = new_data_from_web
config['WEB_DATA']['dynamic_var1'] = new_dynamic_var1
config['WEB_DATA']['dynamic_var2'] = new_dynamic_var2
def save_config(config):
with open(filename, 'w') as configfile:
config.write(configfile)
Example usage::
# Load the configuration
config = load_config()
# Get new data from the web
update_config(config)
# Save the newly updated configuration back to the file
save_config(config)
Related
My project is to make a live graph according to an updating log file generated from the simpy module, but I don't know how to make the st.line_chart -> inf_chart to be a variable so that I can use add_rows to update the chart.
In mainpage.py
if st.button('Run simulator'):
...
infrastructure, application = st.columns(2)
empty = ['']
empty_dataframe = pd.DataFrame(empty,columns=[''])
...
inf_chart = infrastructure.line_chart(empty_dataframe,width=500)
app_chart = application.line_chart(empty_dataframe,width=500)
In UI.py
import mainpage
def draw_infrastructure(element,i,columns):
df = pd.DataFrame(element,index=[i],columns=[columns])
inf_chart.add_rows(df)
draw_infrastructure runs every time when the log file updates, but it becomes two same pages when I import mainpage in UI.py. Is there any solution to pass this variable inf_chart to another file?
I have this following python code for a Flask server. I am trying to have this part of the code list all my vehicles that match the horsepower that I put in through my browser. I want it to return all the car names that match the horsepower, but what I have doesn't seem to be working? It returns nothing. I know the issue is somewhere in the "for" statement, but I don't know how to fix it.
This is my first time doing something like this and I've been trying multiple things for hours. I can't figure it out. Could you please help?
from flask import Flask
from flask import request
import os, json
app = Flask(__name__, static_folder='flask')
#app.route('/HORSEPOWER')
def horsepower():
horsepower = request.args.get('horsepower')
message = "<h3>HORSEPOWER "+str(horsepower)+"</h3>"
path = os.getcwd() + "/data/vehicles.json"
with open(path) as f:
data = json.load(f)
for record in data:
horsepower=int(record["Horsepower"])
if horsepower == record:
car=record["Car"]
return message
The following example should meet your expectations.
from flask import Flask
from flask import request
import os, json
app = Flask(__name__)
#app.route('/horsepower')
def horsepower():
# The type of the URL parameters are automatically converted to integer.
horsepower = request.args.get('horsepower', type=int)
# Read the file which is located in the data folder relative to the
# application root directory.
path = os.path.join(app.root_path, 'data', 'vehicles.json')
with open(path) as f:
data = json.load(f)
# A list of names of the data sets is created,
# the performance of which corresponds to the parameter passed.
cars = [record['Car'] for record in data if horsepower == int(record["Horsepower"])]
# The result is then output separated by commas.
return f'''
<h3>HORSEPOWER {horsepower}</h3>
<p>{','.join(cars)}<p>
'''
There are many different ways of writing the loop. I used a short variant in the example. In more detail, you can use these as well.
cars = []
for record in data:
if horsepower == int(record['Horsepower']):
cars.append(record['Car'])
As a tip:
Pay attention to when you overwrite the value of a variable by using the same name.
I have the following code to enable the file browser using blender:
import bpy
import os
from bpy.props import StringProperty
from bpy_extras.io_utils import ImportHelper
from bpy.types import Operator
sel = ''
class OpenBrowser(bpy.types.Operator):
bl_idname = "open.file"
bl_label = "Select Excel File"
bli_description = "Simulation output excel file"
filter_glob: StringProperty(default = '*.xls;*.xlsx',options = {'HIDDEN'})
filepath: bpy.props.StringProperty(subtype="FILE_PATH")
#somewhere to remember the address of the file
def execute(self, context):
global sel
sel = self.filepath
#self.selected_file = self.filepath
#display = "filepath= "+self.filepath
#print(display) #Prints to console
#Window>>>Toggle systen console
return {'FINISHED'}
def invoke(self, context, event): # See comments at end [1]
context.window_manager.fileselect_add(self)
global sel
sel = self.filepath
#Open browser, take reference to 'self'
#read the path to selected file,
#put path in declared string type data structure self.filepath
return {'RUNNING_MODAL'}
# Tells Blender to hang on for the slow user input
bpy.utils.register_class(OpenBrowser)
#Tell Blender this exists and should be used
# [1] In this invoke(self, context, event) is being triggered by the below command
#but in your script you create a button or menu item. When it is clicked
# Blender runs invoke() automatically.
#execute(self,context) prints self.filepath as proof it works.. I hope.
bpy.ops.open.file('INVOKE_DEFAULT')
print(sel)
The issue I am facing is that I have declared a global variable sel to which I want to save the filepath selected from the user when running the code. However, when I run the script I see that sel has not changed and it is as it was initialized. Could someone please help me on how to access from the class the self.filepath variable? What am I doing wrong here?
If I understand correctly, you want to store that value for later.
I'm not sure why 'sel' doesn't even update in your case, but I think the more correct way would be to use a property like so:
import bpy
# Assign a custom property to an existing type.
bpy.types.Scene.my_sel_value = bpy.props.StringProperty(name="Sel")
# Set property value.
bpy.context.scene.my_sel_value = "Foo"
# Get property value.
print(bpy.context.scene.my_sel_value)
Properties can be added to all ID types, but for "global" values, bpy.types.scene us ussualy used. Though there can be multiple scenes in one project and they will have separate values. Property values are stored when Blender closes.
If you are making an addon, you can also store your value in Addon Preferences. This value will be the same for all blender projects.
Observe the following Python file:
# configmanager.py
"""
ConfigManager controls the modification and validation of config files.
"""
import os
from ruamel import yaml
from voluptuous import Schema
class ConfigManager():
"""
Controls all interaction with configuration files
"""
def __init__(self):
super().__init__()
self.configvalidator = ConfigValidator()
# The config directory inside users home directory.
# Config files will be stored here.
config_dir = os.path.expanduser('~')+'/.config/MyProject/'
# The default config file
config_file = config_dir+'myproject.conf'
# The default configuration
default_config = {
'key1': {},
'key2': {}
}
def _get_config(self):
"""
Get the config file and return it as python dictionary.
Will create the config directory and default config file if they
do not exist.
"""
# Create config directory if it does not exist
if not os.path.exists(self.config_dir):
os.makedirs(self.config_dir)
# Create default config file if it does not exist
if not os.path.isfile(self.config_file):
config_file = open(self.config_file, 'w')
config_file.write(yaml.dump(self.default_config))
# Open config file, and load from YAML
config_file = open(self.config_file, 'r')
config = yaml.safe_load(config_file)
# Validate config
self.configvalidator.validate(config)
return config
def _save_config(self, config):
"""
Save the config file to disk as YAML
"""
# Open current config file
config_file = open(self.config_file, 'w')
# Validate new config
# THE ERROR IS HERE
# If this runs then the config file gets saved as an empty file.
self.configvalidator.validate(config)
# This shows that the 'config' variable still holds the data
print(config)
# This shows that yaml.dump() is working correctly
print(yaml.dump(config))
config_file.write(yaml.dump(config))
def edit_config(self):
"""
Edits the configuration file
"""
config = self._get_config()
# Perform manipulation on config here
# No actual changes to the config are necessary for the bug to occur
self._save_config(config)
class ConfigValidator():
def __init__(self):
super().__init__()
# Config file schema
# Used for validating the config file with voluptuous
self.schema = Schema({
'key1': {},
'key2': {},
})
def validate(self, config):
"""
Validates the data against the defined schema.
"""
self.schema(config)
app = ConfigManager()
app.edit_config()
-
# ~/.config/MyProject/myproject.conf
key1: {}
key2: {}
Description of my module
This is a module I am working on which is for modifying the config file for my project. It accesses the file in ~/.config/MyProject/myproject.conf, which is saved in YAML format, and stores various pieces of information that are used by my program. I have removed as much of the code as I can, leaving only that necessary for understanding the bug.
ConfigManager
ConfigManager is the class containing methods for manipulating my config file. Here it contains three methods: _get_config(), _save_config(), and edit_config(). When instantised, it will get an instance of ConfigValidator (described below), and assign it to self.configvalidator.
_get_config
_get_config() simply opens the file defined by the class variables, specifically ~/.config/MyProject/myproject.conf, or creates the file with default values if it does not exist. The file is saved in YAML format, so this method loads it into a python object, using ruamel.yaml, validates it using self.configvalidator.validate(config) and returns it for use by other pieces of code.
_save_config
_save_config() is where the error occurs, which is described in detail below. It's purpose is to validate the given data, and if it is valid, save it to disk in YAML format.
edit_config
This a generic function, which, in my program, would make specific changes to my config file, depending on the arguments given. In my example, this function simply gets the config file with self._get_config(), and then saves it using self._save_config, without making any changes.
ConfigValidator
This class is for validating my config file using voluptuous. When instantised, it will create the schema that is to be used, and assign it to self.schema. When the validate method is run, it validates the given data using voluptuous.
The error
Observe the line self.configvalidator.validate(config) in ConfigManager._save_config(). This will validate the given data against the schema, and raise an error if it does not pass validation.
But, in the following line config_file.write(yaml.dump(config)), which simply saves the given data to a file as YAML, it will instead save an empty file to disk. (Note: the file is empty, not deleted)
If I disable the validation, by commenting out self.configvalidator.validate(config), then the file is written correctly as YAML.
If self.configvalidator.validate(config) is run, then the config file is saved as an empty file.
My testing
As can be seen with the line print(config), the data in the variable config does not change after being used for validation, yet when being saved to disk, it would seem that config is an empty variable.
print(yaml.dump(config)) shows that ruamel.yaml does work correctly.
If I change edit_config to give invalid data to _save_config, then self.configvalidator.validate(config) will raise an error, as expected. self.configvalidator.validate(config) is running correctly.
End
If self.configvalidator.validate(config) is run, then config_file.write(yaml.dump(config)) saves the config file as an empty file, despite the data in the variable config not changing.
If self.configvalidator.validate(config) is not run, then config_file.write(yaml.dump(config)) saves the file correctly.
That is my error, and it makes absolutely no sense to me.
If your keen to help, then configmanager.py should run correctly (with the error) on your machine, as long as it has access to ruamel.yaml and voluptuous. It will create ~/.config/MyProject/myproject.conf, then save it as empty. Save my example myproject.conf to see how it is then saved as empty when configmanager.py is run. If configmanager.py is run again, when myproject.conf is empty, then a validation error will be raised in _get_config, as expected.
I am so confused by this bug, so if you have any insight it would be greatly appreciated.
Cheers
After reading the config file:
f = "excfg.cfg"
settings = fix.SessionSettings(f)
Is it possible to modify the settings of a particular session dynamically? For example modifying the TargetCompID or the SocketAcceptPort.
I am pretty much looking for an alternative to this:
dictionary = settings.get()
id1 = fix.SessionID(dictionary.getString(fix.BEGINSTRING),
dictionary.getString(fix.SENDERCOMPID), "EX1")
dictionary.setInt(fix.SOCKET_ACCEPT_PORT, 7001)
settings.set(id1, dictionary)
Where I don't create a new SessionID but modify the existing one from the config file, before the SocketAcceptor/SocketInitiator function is called.
You don't need to execute settings.set, since the dictionary returned by settings.get contains references (instead of copies) to the sessions settings:
import quickfix as fix
# print the current port
settings = fix.SessionSettings('/opt/robotfx/etc/fxgo-config/config.txt', True)
session_settings = settings.get(fix.SessionID('FIX.4.4','SENDER','TARGET'))
print('current port=', session_settings.getString('SocketConnectPort'))
# updates the port value, get the session settings again and print the current port
session_settings.setString('SocketConnectPort', '1234')
session_settings = settings.get(fix.SessionID('FIX.4.4','SENDER','TARGET'))
print('current port=', session_settings.getString('SocketConnectPort'))
output:
current port=31337
current port=1234