I'm a newbie to python and programming, so this might be easy, but I couldn't find a suitable answer anywhere. I'm trying to do the following. I would like to have a module with a couple of variables in a module that should be modified by my main module.
Additionally it is not clear from the beginning, if the module with the variables and the variable already exists.
At the moment I'm doing the following:
# test2.py
import os
# creates module test1
if os.path.isfile('test1.py') and os.path.getsize('test1.py') > 8:
pass
else:
txt = open('test1.py','w')
txt.write('testvar = {}')
txt.close()
import test1
testvar = test1.testvar
My testmodule looks as follows:
# test.py
import test2
testvar = test2.testvar
# now modify testvar
txt = open('test1.py','w')
txt.write('testvar = '+repr(testvar))
txt.close()
This works, if run test.py in python but it has the drawback, that I need a separate module for any testvar-like variable. I would prefer if I could have a single module with many such variables and use some sort of test1.testvar.update(entry)-method to change the variable.
Furthermore, if I create an exe-file using py2exe the application doesn't recognise the testvar-variable. So there is a further problem in this method.
The reason why I want all this is to change the variable during many different runs of the programme.
Thank you for any suggestions.
You're trying to use Python to dynamically create new Python code, which you then load into your original program and execute...?
This is a recipe for many kinds of major and minor disasters. Don't do this.
If you need to store data in a persistent way from one run of a program to another, there are many good ways to do it. Python's standard shelve module is a very easy way to do it. You basically just open a file and immediately start using it like a dict object which can store (almost) anything else.
import shelve
sh = shelve.open("myshelf")
sh["foo"] = (1,2,3,4)
sh["bar"] = "I like spam"
sh.close()
sh = shelve.open("myshelf")
print sh["foo"]
print sh.keys()
UPDATE: If you want human-readable output files, try using the widely-used JSON serialization format instead.
Unlike shelf, json module requires you to explicitly save and restore a dictionary object.
The JSON format cannot serialize as many data types as shelf without extra code. For example, it can serialize dict/list, but it can't serialize set and will change a tuple to a list.
Same thing using JSON. Notice that the tuple sh["foo"] comes back as a list when it gets serialized and deserialized:
import json
# Load sh from JSON file or create a new dictionary if it doesn't exist
try:
sh = json.load( open("storage.json","r") )
except IOError:
sh = {}
sh["foo"] = (1,2,3,4)
sh["bar"] = "I like spam"
# Save sh to JSON file
json.dump( sh, open("storage.json","w") );
# Reload it
sh = json.load( open("storage.json","r") )
print sh["foo"]
print sh.keys()
Related
Edit: My first attempt at asking this might be a bit unfocused/poorly worded here's a better explanation of what I'm trying to do:
I'm trying to modify the default behavior of the print function for the entire environment python is running in without having to modify each file that's being run.
I'm attempting to decorate the print function (I know there are many ways to do this such as overriding it but that's not really the question I'm asking) so I can have it print out some debugging information and force it to always flush. I did that like so:
def modify_print(func):
# I made this so that output always gets flushed as it won't by default
# within the environment I'm using, I also wanted it to print out some
# debugging information, doesn't really matter much in the context of this
# question
def modified_print(*args,**kwargs):
return func(f"some debug prefix: ",flush=True,*args,**kwargs)
return modified_print
print = modify_print(print)
print("Hello world") # Prints "some debug prefix Hello World"
However what I'm trying to do is modify this behavior throughout my entire application. I know I can manually decorate/override/import the print function in each file however I'm wondering if there is some way I can globally configure my python environment to decorate this function everywhere. The only way I can think to do this would be to edit the python source code and build the modified version.
EDIT:
Here's the behavior I wanted implemented, thank you Match for your help.
It prints out the line number and filename everywhere you call a print function within your python environment. This means you don't have to import or override anything manually in all of your files.
https://gist.github.com/MichaelScript/444cbe5b74dce2c01a151d60b714ac3a
import site
import os
import pathlib
# Big thanks to Match on StackOverflow for helping me with this
# see https://stackoverflow.com/a/48713998/5614280
# This is some cool hackery to overwrite the default functionality of
# the builtin print function within your entire python environment
# to display the file name and the line number as well as always flush
# the output. It works by creating a custom user script and placing it
# within the user's sitepackages file and then overwriting the builtin.
# You can disable this behavior by running python with the '-s' flag.
# We could probably swap this out by reading the text from a python file
# which would make it easier to maintain larger modifications to builtins
# or a set of files to make this more portable or to modify the behavior
# of more builtins for debugging purposes.
customize_script = """
from inspect import getframeinfo,stack
def debug_printer(func):
# I made this so that output always gets flushed as it won't by default
# within the environment I'm running it in. Also it will print the
# file name and line number of where the print occurs
def debug_print(*args,**kwargs):
frame = getframeinfo(stack()[1][0])
return func(f"{frame.filename} : {frame.lineno} ", flush=True,*args,**kwargs)
return debug_print
__builtins__['print'] = debug_printer(print)
"""
# Creating the user site dir if it doesn't already exist and writing our
# custom behavior modifications
pathlib.Path(site.USER_SITE).mkdir(parents=True, exist_ok=True)
custom_file = os.path.join(site.USER_SITE,"usercustomize.py")
with open(custom_file,'w+') as f:
f.write(customize_script)
You can use usercustomize script from the site module to achieve something like this.
First, find out where your user site-packages directory is:
python3 -c "import site; print(site.USER_SITE)"
/home/foo/.local/lib/python3.6/site-packages
Next, in that directory, create a script called usercustomize.py - this script will now be run first whenever python is run.
One* way to replace print is to override the __builtins__ dict and replace it with a new method - something like:
from functools import partial
old_print = __builtins__['print']
__builtins__['print'] = partial(old_print, "Debug prefix: ", flush=True)
Drop this into the usercustomize.py script and you should see all python scripts from then on being overridden. You can temporarily disable calling this script by calling python with the -s flag.
*(Not sure if this is the correct way of doing this - there may be a better way - but the main point is that you can use usercustomize to deliver whatever method you choose).
There's no real reason to define a decorator here, because you are only intending to apply it to a single, predetermined function. Just define your modified print function directly, wrapping it around __builtins__.print to avoid recursion.
def print(*args, **kwargs):
__builtins.__print(f"some debug prefix: ", flush=True, *args, **kwargs)
print("Hello world") # Prints "some debug prefix Hello World"
You can use functools.partial to simplify this.
import functools
print = functools.partial(__builtins.__print, f"some debug prefix: ", flush=True)
I have a Python script that runs 24hrs a day.
A module from this script is using variables values that I wish to change from time to time, without having to stop the script, edit the module file, then launch the script again (I need to avoid interruptions as much as I can).
I thought about storing the variables in a separate file, and the module would, when needed, fetch the new values from the file and use them.
Pickle seemed a solution but is not human readable and therefore not easily changeable. Maybe a JSON file, or another .py file I import over again ?
Another advantage of doing so, for me, is that in case of interruption (eg. server restart), I can resume the script with the latest variable values if I load them from a separate file.
Is there a recommended way of doing such things ?
Something along the lines :
# variables file:
variable1 = 10
variable2 = 25
# main file:
while True:
import variables
print('Sum:', str(variable1+variable2))
time.sleep(60)
An easy way to maintain a text file with variables would be the YAML format. This answer explains how to use it, basically:
import yaml
stream = open("vars.yaml", "r")
docs = yaml.load_all(stream)
If you have more than a few variables, it may be good to check the file descriptor to see if the file was recently updated, and only re-load variables when there was a change in the file.
import os
last_updated = os.path.getmtime('vars.yaml')
Finally, since you want avoid interruption of the script, it may be good to have the script catch any errors in the YAML file and warn the user, instead of just throwing an exception and die. But also remember that "errors should never pass silently". What is the best approach here would depend on your use-case.
I´m starting in python. I have four functions and are working OK. What I want to do is to save them. I want to call them whenever I want in python.
Here's the code my four functions:
import numpy as ui
def simulate_prizedoor(nsim):
sim=ui.random.choice(3,nsim)
return sims
def simulate_guess(nsim):
guesses=ui.random.choice(3,nsim)
return guesses
def goat_door(prizedoors, guesses):
result = ui.random.randint(0, 3, prizedoors.size)
while True:
bad = (result == prizedoors) | (result == guesses)
if not bad.any():
return result
result[bad] = ui.random.randint(0, 3, bad.sum())
def switch_guesses(guesses, goatdoors):
result = ui.random.randint(0, 3, guesses.size)
while True:
bad = (result == guesses) | (result == goatdoors)
if not bad.any():
return result
result[bad] = ui.random.randint(0, 3, bad.sum())
What you want to do is to take your Python file, and use it as a module or a library.
There's no way to make those four functions automatically available, no matter what, 100% percent of the time, but you can do something very close.
For example, at the top of your file, you imported numpy. numpy is a module or library which has been set up so it's available any time you run python, as long as you import it.
You want to do the same thing -- save those 4 functions into a file, and import them whenever you want them.
For example, if you copy and paste those four functions into a file named foobar.py, then you can simply do from foobar import *. However, this will only work if you're running Python in the same folder where you saved your code.
If you want to make your module available system-wide, you have to save it somewhere on the PYTHONPATH. Usually, saving it to C:\Python27\Lib\site-packages will work (assuming you're running Windows).
If you decide to put them anywhere in your project folder don`t forget to create a blank init.py file so python can see them. A better answer can be provided here : http://docs.python.org/2/tutorial/modules.html
Save them in a file - this makes them a module.
If you put them in a file called mymod.py, in python you can load them as follows
from mymod import *
simulate_prizedoor(23)
Quick solution, without having to explicitly create a file - relies on IPython and its storemagic
IPython 4.0.1 -- An enhanced Interactive Python.
details.
In [1]: def func(a):
...: print a
...:
In [2]: func = _i #gets the previous input
In [3]: store func #store(magic) the input
#(auto-magic enabled or would need '%store')
Stored 'func' (unicode)
In [4]: exit
IPython 4.0.1 -- An enhanced Interactive Python.
In [1]: store -r func #retrieve stored string
In [2]: exec func #execute string as python code
In [3]: func(10)
10
Once you had stored all your functions just once, then you can restore them all with store -r, and then exec func once for each function, in each new session.
(Came across this question while looking for a solution for 'quick saving' functions (most convenient way) while in an interactive python session - adding my current best solution for future readers)
I've got a python package which outputs considerable help text from: help(package)
I would like to export this help text to a file, in the format in which it's displayed by help(package)
How might I go about this?
pydoc.render_doc(thing) to get thing's help text as a string. Other parts of pydoc like pydoc.text and pydoc.html can help you write it to a file.
Using the -w modifier in linux will write the output to a html in the current directory, for example;
pydoc -w Rpi.GPIO
Puts all the help() text that would be presented from the command help(Rpi.GPIO) into a nicely formatted file Rpi.GPIO.html, in the current directory of the shell
This is a bit hackish (and there's probably a better solution somewhere), but this works:
import sys
import pydoc
def output_help_to_file(filepath, request):
f = open(filepath, 'w')
sys.stdout = f
pydoc.help(request)
f.close()
sys.stdout = sys.__stdout__
return
And then...
>>> output_help_to_file(r'test.txt', 're')
An old question but the newer recommended generic solution (for Python 3.4+) for writing the output of functions that print() to terminal is using contextlib.redirect_stdout:
import contextlib
def write_help(func, out_file):
with open(out_file, 'w') as f:
with contextlib.redirect_stdout(f):
help(func)
Usage example:
write_help(int, 'test.txt')
To get a "clean" text output, just as the built-in help() would deliver, and suitable for exporting to a file or anything else, you can use the following:
>>> import pydoc
>>> pydoc.render_doc(len, renderer=pydoc.plaintext)
'Python Library Documentation: built-in function len in module builtins\n\nlen(obj, /)\n Return the number of items in a container.\n'
If you do help(help) you'll see:
Help on _Helper in module site object:
class _Helper(__builtin__.object)
| Define the builtin 'help'.
| This is a wrapper around pydoc.help (with a twist).
[rest snipped]
So - you should be looking at the pydoc module - there's going to be a method or methods that return what help(something) does as a string...
Selected answer didn't work for me, so I did a little more searching and found something that worked on Daniweb. Credit goes to vegaseat. https://www.daniweb.com/programming/software-development/threads/20774/starting-python/8#post1306519
# simplified version of sending help() output to a file
import sys
# save present stdout
out = sys.stdout
fname = "help_print7.txt"
# set stdout to file handle
sys.stdout = open(fname, "w")
# run your help code
# its console output goes to the file now
help("print")
sys.stdout.close()
# reset stdout
sys.stdout = out
The simplest way to do that is via using
sys module
it opens a data stream between the operation system and it's self , it grab the data from the help module then save it in external file
file="str.txt";file1="list.txt"
out=sys.stdout
sys.stdout=open('str_document','w')
help(str)
sys.stdout.close
The cleanest way
Assuming help(os)
Step 1 - In Python Console
import pydoc
pydoc.render_doc(os, renderer=pydoc.plaintext)`
#this will display a string containing help(os) output
Step 2 - Copy string
Step 3 - On a Terminal
echo "copied string" | tee somefile.txt
If you want to write Class information in a text file. Follow below steps
Insert pdb hook somewhere in the Class and run file
import pdb; pdb.set_trace()
Perform step 1 to 3 stated above
In Windows, just open up a Windows Command Line window, go to the Lib subfolder of your Python installation, and type
python pydoc.py moduleName.memberName > c:\myFolder\memberName.txt
to put the documentation for the property or method memberName in moduleName into the file memberName.txt. If you want an object further down the hierarchy of the module, just put more dots. For example
python pydoc.py wx.lib.agw.ultimatelistctrl > c:\myFolder\UltimateListCtrl.txt
to put the documentation on the UltimateListCtrl control in the agw package in the wxPython package into UltimateListCtrl.txt.
pydoc already provides the needed feature, a very well-designed feature that all question-answering systems should have. The pydoc.Helper.init has an output object, all output being sent there. If you use your own output object, you can do whatever you want. For example:
class OUTPUT():
def __init__(self):
self.results = []
def write(self,text):
self.results += [text]
def flush(self):
pass
def print_(self):
for x in self.results: print(x)
def return_(self):
return self.results
def clear_(self):
self.results = []
when passed as
O = OUTPUT() # Necessarily to remember results, but see below.
help = pydoc.Helper(O)
will store all results in the OUTPUT instance. Of course, beginning with O = OUTPUT() is not the best idea (see below). render_doc is not the central output point; output is. I wanted OUTPUT so I could keep large outputs from disappearing from the screen using something like Mark Lutz' "More". A different OUTPUT would allow you to write to files.
You could also add a "return" to the end of the class pydoc.Helper to return the information you want. Something like:
if self.output_: return self.output_
should work, or
if self.output_: return self.output.return_()
All of this is possible because pydoc is well-designed. It is hidden because the definition of help leaves out the input and output arguments.
Using the command line we can get the output directly and pipe it to whatever is useful.
python -m pydoc ./my_module_file.py
-- the ./ is important, it tells pydoc to look at your local file and not attempt to import from somewhere else.
If you're on the mac you can pipe the output to pbcopy and paste it into a documentation tool of your choice.
python -m pydoc ./my_module_file.py | pbcopy
I want to input code in Python like \input{Sources/file.tex}. How can I do it in Python?
[added]
Suppose I want to input data to: print("My data is here"+<input data here>).
Data
1, 3, 5, 5, 6
The built-in execfile function does what you ask, for example:
filename = "Sources/file.py"
execfile( filename )
This will execute the code from Sources/file.py almost as if that code were embedded in the current file, and is thus very similar to #include in C or \input in LaTeX.
Note that execfile also permits two optional arguments allowing you to specify the globals and locals dicts that the code should be executed with respect to, but in most cases this is not necessary. See pydoc execfile for details.
There are occasional legitimate reasons to want to use execfile. However, for the purpose of structuring large Python programs, it is conventional to separate your code into modules placed somewhere in the PYTHONPATH and to load them using the import statement rather than executing them with execfile. The advantages of import over execfile include:
Imported functions get qualified with the name of the module, e.g. module.myfunction instead of just myfunction.
Your code doesn't need to hard-code where in the filesystem the file is located.
You can't do that in Python. You can import objects from other modules.
otherfile.py:
def print_hello():
print "Hello World!"
main.py
import otherfile
otherfile.print_hello() # prints Hello World!
See the python tutorial
Say you have code in "my_file.py". Any line which is not in a method WILL get executed when you do:
import my_file
So for example if my_file.py has the following code in it:
print "hello"
Then in the interpreter you type:
import my_file
You will see "hello".
My question was clearly too broad, as the variety of replies hint -- none of them fully attack the question. The jchl targets the scenario where you get python-code to be executed. The THC4k addresses the situation where you want to use outside objects from modules. muckabout's reply is bad practice, as Xavier Ho mentioned, why on earth it uses import when it could use exec as well, the principle of least privileges to the dogs. One thing is still missing, probably because of the conflict between the term python-code in the title and the addition of data containing integers -- it is hard to claim that data is python-code but the code explains how to input data, evaluations and executable code.
#!/usr/bin/python
#
# Description: it works like the input -thing in Tex,
# you can fetch outside executable code, data or anything you like.
# Sorry I don't know precisely how input(things) works, maybe misusing terms
# or exaggerating.
#
# The reason why I wanted input -style thing is because I wanted to use more
# Python to write my lab-reports. Now, I don't need to mess data with
# executions and evalutions and data can be in clean files.
#TRIAL 1: Execution and Evaluation not from a file
executeMe="print('hello'); a = 'If you see me, it works'";
exec( executeMe )
print(a);
#TRIAL 2: printing file content
#
# and now with files
#
# $ cat IwillPrint007fromFile
# 007
f = open('./IwillPrint007fromFile', 'r');
msg = f.read()
print("If 007 == " + msg + " it works!");
# TRIAL 3: Evaluation from a file
#
# $cat IwillEvaluateSthing.py
# #!/usr/bin/python
# #
# # Description:
#
#
# evaluateMe = "If you see me again, you are breaking the rules of Sky."
f = open('./IwillEvaluateSthing.py', 'r');
exec(f.read());
print(evaluateMe);