I want to keep a piece of code in a txt file and execute it from my Python script.
For example in the txt file there is
print("ok")
I want my programme to print ok and not print print("ok "). How can I do this?
Doing what you want usually is a security risk, but not necessarily so.
You'll definitely have to notify the user about potential risk.
There is more than one program using execfile() or compile() and exec statement to provide plug-ins system.
There is nothing so ugly about it, you just have to know what are you doing, when and where.
Both execfile(), eval() and exec statement allow you to specify a scope in which will your code be executed/evaluated.
myscope = {}
execfile("myfile.txt", myscope)
This will prevent the new code being mixed with the old one. All variables classes, functions and modules from myfile.txt will be contained in myscope dictionary.
This, however, does not prevent malicious code to delete all files it can from your disk or something similar.
Python 2 has a nice module called rexec, but from Python 2.2 it doesn't work any more.
It implements execfile() exec statement and eval() in restricted environment.
Though it doesn't work it is there, and you can dig through the code to see how it is done.
So, you see, it is possible to allow only secure code to execute (well, as secure as it can be) from external source.
There is another way too.
You can load the file, compile the code from it and then inspect it to see what it does. Then say, yes I'll execute it or no, I won't. This is, however, a little bit more work and a lot of complications.
But, I don't think that it'll be necessary to go through all that stuff. Please elaborate some more on your problem. What exactly do you mean by level editor?
I don't think external code is a solution for that.
You are looking for eval function
eval is a function that takes arbitrary python code represented as a string and execute it at run time.
Example
>>> x = 1
>>> y = eval('x+1')
>>> print(y)
2
It works in both Python 2.x and 3.x
Check the documentation : https://docs.python.org/2.7/library/functions.html#eval
So I wanted to do the same thing. I came across this: GeeksforGeeks.
With this we could make a text file. Let's say it's called myfile.txt and in the first line we will add print("ok") in the second add A += 1.
Now let's move over to the script editor.
# open txt file
f = open("myfile.txt", "r")
# add the data in the file the a var
data = f.readlines()
# remove unwanted \n for new lines and '' left over by the code above
# readlines() returns a list so we need to convert data to str
# in data[] add the line you wish to read from
info = str(data[0]).strip("\n").strip("'")
# run the code
exec(info)
# running the A += 1 in the txt file
# run this
A = 0
info = str(data[1]).strip("\n").strip("'")
while A == 0:
print(A)
exec(info)
print(A)
# if you wanted too you can even define a variable with this
alist = ["B = 0", "B += 1", "print(B)"]
runner = [0, 1, 2, 1, 2, 0, 2]
for i in range(len(runner)):
exec(alist[int(runner[i])])
Related
lets say i have a folder which contains the following files:
f1.py
f2.py
f3.py
in f1.py i got this code:
#O = "Random string"
print("ABCD")
#P = "Random string"
but in f2.py and f3.py i have this code:
#M = "Random string"
print("EFGH")
#Z = "Random string"
And i want to change the strings in the 'print' function in f2.py and f3.py to the string i have in print in f1.py, and run all the files in the folder after changing the strings, using f1.py
It would be best to have more context why you want to do this.
This is possible, but in 99% of the cases it's not a god idea to write self modifying code, though it can be a lot of fun.
In fact you do not really write self modifying code, but more one piece of code modifying other files. But this is also rarely to be recommended.
What's more usual is, that one script analyzes / parses f1.py, extracts the data writes some data into a file (e.g. a json file)
and f2.py and f3.py read the data from that file and do then print this data.
Is there a particular reason you want to have code, that is modifying other python files.
If you really want to have f2.py and f3.py modified, then there is another solution, which is called templating (you can for example use Jinji).
In this case you have two template files f2.py.template and f3.py.template.
you write a file parsing f1.py, extracting the data and creates f2.py from f2.py.template and the extracted data. (Same for f3.py.template and f3.py)
If you're really 100% sure, that you really want what you ask for.
Yes it is possible:
you write a script, tht opens and reads f1.py line by line, looks for the line "#O = ", then memorizes the next line.
Then it reads f2.py line by line and writes it to another file (e.g. next_version_of_f2.py). it reads in a line and writes it out until it encounters the line #M = "Random string in f2.py In this case the line will be written out, the desired print will be written out, the print line from f2.py will be read and ignored and then you read and write all the other lines.
Then you close f2.py and next_version_of_f2.py, rename f2.py into f2.py.old and rename next_version_of_f2.py to f2.py
This is certainly possible but probably inadvisable.
Editing code should typically be a separate action from executing it (even when we use a single tool that can do both, like a lot of modern IDEs).
It suggests a poor workflow. If you want f2.py to print("ABCD"), then it should be written that way.
It's confusing. In order to understand what f2.py does, you have to mentally model the entirety of f1.py and f2.py, and there's no indication of this in f2.py.
It invites all kinds of difficult-to-debug situations. What happens if f1.py is run twice at the same time? What if two different versions of f1.py are run at the same time? Or if I happen to be reading f2.py when you run f1.py? Or if I'm editing f2.py and save my changes while you're running f1.py?
It's a security problem. For f1.py to edit f2.py, the user (shell, web-server, or other surface) calling f1.py has to have edit permissions on f2.py. That means that if they can get f1.py to do something besides what you intended (specifically, if they can get their own text in place of "ABCD"), then they can get arbitrary code execution in everyone else's runtime!
Note that it's perfectly fine to have code that generates or edits other code. The problem is when a program (possibly spanning multiple files) edits its own source.
gelonida discusses some options, which are fine and appropriate for certain contexts such as managing user-specific configuration or building documents. That said, if you're familiar with functions, variables, imports, and other basics of computer science, then you may not even need a config.json file or a template engine.
Reconsider the end result you're trying to accomplish. Do some more research/reading, and if you're still stuck start a new question about the bigger-picture task.
complaints about XYZ problems are old hat, so here's how to do what you want, even though it's awful and you really shouldn't.
f1.py
import os
import re
import sys
#O = "Random string"
print("ABCD")
#P = "Random string"
selector = re.compile(
r'#([A-Z]) = \"Random string\"\nprint\(\"([A-Z]{4})\"\)\n#([A-Z]) = \"Random string\"')
template = '''#{} = "Random string"
print("{}")
#{} = "Random string"'''
own_file_name = os.path.abspath(__file__)
own_directory = os.path.dirname(own_file_name)
def read_file(name: str) -> str:
with open(name, 'r') as f:
return f.read()
find_replacement = selector.match(read_file(own_file_name))
replacement = find_replacement.group(1) if find_replacement else False
if not replacement:
sys.exit(-1)
def make_replacement(reg_match) -> str:
return template.format(reg_match.group(1), replacement, reg_match.group(3))
for dir_entry in os.listdir(own_directory):
if dir_entry.is_file():
original = read_file(dir_entry.path)
with open(dir_entry.path, 'w') as out_file:
out_file.write(selector.sub(make_replacement, original))
# this will cause an infinite loop, but you technically asked for it :)
for dir_entry in os.listdir(own_directory):
if dir_entry.is_file():
exec(read_file(dir_entry.path))
I want to be clear that the above is a joke. I haven't tested it, and I desperately hope it won't actually solve any problems for you.
I am trying to write some results I get from a function for a range but I don't understand why the file is empty. The function is working fine because I can see the results in the console when I use print. First, I'm creating the file which is working because it is created; the output file name is taken from a string, and that part is working too. So the following creates the file in the given path:
report_strategy = open(output_path+strategy.partition("strategy(")[2].partition(",")[0]+".txt", "w")
it creates a text file with the name taken from a string named "strategy", for example:
strategy = "strategy(abstraction,Ent_parent)"
a file called "abstraction.txt" is created in the output path folder. So far so good. But I can't get to write anything to this file. I have a range of a few integers
maps = (175,178,185)
This is the function:
def strategy_count(map_path,map_id)
The following loop does the counting for each item in the range "maps" to return an integer:
for i in maps:
report_strategy.write(str(i), ",", str(strategy_count(maps_path,str(i))))
and the file is closed at the end:
report_strategy.close()
Now the following:
for i in maps:
print str(i), "," , strategy_count(maps_path,str(i))
does give me what I want in the console:
175 , 3
178 , 0
185 , 1
What am I missing?! The function works, the file is created. I see the output in the console as I want, but I can't write the same thing in the file. And of course, I close the file.
This is a part of a program that reads text files (actually Prolog files) and runs an Answer Set Programming solver called Clingo. Then the output is read to find instances of occurring strategies (a series of actions with specific rules). The whole code:
import pmaps
import strategies
import generalization
# select the strategy to count:
strategy = strategies.abstraction_strategy
import subprocess
def strategy_count(path,name):
p=subprocess.Popen([pmaps.clingo_path,"0",""],
stdout=subprocess.PIPE,stderr=subprocess.STDOUT,stdin=subprocess.PIPE)
#
## write input facts and rules to clingo
with open(path+name+".txt","r") as source:
for line in source:
p.stdin.write(line)
source.close()
# some generalization rules added
p.stdin.write(generalization.parent_of)
p.stdin.write(generalization.chain_parent_of)
# add the strategy
p.stdin.write(strategy)
p.stdin.write("#hide.")
p.stdin.write("#show strategy(_,_).")
#p.stdin.write("#show parent_of(_,_,_).")
# close the input to clingo
p.stdin.close()
lines = []
for line in p.stdout.readlines():
lines.append(line)
counter=0
for line in lines:
if line.startswith('Answer'):
answer = lines[counter+1]
break
if line.startswith('UNSATISFIABLE'):
answer = ''
break
counter+=1
strategies = answer.count('strategy')
return strategies
# select which data set (from the "pmaps" file) to count strategies for:
report_strategy = open(pmaps.hw3_output_path+strategy.partition("strategy(")[2].partition(",")[0]+".txt", "w")
for i in pmaps.pmaps_hw3_fall14:
report_strategy.write(str(i), ",", str(strategy_count(pmaps.path_hw3_fall14,str(i))))
report_strategy.close()
# the following is for testing the code. It is working and there is the right output in the console
#for i in pmaps.pmaps_hw3_fall14:
# print str(i), "," , strategy_count(pmaps.path_hw3_fall14,str(i))
write takes one argument, which must be a string. It doesn't take multiple arguments like print, and it doesn't add a line terminator.
If you want the behavior of print, there's a "print to file" option:
print >>whateverfile, stuff, to, print
Looks weird, doesn't it? The function version of print, active by default in Python 3 and enabled with from __future__ import print_function in Python 2, has nicer syntax for it:
print(stuff, to, print, out=whateverfile)
The problem was with the write which as #user2357112 mentioned takes only one argument. The solution could also be joining the strings with + or join():
for i in maps:
report.write(str(i)+ ","+str(strategy_count(pmaps.path_hw3_fall14,str(i)))+"\n")
#user2357112 your answer might have the advantage of knowing if your test debug in the console produces the write answer, you just need to write that. Thanks.
Is it possible (not necessarly using python introspection) to print the source code of a script?
I want to execute a short python script that also print its source (so I can see which commands are executed).
The script is something like this:
command1()
#command2()
command3()
print some_variable_that_contain_src
The real application is that I want to run a script from IPython with the run -i magic and have as output the source (i.e. the commands executed). In this way I can check which commands are commented at every execution. Moreover, if executed in a Notebook I leave a trace of which commands have been used.
Solution
Using korylprince solution I end up with this one-liner to be put at the beginning of the script:
with open(__file__) as f: print '\n'.join(f.read().split('\n')[1:])
This will print the script source except the first line (that would be only noise). It's also easy to modify the slicing in order to print a different "slice" of the script.
If you want to print the whole file instead, the one-liner simplifies to:
with open(__file__) as f: print f.read()
As long as you're not doing anything crazy with packages, put this at the top of your script
with open(__file__) as f:
print f.read()
Which will read in the current file and print it out.
For python 3 make sure to use instead
print(f.read())
For the most simple answer:
import my_module
print open(my_module.__file__).read()
I also tried using the inspect package.
import inspect
import my_module
source_list = inspect.getsourcelines(my_module)
Will give you a list of strings with the source code defined in it
for line in source_list[0]:
print line
Will print out the entire source code in a readable manner
I've done a fair amount of lurking on SO and a fair amount of searching and reading, but I must also confess to being a relative noob at programming in general. I am trying to learn as I go, and so I have been playing with Python's NLTK. In the script below, I can get everything to work, except it only writes what would be the first screen of a multi-screen output, at least that's how I am thinking about it.
Here's the script:
#! /usr/bin/env python
import nltk
# First we have to open and read the file:
thefile = open('all_no_id.txt')
raw = thefile.read()
# Second we have to process it with nltk functions to do what we want
tokens = nltk.wordpunct_tokenize(raw)
text = nltk.Text(tokens)
# Now we can actually do stuff with it:
concord = text.concordance("cultural")
# Now to save this to a file
fileconcord = open('ccord-cultural.txt', 'w')
fileconcord.writelines(concord)
fileconcord.close()
And here's the beginning of the output file:
Building index...
Displaying 25 of 530 matches:
y . The Baobab Tree : Stories of Cultural Continuity The continuity evident
regardless of ethnicity , and the cultural legacy of Africa as well . This Af
What am I missing here to get the entire 530 matches written to the file?
text.concordance(self, word, width=79, lines=25) seem to have other parameters as per manual.
I see no way to extract the size of concordance index, however, the concordance printing code seem to have this part: lines = min(lines, len(offsets)), therefore you can simply pass sys.maxint as a last argument:
concord = text.concordance("cultural", 75, sys.maxint)
Added:
Looking at you original code now, I can't see a way it could work before. text.concordance does not return anything, but outputs everything to stdout using print. Therefore, the easy option would be redirection stdout to you file, like this:
import sys
....
# Open the file
fileconcord = open('ccord-cultural.txt', 'w')
# Save old stdout stream
tmpout = sys.stdout
# Redirect all "print" calls to that file
sys.stdout = fileconcord
# Init the method
text.concordance("cultural", 200, sys.maxint)
# Close file
fileconcord.close()
# Reset stdout in case you need something else to print
sys.stdout = tmpout
Another option would be to use the respective classes directly and omit the Text wrapper. Just copy bits from here and combine them with bits from here and you are done.
Update:
I found this write text.concordance output to a file Options
from the ntlk usergroup. It's from 2010, and states:
Documentation for the Text class says: "is intended to support
initial exploration of texts (via the interactive console). ... If you
wish to write a program which makes use of these analyses, then you
should bypass the Text class, and use the appropriate analysis
function or class directly instead."
If nothing has changed in the package since then, this may be the source of your problem.
--- previously ---
I don't see a problem with writing to the file using writelines():
file.writelines(sequence)
Write a sequence of strings to the file. The sequence can be any
iterable object producing strings, typically a list of strings. There
is no return value. (The name is intended to match readlines();
writelines() does not add line separators.)
Note the italicized part, did you examine the output file in different editors? Perhaps the data is there, but not being rendered correctly due to missing end of line seperators?
Are you sure this part is generating the data you want to output?
concord = text.concordance("cultural")
I'm not familiar with nltk, so I'm just asking as part of eliminating possible sources for the problem.
I have written a few lines of code in Python to see if I can make it read a text file, make a list out of it where the lines are lists themselves, and then turn everything back into a string and write it as output on a different file. This may sound silly, but the idea is to shuffle the items once they are listed, and I need to make sure I can do the reading and writing correctly first. This is the code:
import csv,StringIO
datalist = open('tmp/lista.txt', 'r')
leyendo = datalist.read()
separando = csv.reader(StringIO.StringIO(leyendo), delimiter = '\t')
macrolist = list(separando)
almosthere = ('\t'.join(i) for i in macrolist)
justonemore = list(almosthere)
arewedoneyet = '\n'.join(justonemore)
with open('tmp/randolista.txt', 'w') as newdoc:
newdoc.write(arewedoneyet)
newdoc.close()
datalist.close()
This seems to work just fine when I run it line by line on the interpreter, but when I save it as a separate Python script and run it (myscript.py) nothing happens. The output file is not even created. After having a look at similar issues raised here, I have introduced the 'with' parameter (before I opened the output file through output = open()), I have tried flushing as well as closing the file... Nothing seems to work. The standalone script does not seem to do much, but the code can't be too wrong if it works on the interpreter, right?
Thanks in advance!
P.S.: I'm new to Python and fairly new to programming, so I apologise if this is due to a shallow understanding of a basic issue.
Where are the input file and where do you want to save the output file. For this kind of scripts i think that it's better use absolute paths
Use:
open('/tmp/lista.txt', 'r')
instead of:
open('tmp/lista.txt', 'r')
I think that the error can be related to this
It may have something to do with where you start your interpreter.
Try use a absolute path /tmp/randolista.txt instead of relative path tmp/randolista.txt to isolate the problem.