I am writing an AI that runs commands off of text file modules. In the folder in which my python program is located are a group of text files. They each have sets of keyword-command sets formatted like this:
keyword 1,function 1|keyword 2,function 2
My program loops through all these files and creates a list of keyword-command sets. For example, from 2 text files,
keyword 1,function 1|keyword 2,function 2 and keyword 3,function 3,
the list generated is
[['keyword 1', 'function 1'], ['keyword 2', 'function 2'], ['keyword 3', 'function 3']].
Now the function portions are commands run via the exec command, but I would like to have the ability to execute multiple lines of code for each function. I am thinking I will accomplish this by adding a special symbol to symbolize a new line and add the commands to a list, then iterate through them. My question is are there any symbol I could safely use that won't mess up any other commands that may use those symbols? For example, if I use %, it would mess up the modulo command.
Here is my code as of now in case you need it, although I don't really think you would.
# Setup
import os
import locale
# Load modules
functions = []
print(str(os.getcwd()))
print(str(os.getcwd().replace('ZAAI.py', '')))
for file in os.listdir(os.getcwd().replace('ZAAI.py', '')):
if file.endswith('.txt'):
openFile = open(os.getcwd().replace('ZAAI.py', '') + file, encoding=locale.getpreferredencoding())
openFileText = openFile.read()
print(openFileText)
for item in openFileText.split('|'):
functions.append(item.split(','))
print(functions)
Well, python supports multiple expressions/statements on a single line using a semi-colon ;
a = 1; b = 2; c = a + b; print c
So, you don't need to create your own newline symbol to handle multiline python scripts. That being said, you should probably not do this.
You're essentially creating a somewhat limited plugin architecture. People have done this before. There are lots of options for doing this in python. I can just imagine the amount of frustration someone could have looking at one of your "plugin" files with dozens of commands, each with a 30 line python script on a single line.
According to the documentation on literals, The $ and ? characters are not used in Python for any purpose other than string literals and comments.
Related
I would love to create a python script that loops through a bunch of files in a folder and replaces a bit of formatted text with a variable from the script.
for a (very simplified) example, a folder contains two files:
greeting.sql with the text
select greeting
from greeting_list
where name = {name}
and
credentials.sql with the text
select credentials
from credentials_list
where name = {name}
The python script could then pull in these text files, and interpolate the script, replacing the {name} variable in the text with a pre-defined variable, as name = 'Bob'.
Something like
fd = open('greeting.sql', 'r')
greeting = fd.read()
fd.close()
names = ['Jeff','Melanie','Nitin','Satya','Serena']
[run_sql(greeting.format(n)) for n in names]
I usually use interpolated strings, as f'Greeting for {name} is {greeting}' so I'm highly confident my usage of .format() here doesn't make sense.
The point of this operation would be to allow analysts to update and maintain SQL code, while having a python script pull in the correct version of the script and run it with a variety of variables.
Well, the problem was very definitely that I was lazy with my .format() knowledge because f-strings seemed so much easier. The solution is quite simple:
With the same greeting.sql file above,
py_name = 'Bob'
print(greeting.format(name = py_name))
Works beautifully
I have a little homework webpage. I'm trying to automate the production of the webpage each week.
I know very little about this. Wednesday I asked about importing Python functions that I made. I think I have grasped the concept. I've tested everything in Idle. All the modules work.
In my Python shell, (I use Idle in Ubuntu), everything does what I want.
I first append the paths:
for i in range(0, len(pyPaths)):
sys.path.append(pyPaths[i])
then import the modules:
from makeRBsInlineV1 import makeHTMLrbsNums
from makeCheckboxesInlineV1 import makeHTMLCBs
from makeDropdownboxesInlineV1 import makeDropdownboxes
from createhtmlTableInlineV1 import makeHTMLtable
from makeRBsInlineV2 import makeHTML_RBs
from readLinesTextboxesInlineV1 import readLinesmakeTBs
from makeThankyouPHPInlineV1 import makeThankyouPHP
All these modules return a text string which is a mixture of html tags + my text. It displays nicely in Firefox.
In Idle I just write, for example:
myString = readLinesTextboxesInlineV3()
it asks me a few questions and off it goes. Afterwards, myString is ready to be put in the webpage text string.
However, in bash, I cannot write
myString = readLinesTextboxesInlineV3()
in bash, I cannot write
myString = input('Enter the name of the module you want. ')
and then enter,
readLinesTextboxesInlineV3()
because then myString is just the input text, not the function.
Each week things are different, I may need to run 2 or more of the modules, add the result strings, then write them in the webpage text string at the correct place.
I run this in a
while True:
loop, so I can add strings from various modules.
For any given week, I don't know which module I want to use, so I make them all available.
I have a list of all the modules:
pyFiles = ['makeCheckboxesInlineV1()', 'dropdownboxesInlineV1()',
'createhtmlTableInlineV1()', 'makeRBsInlineV2()',
'readLinesTextboxesInlineV3()', 'makeThankyouPHPInlineV1()']
How do I assign myString to any 1 of the modules above when I run makeWebpage.py in bash?
I tried this:
pyFiles = [makeHTMLCBs(), makeDropdownboxes(), makeHTMLtable(), makeHTML_RBs(), readLinesmakeTBs(), makeThankyouPHP()]
Declare a list of the functions. Trouble is, as soon as I do that, Python wants to run the first function immediately. I was hoping I could enter a number and run say:
myString = pyFiles[3]
The best solution for you is to add arguments to your python script that can be added as flag to your command line.
This way you could call your script like the following from the shell (example):
python makeWebpage.py --module "makeCheckboxesInlineV1"
Python standard library provide argparse module for that specific need. It's really easy to handle and transform shell argument into python variable.
I recommend that you read that blog post which I found myself really useful. The part you need the most is What about argparse?
Add something like this in your main:
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group()
group.add_argument('-m', '--module', action='store_true')
args = parser.parse_args()
module = args.module # The variable module now is a String containing "makeCheckboxesInlineV1"
I have a problem with Python and need your help.
Take a look at this code:
import os
os.chdir('''C:\\Users\\Admin\\Desktop\\Automate_the_Boring_Stuff_onlimematerials_v.2\\automate_online-materials\\example.xlsx''')
The os.chdir() did not work because the directory I put in between the ''' and ''' is considered as raw string. Note that a line is no more than 125 characters, so I have to make a new line.
So how can I fix this?
You can split your statement into multiple lines by using the backslash \ to indicate that a statement is continued on the new line.
message = 'This message will not generate an error because \
it was split by using the backslash on your \
keyboard'
print(message)
Output
This message will not generate an error because it was split by using the backslash on your keyboard
Lines can be longer than 125 characters, but you should probably avoid that. You have a few solutions:
x = ('hi'
'there')
# x is now the string "hithere"
os.chdir('hi'
'there') # does a similar thing (os.chdir('hithere'))
You could also set up a variable:
root_path = "C:\\Users\\Admin\\Desktop"
filepath = "other\\directories" # why not just rename it though
os.chdir(os.path.join(root_path, filepath))
Do these work for you?
I'm also curious why you have to chdir there; if it's possible, you should just run the python script from that directory.
I'm kind of new to python, but something I find myself doing in bash a lot is prepending and appending strings to filenames with parameter expansion.
e.g.
for file in *.txt ; do mkdir ${file%.*} ; mv $file ${file%.*}/ ; done
Would be an example for stripping off the extension of a load of files, making directories based on those names, and then moving the files inside their namesake folders now.
If I want to achieve a similar thing, such as rename the output of a function based on the input file name (below is an example of a Biopython function), I've seen a few ways to do it with string concatenation etc, but without bracketing and so on, it looks confusing and like it might create parsing errors with spaces, quotes and so on being all over the place potentially.
SeqIO.convert(genbank, 'genbank', genbank[:-3]+'tmp', 'fasta')
There are other threads on here about using rsplit, string concatenation and so on, but is one of these more 'correct' than another?
String concatenation is really nice and works great in simple commands like print(), but when adding to commands that are expecting separated values, it strikes me as a little messy?
You can use os.path.splitext which is build especially for file names:
>>> import os
>>>
>>> fname = '/tmp/foo/bar.baz'
>>> sp = os.path.splitext(fname)
>>> sp
('/tmp/foo/bar', '.baz')
Extracting the name of the file without extension:
>>> os.path.basename(sp[0])
'bar'
And formatting a new file name:
>>> "{}.txt".format(os.path.basename(sp[0]))
'bar.txt'
In general, when manipulating file names and paths I try to just use os.path, since it already handles edge cases, can normalize paths from things like /..//./././, etc.
I want to keep a piece of code in a txt file and execute it from my Python script.
For example in the txt file there is
print("ok")
I want my programme to print ok and not print print("ok "). How can I do this?
Doing what you want usually is a security risk, but not necessarily so.
You'll definitely have to notify the user about potential risk.
There is more than one program using execfile() or compile() and exec statement to provide plug-ins system.
There is nothing so ugly about it, you just have to know what are you doing, when and where.
Both execfile(), eval() and exec statement allow you to specify a scope in which will your code be executed/evaluated.
myscope = {}
execfile("myfile.txt", myscope)
This will prevent the new code being mixed with the old one. All variables classes, functions and modules from myfile.txt will be contained in myscope dictionary.
This, however, does not prevent malicious code to delete all files it can from your disk or something similar.
Python 2 has a nice module called rexec, but from Python 2.2 it doesn't work any more.
It implements execfile() exec statement and eval() in restricted environment.
Though it doesn't work it is there, and you can dig through the code to see how it is done.
So, you see, it is possible to allow only secure code to execute (well, as secure as it can be) from external source.
There is another way too.
You can load the file, compile the code from it and then inspect it to see what it does. Then say, yes I'll execute it or no, I won't. This is, however, a little bit more work and a lot of complications.
But, I don't think that it'll be necessary to go through all that stuff. Please elaborate some more on your problem. What exactly do you mean by level editor?
I don't think external code is a solution for that.
You are looking for eval function
eval is a function that takes arbitrary python code represented as a string and execute it at run time.
Example
>>> x = 1
>>> y = eval('x+1')
>>> print(y)
2
It works in both Python 2.x and 3.x
Check the documentation : https://docs.python.org/2.7/library/functions.html#eval
So I wanted to do the same thing. I came across this: GeeksforGeeks.
With this we could make a text file. Let's say it's called myfile.txt and in the first line we will add print("ok") in the second add A += 1.
Now let's move over to the script editor.
# open txt file
f = open("myfile.txt", "r")
# add the data in the file the a var
data = f.readlines()
# remove unwanted \n for new lines and '' left over by the code above
# readlines() returns a list so we need to convert data to str
# in data[] add the line you wish to read from
info = str(data[0]).strip("\n").strip("'")
# run the code
exec(info)
# running the A += 1 in the txt file
# run this
A = 0
info = str(data[1]).strip("\n").strip("'")
while A == 0:
print(A)
exec(info)
print(A)
# if you wanted too you can even define a variable with this
alist = ["B = 0", "B += 1", "print(B)"]
runner = [0, 1, 2, 1, 2, 0, 2]
for i in range(len(runner)):
exec(alist[int(runner[i])])