I wrote a python script that works. The first line of my script is reading an hdf5 file
readFile = h5py.File('FileName_00','r')
After reading the file, my script does several mathematical operations, successfully working. In the output I got function F.
Now, I want to repeat the same script for different files. Basically, I only need to modify FileName_00 by FimeName_01 or ....FileName_10. I was thinking to create a script that call this script!
I never wrote a script that call another script, so any advice would be appreciable.
One option: turn your existing code into a function which takes a filename as an argument:
def myfunc(filename):
h5py.file(filename, 'r')
...
Now, after your existing code, call your function with the filenames you want to input:
myfunc('Filename_00')
myfunc('Filename_01')
myfunc('Filename_02')
...
Even more usefully, I definitely recommend looking into
if(__name__ == '__main__')
and argparse (https://docs.python.org/3/library/argparse.html) as jkr noted.
Also, if you put your algorithm in a function like this, you can import it and use it in another Python script. Very useful!
Although there are certainly many ways to achieve what you want without multiple python scripts, as other answerers have shown, here's how you could do it.
In python we have this function os.system (learn more about it here: https://docs.python.org/3/library/os.html#os.system). Simply put, you can use it like this:
os.system("INSERT COMMAND HERE")
Replacing INSERT COMMAND HERE with the command you use to run your python script. For example, with a script named script.py you could conceivably (depending on your environment) include the following line of code in a secondary python script:
os.system("python script.py")
Running the secondary python script would run script.py as well. FWIW, I don't necessarily think this is the best way to accomplish your goal -- I tend to agree with DraftyHat's solution in most circumstances. But in case you were curious, this is certainly an option in python. I've used this functionality in the past, albeit not to run other python scripts, but to execute commands in the shell. Hope this helps!
Related
Original question: I am running a python script
python script.py #runs/run_1/parameters.txt
Is there some way to access the string runs/run_1 in my script?
Actual question: I noticed that what I really need is a little different from the above. Independently from the directory from which I run the script, I need to get the location of parameters.txt.
For example, when I'm running the script from the directory runs/run_1 itself, I still need to get the path. I could do this with os.getcwd().
But when I'm running the code from PyCharm, I pass #/runs/run_1/parameters.txt as a parameter, while the script itself lives in some other directory. Here, I would need to read from sys.argv as suggested in the comments below.
For now I will have to do differentiate these cases with an if-statement checking whether the string before parameters.txt in python script.py #.../parameters.txt is empty. Is there a better way?
A little bit of an ugly question, but I didn't find existing SO posts which cover it.
Right now I need to use an existing python tool available on this github
This is a rather big piece of code with a lot of dependencies which I don't want to mess with. In a nutshell one can run its module by passing the command line arguments, for example:
timesearch.py timesearch -r "subreddit1" -l "1466812800" -up "1498348800"
Now, I need to run this tool a bunch of times using a for loop, passing over different argument values each time. The tool also prints out some output into command line when you run it - and I would like to intercept and print it out from my python script as well. Finally, I need to ensure that before I move on in my loop and run the tool another time that current execution of the timesearch tool is completed.
One side note here - I do need to ensure that the timesearch is executed using same environment which I use to run my main script with for loop.
I am trying to understand what is the best way to do it.
If I just go for this it doesn't work:
import os
#for loop will go here
os.system('python timesearch.py timesearch -r "ethereum" -l "1466812800" -up "1498348800"')
It fails due to several reasons - it doesn't use the environment in which I am writing my script with a loop, it also doesn't capture the print output of timesearch.
Any advice on how to achieve it?
Just to highlight - I can't just go and pull function I need in timesearch, since it calls the __init__ to set up some things based on the arguments you pass.
I wouldn't call python script with os.system. There is basically one function which you need to use: main(sys.argv[1:])
https://github.com/voussoir/timesearch/blob/master/timesearch/__init__.py#L435.
I am aware that there are multiple libraries for both languages (R/Python) to call modules from the other one. I am looking for a way to have the backend of my code running in python mainly because of .pyc and speed, and also the front end running in R so I can have a Shiny app. I couldn't find a way to make python machine for the backend. If anyone knows how to do it in R/Rstudio please respond.
I don't have any good benchmarks on it's speed, but the reticulate package is the best way I know of to pass data to and from a python script without using a webserver. It lets you import python objects into R where they will act like R objects, accepting arguments and returning values.
There were a few wonky problems that I had when I just wanted to run functions from a single file. It ran into problems with import statements and multiple functions that called on each other. What worked well was to run the import statements separately (see the sapply() statement below) and to merge all the code in my python script into a single object. This worked nicely and seemed about as fast as running it in python normally (though I haven't done any real benchmarking)
library(reticulate)
use_python(python = '/usr/bin/python') # optionally specify python location
# Import statements are here, not in the file
sapply(c("import mysql.connector", "import re"), py_run_string)
# File contains only the definition of class MismatchFinder
source_python("python_script.py")
# Now we can call on that python object from R
result <- MismatchFinder()$find_mismatch(arg1, arg2)
My impression is that it might be simpler if you make your python code into a module and load it with: py_module <- import_from_path('my_python_module', path = 'PATH') but I didn't try that.
Hope this helps!
I believe what you are looking for is the code below. It will run a python script in R.
system('python3 file_name.py')
I'm profiling a Python 3.4 script within an interactive shell environment (IDE, if it matters). Normally I use cProfile to profile functions. However, this time I have some top-level code in the script. By "top-level" I mean that the code is not inside a function definition. cProfile.run won't accept the filename - normally I would pass it a function.
To get around this, I wrap the top-level code in a main() function, execute it to create main in the shell namespace, then run cProfiler.run('main()'). This is pretty annoying - I would like to fool around with several variables generated in the top-level code, and I'd rather not try to return them all from main().
I have carefully read the similar questions How can you profile a python script? and How to profile my code?.
They give great solutions for profiling top-level code from the command line and for profiling functions from a shell, but I don't think they address this specific question.
I have a kluge for getting this done, but I think there's probably a better way out there.
cProfile.run(compile(open(filename, "rb").read(), filename, 'exec'))
where filename is the filename of the script.
is this possible? I want to execute a different scriptfile from a iron python script. My main file is getting way to big and I want to make it more readable.
Why don't you reorganize your script into, say a Python module? Have a look at the Python Docs about Modules
#OP's comment: If you truely have a ~5000 lines of code in a single function you should definitely think about reorganizing your code. If you are sure about that "execute a different script" thing, take a look at subprocess.Popen, although I wouldn't advise it.