Sorry for the beginner question, but I can't figure out cProfile (I'm really new to Python)
I can run it via my terminal with:
python -m cProfile myscript.py
But I need to run it on a webserver, so I'd like to put the command within the script it will look at. How would I do this? I've seen stuff using terms like __init__ and __main__ but I dont really understand what those are.
I know this is simple, I'm just still trying to learn everything and I know there's someone who will know this.
Thanks in advance! I appreciate it.
I think you've been seeing ideas like this:
if __name__ == "__main__":
# do something if this script is invoked
# as python scriptname. Otherwise, gets ignored.
What happens is when you call python on a script, that file has an attribute __name__ set to "__main__" if it is the file being directly called by the python executable. Otherwise, (if it is not directly called) it is imported.
Now, you can use this trick on your scripts if you need to, for example, assuming you have:
def somescriptfunc():
# does something
pass
if __name__ == "__main__":
# do something if this script is invoked
# as python scriptname. Otherwise, gets ignored.
import cProfile
cProfile.run('somescriptfunc()')
This changes your script. When imported, its member functions, classes etc can be used as normal. When run from the command-line, it profiles itself.
Is this what you're looking for?
From the comments I've gathered more is perhaps needed, so here goes:
If you're running a script from CGI changes are it is of the form:
# do some stuff to extract the parameters
# do something with the parameters
# return the response.
When I say abstract out, you can do this:
def do_something_with_parameters(param1, param2):
pass
if __name__ = "__main__":
import cProfile
cProfile.run('do_something_with_parameters(param1=\'sometestvalue\')')
Put that file on your python path. When run itself, it will profile the function you want profiling.
Now, for your CGI script, create a script that does:
import {insert name of script from above here}
# do something to determine parameter values
# do something with them *via the function*:
do_something_with_parameters(param1=..., param2=...)
# return something
So your cgi script just becomes a little wrapper for your function (which it is anyway) and your function is now self-testing.
You can then profile the function using made up values on your desktop, away from the production server.
There are probably neater ways to achieve this, but it would work.
Related
I have two main python scripts in my project:
sftpConnector.py and fileIterator.py
sftpConnector.py establishes connection to an SFTP server and downloads some file, it however does not call any other script. It has only function: establisher()
While, fileiterator.py also has one function iterator() it calls two other scripts: folderGenerator.py and dataProcessor.py()
When independently called, both the scripts, sftpConnector.py and fileIterator.py execute properly without any issues, however, when I try to create a main.py file to call these two scripts one after the other, it does not work.
The main.py looks like this:
import sftpConnector
import fileIterator
if __name__ == 'main':
folder_name = input('File name:\n')
print('folder_name')
sftpConnector.establisher(folder_name)
fileIterator.iterator(folder_name)
This main.py file runs the first script and function, sftpConnector.establisher(folder_name) however it does not even go inside the second script. I am not sure what's going on here.
Also, not sure if this will help, but neither of the script are returning anything.
Your run guard should be
if __name__ == "__main__":
That’s how to ensure that your code only runs if run as a module (versus being imported). You accidentally checked for the wrong name (“main”) and I’m guessing that’s why the code in your if block isn’t running. It’s an easy mistake to make as it’s easy to assume that __name__ refers to the name of your file when you run your script directly, but it doesn’t; it refers to the environment where the toplevel code is run. The line if __name__ == '__main__': is asking the question: “Is this script the toplevel one?”
There is a python script start_test.py.
There is a second python script siple_test.py.
# pseudo code:
start_test.py --calls--> subprocess(python.exe simple_test.py, args_simple_test[])
The python interpreter for both scripts is the same. So instead of opening a new instance, I want to run simple_test.py directly from start_test.py. I need to preserve the sys.args environment. A nice to have would be to actually enter following code section in simple_test.py:
# file: simple_test.py
if __name__ == '__main__':
some_test_function()
Most important is, that the way should be a universal one, not depending on the content of the simple_test.py.
This setup would provide two benefits:
The call is much less resource intensive
The whole stack of simple_test.py can be debugged with pycharm
So, how do I execute the call of a python script, from a python script, without starting a new subprocess?
"Executing a script" is a somewhat blurry term.
Typically the if __name__== "__main__": part does the argument (sys.argv) decoding and then calls a worker function with explicit parameters. For clarity: It should not do anything else, since this additional work can't be called without creating a new process causing all the overhead you are trying to avoid.
You simply bypass that and call this implementing routine directly.
So you end up with start_test.py containing something like:
from simple_test import worker
# ...
worker(typed_arg1, typed_arg2)
I have a python script (we'll say "script.py") that I want to grab values from using a separate GUI python script ("GUI.py"). When I run my GUI.py script, I want to have these text fields in the GUI sent over to script.py after clicking a button in the GUI. I am thinking that my best solution might be to have the GUI.py create another script ("lib.py") that holds all these values and it just writes to it. It then runs the "script.py" script. So all in all the GUI.py will have a function that when called will look something like this:
def on_button(self):
username = self.usernameInput.get()
file = open(“lib.py”,”w”)
file.write(“username = ” + username)
os.system("script.py")
I think this will work, but I am just wondering, does this seem like a practical solution? Let me know what you guys think.
No, I don't think this is the practical solution.
Do you consider instead making the python script you want to run into a module or package that you can call directly inside your GUI? I think that is the cleanest approach. For using your scripts as modules, see the docs or for 2.7.
Basically a module is a python file, script.py, and as long as it is in the python path (say, your current directory), you can import it:
from script import action
So you could try:
def on_button(self):
username = self.usernameInput.get()
result = action(username) # and any other args you want to pass
print(result)
That is, if the script in question uses a if __name__ == "__main__": statement (or can otherwise be run from the command line), try putting the operations in some def action(args): function and importing it into your GUI.
Please excuse what I know is an incredibly basic question that I have nevertheless been unable to resolve on my own.
I'm trying to switch over my data analysis from Matlab to Python, and I'm struggling with something very basic: in Matlab, I write a function in the editor, and to use that function I simply call it from the command line, or within other functions. The function that I compose in the matlab editor is given a name at the function definition line, and it's generally best for the function name to match the .m file name to avoid confusion.
I don't understand how functions differ in Python, because I have not been successful translating the same approach there.
For instance, if I write a function in the Python editor (I'm using Python 2.7 and Spyder), simply saving the .py file and calling it by its name from the Python terminal does not work. I get a "function not defined" error. However, if I execute the function within Spyder's editor (using the "run file" button), not only does the code execute properly, from that point on the function is also call-able directly from the terminal.
So...what am I doing wrong? I fully appreciate that using Python isn't going to be identical to Matlab in every way, but it seems that what I'm trying to do isn't unreasonable. I simply want to be able to write functions and call them from the python command line, without having to run each and every one through the editor first. I'm sure my mistake here must be very simple, yet doing quite a lot of reading online hasn't led me to an answer.
Thanks for any information!
If you want to use functions defined in a particular file in Python you need to "import" that file first. This is similar to running the code in that file. Matlab doesn't require you to do this because it searches for files with a matching name and automagically reads in the code for you.
For example,
myFunction.py is a file containing
def myAdd(a, b):
return a + b
In order to access this function from the Python command line or another file I would type
from myFunction import myAdd
And then during this session I can type
myAdd(1, 2)
There are a couple of ways of using import, see here.
You need to a check for __main__ to your python script
def myFunction():
pass
if __name__ == "__main__":
myFunction()
then you can run your script from terminal like this
python myscript.py
Also if your function is in another file you need to import it
from myFunctions import myFunction
myFunction()
Python doesn't have MATLAB's "one function per file" limitation. You can have as many functions as you want in a given file, and all of them can be accessed from the command line or from other functions.
Python also doesn't follow MATLAB's practice of always automatically making every function it can find usable all the time, which tends to lead to function name collisions (two functions with the same name).
Instead, Python uses the concept of a "module". A module is just a file (your .py file). That file can have zero or more functions, zero or more variables, and zero or more classes. When you want to use something from that file, you just import it.
So say you have a file 'mystuff.py':
X = 1
Y = 2
def myfunc1(a, b):
do_something
def myfunc2(c, d):
do_something
And you want to use it, you can just type import mystuff. You can then access any of the variables or functions in mystuff. To call myfunc2, you can just do mystuff.myfunc2(z, w).
What basically happens is that when you type import mystuff, it just executes the code in the file, and makes all the variables that result available from mystuff.<varname>, where <varname> is the name of the variable. Unlike in MATLAB, Python functions are treated like any other variable, so they can be accessed just like any other variable. The same is true with classes.
There are other ways to import, too, such as from mystuff import myfunc.
You run python programs by running them with
python program.py
This question already has answers here:
Why doesn't the main() function run when I start a Python script? Where does the script start running (what is its entry point)?
(5 answers)
Closed 5 months ago.
Consider:
#! /usr/bin/python
def main():
print("boo")
This code does nothing when I try to run it in Python 3.3. No error or anything.
What’s wrong?
gvim script
chmod 775 script
./script
You still have to call the function.
def main(): # declaring a function just declares it - the code doesn't run
print("boo")
main() # here we call the function
I assumed you wanted to call the print function when the script was executed from the command line.
In Python you can figure out if the script containing a piece of code is the same as the script which was launched initially by checking the __name__ variable against __main__.
#! /usr/bin/python
if __name__ == '__main__':
print("boo")
With just these lines of code:
def main():
print("boo")
you're defining a function and not actually invoking it. To invoke the function main(), you need to call it like this:
main()
You need to call that function. Update the script to:
#! /usr/bin/python
def main():
print("boo")
# Call it
main()
In Python, if you want to write a script to perform a series of small tasks sequentially, then there is absolutely no need to write a function to contain them.
Just put each on a line on its own; or use an expression delimiter like ; (not really recommended, but you can do is you so desire), likewise:
task1
task2
task3
task4
or
task1; task2; task3; (again **not** really recommended, and certainly not pythonic)
In your case your code could be turned to something like:
print('boo')
print('boo2')
print('boo3')
and it would still act as you expect it to, without the main() method, as they get evaluated sequentially.
Please note that the reason you might want to create a function for these series of tasks is:
to present a nice interface (to clients of the code),
or to encapsulate repeated logic
There might be more uses, but that's the first I can come up with, and serve to prove my point.
Now, if you feel compelled to write code that resembles the main() method in other programming languages, then please use the following Python idiom (as stated by other users so far):
if __name__ == '__main__':
doSomething()
The above is working as follows:
When you import a Python module, it gets a string (usually, the name under which it was imported) assigned as its __name__ attribute.
When you execute a script directly (by invoking the Python vm and passing it the script's name as an argument), the __name__ attribute is set to __main__
So when you use the above idiom, you can both use the script as a pluggable module by importing it at will, or just execute it directly to have the series of expressions under the if __name__ == '__main__': be evaluated directly.
Should you feel the need to dig through more information, my sources were the following:
Python documentation: Modules
Python documentation: Executing modules as scripts
Python documentation: The data model (search for __name__)
If you find the other answers confusing or intimidating, here's a parable which should hopefully help. Look at the following Python program:
a = 34
When it runs, it does something: before exiting the script, Python learns that there is a variable a and that its value is the integer 34. It doesn't do anything with this information, but it's a complete program which does something. In order for it to produce some actual value, it needs to interact with its environment, though. What does it do with this value? It could create 34 directories, or ping the 34th server in your data center, or check the strength of the passwords of the newest 34 users in your database, or whatever; or just print something.
a = 34
print(a)
The following program is in some sense very similar to the first one.
def b():
a = 34
print(a)
When you run this program, it does something: Python now knows that there is a function named b, and that it doesn't take any arguments, and that it contains some syntactically valid Python code which will be run when some other code calls it, but it doesn't actually do anything with this code yet. In order to observe any value being produced by the code in the function, you have to actually call it:
b()
(As an aside, maybe also note that the local variable a inside the function declaration b is distinct from the global variable with the same name.)