I'm new to python and I'd like to use pprofile, but I don't get it running. For example
#pprofile --threads 0 test.py
gives me the error
bash: pprofile: Command not found.
I've tried to run pprofiler as a module, like described here: https://github.com/vpelletier/pprofile , using the following script:
#!/usr/bin/env sc_python3
# coding=utf-8
import time
import pprofile
def someHotSpotCallable():
profiler = pprofile.Profile()
with profiler:
time.sleep(2)
time.sleep(1)
profiler.print_stats()
Running this script gives no output. Changing the script in the following way
#!/usr/bin/env sc_python3
# coding=utf-8
import time
import pprofile
def someHotSpotCallable():
profiler = pprofile.Profile()
with profiler:
time.sleep(2)
time.sleep(1)
profiler.print_stats()
print(someHotSpotCallable())
gives the Output
Total duration: 3.00326s
None
How du I get the line-by-line table-output, shown on https://github.com/vpelletier/pprofile?
I'm using Python 3.4.3, Version 2.7.3 is giving the same output (only Total duration) on my System.
Do I have to install anything?
Thanks a lot!
pprofile author here.
To use pprofile as a command, you would have to install it. The only packaging I have worked on so far is via pypi. Unless you are using a dependency-gathering tool (like buildout), the easiest is probably to setup a virtualenv and install pprofile inside it:
$path_to_your_virtualenv/bin/pip install pprofile
Besides this, there is nothing else to install: pprofile only depends on python interpreter features (more on this just below).
Then you can run it like:
$path_to_your_virtualenv/bin/pprofile <args>
Another way to run pprofile would be to fetch the source and run it as a python script rather than as a standa alone command:
$your_python_interpreter $path_to_pprofile/pprofile.py <args>
Then, about the surprising output: I notice your shebang mentions "sc_python3" as interpreter. What implementation of python interpreter is this ? Would you have some non-standard modules loaded on interpreter start ?
pprofile, in deterministic mode, depends on the interpreter triggering special events each time a line changes, each time a function is called or each time it returns, and, just for completeness, it also monitors when threads are created as the tracing function is a thread local. It looks like that interpreter does not trigger these events. A possible explanation would be that something else is competing with pprofile for these events: only one function can be registered at a time. For example code coverage tools and debuggers may use this function (or another closely related one in standard sys module, setprofile). Just for completeness, setprofile was insufficient for pprofile as it only triggers events on function call/return.
You may want to try the statistic profiling mode of pprofile at the expense of accuracy (but for an extreme reduction in profiler overhead), although there pprofile has to rely on another interpreter feature: the ability to list the call stack of all running threads, sadly expected to be less portable than other features of the standard the sys module.
All these work fine in CPython 2.x, CPython 3.x, pypy and (it has been contributed but I haven't tested it myself) IronPython.
Related
I am trying to run a GUI test using pytest and pywinauto. When I run the code normally, it does not complain.
However, when I am doing it via pytest, it throws a bunch of errors:
Windows fatal exception: code 0x8001010d
Note that the code still executes without problems and the cases are marked as passed. It is just that the output is polluted with these weird Windows exceptions.
What is the reason for this. Should I be concerned?
def test_01():
app = Application(backend='uia')
app.start(PATH_TO_MY_APP)
main = app.window(title_re="MY_APP")
main.wait('visible', timeout=8) # error occurs here
time.sleep(0.5)
win_title = f"MY_APP - New Project"
assert win_title.upper() == main.texts()[0].upper() # error occurs here
This is an effect of a change introduced with pytest 5.0.0. From the release notes:
#5440: The faulthandler standard library module is now enabled by default to help users diagnose crashes in C modules.
This functionality was provided by integrating the external pytest-faulthandler plugin into the core, so users should remove that plugin from their requirements if used.
For more information see the docs: https://docs.pytest.org/en/stable/usage.html#fault-handler
You can mute these errors as follows:
pytest -p no:faulthandler
I had the same problem with Python 3.7.7 32-bit and pytest 5.x.x. It was solved by downgrading pytest to v.4.0.0:
python -m pip install pytest==4.0
Perhaps all Python versions are not compatible with the newest pytest version(s).
My workaround for now is to install pytest==4.6.11
With 5.0.0 the problem occurs the first time.
W10 box.
Have this problem using pytest 6.2.5 and pytest-qt 4.0.2.
I tried np8's idea: still got a horrible crash (without message).
I tried Felix Zumstein's idea: still got a horrible crash (without message).
Per this thread it appears the issue (in 'Doze) is a crap DLL.
What's strange is that pytest-qt and the qtbot fixture seem to work very well... until I get to this one test. So I have concluded that I have done something too complicated in terms of mocking and patching for this crap 'Doze DLL to cope with.
For example, I mocked out two methods on a QMainWindow subclass which is created at the start of the test. But removing these mocks did not solve the problem.
I have so far spent about 2 hours trying to understand what specific feature of this test is so problematic. I am in fact trying to verify the functioning of a method on my main window class which "manufactures" menu items (QWidgets.QAction) based on about 4 parameters.
At this stage I basically have no idea what this "problem feature" is, but it might be the business of inspecting and examining the returned QAction object.
I have a script file which uses the function parallel_map.
( source code of qutip.parallel.parallel_map) from a package QuTiP . As one would see on clicking the source code for the function, it uses the multiprocess module of python.
I looked at answers of serial version of this question. I decided upon Snakeviz on reading zaxiliu's solution.
But naively trying it on my code fails. So what must I do to profile my code? My heart is not set on Snakeviz. I don't mind using any other graphical tool.
Doesn't satisfy the question requirements fully, but will work if nothing else is available
Try using serial_map instead of parallel_map from the same module.
Replace (or better yet comment out) the line
from qutip.parallel import parallel_map
with
from qutip.parallel import serial_map
Now you have a serial implementation of code. This can be profiled using those described in serial version of your question.
After this (assuming you go ahead with Snakeviz)
make the profile file
python -m cProfile -o program.prof my_program.py
Run Snakeviz on the profile file generated in the previous step
snakeviz program.prof
I want to use JMETER to run python scripts, compile my python scripts with Jython, but show importError requsets, I don't know how to install this modules (package) in Jmeter
enter image description here
JSR223 description
Install Jython
Install requests module like:
\\location\\of\\jython\\jython -m pip install requests
Add the following line to the beginning of your script in the JSR223 Sampler:
import sys
sys.path.append('\\location\\of\\jython\\site-packages')
However the above approach is more like a "workaround":
if you plan to run the Python code for higher loads - it makes sense to re-write it in Groovy as JMeter will be loading a 35MB beast into memory in order to launch Python interpreter each time JSR223 Sampler will be called
if you don't plan to run the Python code for high loads - it will be much easier to use underlying Python installation in the operating system and call the Python interpreter using OS Process Sampler
As a first step, you need to add jython standalone jar in lib folder of jmeter:
http://www.jython.org/downloads.html
If issue still persists, show your JSR223 Sampler.
I am curious about running pip.
Everytime I ran pip in command shell in windows like that
c:\python27\script>pip install numpy
But, I wondered if I can run it in python idle.
import pip
pip.install("numpy")
Unfortunately, it is not working.
Still cannot comment so I added another answer. Pip has had several entrypoints in the past. And it's not recommended to call pip directly or in-process (if you still want to do it, "runpy" is kind of recommended):
import sys
import runpy
sys.argv=["pip", "install", "packagename"]
runpy.run_module("pip", run_name="__main__")
But this should also work:
try:
from pip._internal import main as _pip_main
except ImportError:
from pip import main as _pip_main
_pip_main(["install", "packagename"])
This question is, or should be, about how to run pip from a python program. IDLE is not directly relevant to this version of the quesiton.
To expand on J. J. Hakala's comment: a command-line such as pip install pillow is split on spaces to become sys.argv. When pip is run as a main module, it calls pip.main(sys.argv[1:]). If one imports pip, one may call pip.main(arg_line.split()), where arg_line is the part of the command line after pip.
Last September (2015) I experimented with using this unintended API from another python program and reported the initial results on tracker issue 23551. Discussion and further results followed.
The problem with executing multiple commands in one process is that some pip commands cache not only sys.path, which normally stays constant, but also the list of installed packages, which normally changes. Since pip is designed to run one command per process, and then exit, it never updates the cache. When pip.main is used to run multiple commands in one process, commands given after the caching may use a stale and no-longer-correct cache. For example, list after install shows how things were before the install.
A second problem for a program that wants to examine the output from pip is that it goes to stdout and stderr. I posted a program that captures these streams into program variables as part of running pip.
Using a subprocess call for each pip command, as suggested by L_Pav, though less efficient, solves both problems. The communicate method makes the output streams available. See the subprocess doc.
At moment there are no official way to do it, you could use pip.main but you current idle session will not 'see' this installed package.
There been a lot a discussion over how to add a "high level" programmatic API for pip, it's seems promising.
Actually, I think, you can use subprocess.Popen(apt-get numpy), not sure how to do it with PIP though.
If your on a Mac you should be able to do it like this:
Go to your IDLE.
Run help('modules').
Find the HTML module.
Run help('HTML')
There should pop up a file map, for example this/file/map/example/.
Go to the finder and do command+shift+g and paste the file map there. Please delete the last file, because then your gonna go to the modules files.
There are all the modules. If you want to add modules, download the files of the module and put them there.
I hope this helps you.
I have a simple module and a basic def. Module name is example315.py and the def is
def right_justify(s)
print(s)
This works fine when I import example315 and then call example315.right_justify("hello world")
If I change my def to not return anything (in fact I can change it in any way) and then run the function again (AFTER saving my module of course) iit still does the print.
Short of exiting IDLE and starting over I can't seem to get it to work.
Any help appreciated
The module is loaded once per session, you have to re-load it when you change it.
From the Python tutorial on modules:
For efficiency reasons, each module is only imported once per
interpreter session. Therefore, if you change your modules, you must
restart the interpreter – or, if it’s just one module you want to test
interactively, use reload(), e.g. reload(modulename).
The problem you're facing is the fact that IDLE has already imported and built its internal representation of your module. Editing the file on disk won't reflect on the now imported memory-resident version in IDLE. You should be able to get the behavior you're looking for with:
example315 = reload(example315)
And here's some source: Python Docs Source