Use Intel Pin to instrument Python scripts - python

I need to instrument a Python script using Intel Pin for ChampSim simulator.
The problem is, whenever I run the tool, the script does not seem to run as nothing is printed. Moreover, no matter how long/complex the script is, the trace always ends up with a size of 62M (this is also the case when I simply instrument the interpreter without any script).
I tried running the solution from this post, but it didn't work either. For reference, I am running the following command:
../../../pin -t obj-intel64/champsim_tracer.so -- ./python_script.py
Is it even possible to instrument a Python script? If yes, please detail the steps. Thanks!

Related

Does each IDE needs its own Python?

I have PyCharm, Visual Studio Code and Jupyter Lab installed on my PC, and the PC has Python 3.11 installed?
Would someone please educate me on this topic?
If I want run 3 Python programs simultaneously (each program uses its own IDE). Would each IDE run faster if each IDE has its own Python program? Right now all IDEs are using the same Python 3.11 and things are working fine. Just for my own knowledge, I am just curious of this topic. Thank you in advance for educating me!
You have to discriminate the two following concepts: program and process.
Program
A program is a list of instructions. It's generally written on a file.
It can be a binary text but also an ASCII text.
A program does nothing. It is basically a static text.
Process
A process is the execution of a program by your operating system.
A process has a dedicated memory and execution time. Its role is to "play" your code, instruction by instruction.
In your example, python.exe is a program.
When you execute it, your operating system will create a dedicated process.
So, when you run several IDEs, you simply run several process of your same program.
Thus, there is absolutely no loss in terms of performance.

Screenshotting the windows desktop when working through WSL

I'm primarily using Windows, where I run WSL2. So from a python script running in the subsystem, I would like to screenshot whatever is on windows monitor, as simple as such:
v1
import mss
import os
os.environ['DISPLAY'] = ':0'
with mss.mss() as sct:
sct.shot()
This gives only gives "Segmentation fault" error and no image. So I tried to setup vcxsrv in Windows and I'm able to open stuff from my subsystem in Windows through the server, however I cant get it the other way around..
I just want access to the windows screen so I can screenshot it. Any help about how to access the monitor through wsl would be greatly appreciated, I can't find much on google..
The problem with your attempted solution is that the WSL/Linux Python's mss, as you've found, isn't able to capture the Windows desktop. Being the Linux version of MSS, it will only be able to communicate with Linux processes and protocols like X. Starting up VcXsrv might get you part of the way there, in that you might be able to capture X output, but you may need to be running the Python app from inside a terminal that is running in a X window.
Regardless, you've said that your goal is to capture the entire Windows desktop, not just the X output in VcXsrv. To do that, you'll need to use a Windows process of some sort.
But don't worry; using WSL's interop, you can still do that from inside WSL/Linux Python. We just need to call out to a Windows .exe of some sort.
There are literally dozens of third-party apps that you could use in Windows to create a screenshot. But I prefer to use a solution that doesn't require any additional installation.
So I resorted to PowerShell for this, since you can easily call powershell.exe and pass in a script from WSL. There are a number of examples here on Stack Overflow, but I ended up going slightly "lower tech" to try to simplify a bit. The code here is most similar to this solution, so refer to that if you want to expand on this.
From WSL/Linux Python:
import os
os.system("""
powershell.exe \"
Add-Type -AssemblyName System.Windows.Forms
[Windows.Forms.Sendkeys]::SendWait('+{Prtsc}')
\$img = [Windows.Forms.Clipboard]::GetImage()
\$img.Save(\\\"\$env:USERPROFILE\\Pictures\\Screenshots\\screenshot.jpg\\\", [Drawing.Imaging.ImageFormat]::Jpeg)\"
""")
That essentially sends the ShiftPrintScreen key chord to capture the current desktop to the clipboard, then saves the clipboard. It can get slightly hairy with the quoting, since you are essentially wrapping PowerShell inside a /bin/sh inside a Python script.
Note that, even though you are in Linux Python, since it's the Windows PowerShell that we are calling, it takes the Windows path format (C:\Users\Username\Pictures...) rather than the Linux version (/mnt/c/Users/...).
While I didn't have any timing issues with this, you may need to insert small delays. Again, refer to the existing answer for that. This solution is primarily to explain how to do it through WSL's Python using PowerShell.
if you are on a laptop you could hit the windows button and prtsc to take a screenshot and if you are on PC you could use OBS its recording software that can do pretty much anything.

Any command like '%debug' (in jupyter) when running python script?

In jupyter, when an error occurs, we can continuously debug the error with the command %debug. I want to know if there is the similar way in running python script (.py) in the shell.
I know that I can use pdb to make some break points, but I just want to know the way without such a pre-processing (because re-running the code until the error costs a lot of time).
In general, no: it depends on "the shell" that you are running. Jupyter launches with a lot of instrumentation in support of its debugger, assuming that you're using Jupyter because you want those capabilities at the ready.
I presume that you're using some UNIX shell (since you mention pdb); implicitly loading superfluous software is antithetical to the UNIX philosophy.
I think that what you'll need is one of the "after" debugger modes, although that will still leave you without information from just before the error point: those packages cannot do much to trace the history of problem variables.

Python Process (Activity Monitor) Doesn't Stop

I recently started using Python. I realized that even the simplest program starts a Python process that never ends and causes my computer to overheat if I don't manually kill that process.
I've even seen multiple Python processes running at the same time after running a few easy Python programs (Hello, World!) in a row.
I updated to the latest Python version (Python 3.9.3), installed all the Certificates, and tested a few different programs to see if it happens every timeā€”it does. I am using VSC as the IDE but the same situation happens even if I use IDLE.
My questions are: Is this an interpreter issue or a Mac issue? Can it be fixed?
Thank you very much.

Some powershell cmd-lets not available if run from python?

So this is an unusual one, and perhaps I am simply missing the obvious, but I have the following python code that creates a powershell script and runs it.
# Create the PowerShell file
f = open("getKey.ps1", "w")
f.write('$c = Get-BitlockerVolume -MountPoint C:\n')
f.write('$c.KeyProtector[1].RecoveryPassword | Out-File C:\\Temp\\recovery.key\n')
# Invoke Script
startPS = subprocess.Popen([r'C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe',
'-ExecutionPolicy', 'Unrestricted', './getKey.ps1'], cwd=os.getcwd())
result = startPS.wait()
When this is run, it gives me the following error:
The term 'Get-BitlockerVolume' is not recognized as the name of a cmdlet, function, script file, or operable program.
However, if I then go and manually run the generated script, it works perfectly. To add to the oddity, if I run the same command exactly as above ie:
C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe -ExecutionPolicy Unrestricted ./getKey.ps1
it also works exactly as expected.
Clearly, the above error is a powershell error, so it is successfully running the script. It almost seems like powershell somehow knows that this is being run from python and has some restricted library of commands when a script is run from a particular source. I grant that that idea makes no real sense, but it's certainly how things appear.
I don't think this is a permissions issue, because when you run the same command from an unelevated powershell prompt, you get an Access is denied type error, rather than a command doesn't exist kind of error.
Anyway, any help would be greatly appreciated!
Edits
Edit: New evidence to help figure this out:
It's definitely an issue of cmdlets being loaded properly. If I programmatically run a script to dump the list of all available commands to a text file, it is only about 2/3's as big as if I do so through a powershell prompt directly
I bet Python is running as a 32-bit process on 64-bit Windows. In this case, you'll end up running 32-bit PowerShell, which in practice is a Bad Thing since many PowerShell modules depend on native binaries that may not have 32-bit equivalents. I hit this with IIS Manager commandlets--the commandlets themselves are registered in 32-bit PowerShell, but the underlying COM objects they rely on are not.
If you need to run 64-bit PowerShell from a 32-bit process, specify the path as %SystemRoot%\SysNative\WindowsPowerShell\v1.0\PowerShell.exe instead of System32.
System32 is actually virtualized for 32-bit processes and refers to the 32-bit binaries in %SystemRoot%\SysWow64. This is why your paths (and PSMODULEPATH) will look the same, but aren't. (SysNative is also a virtualized path that only exists in virtualized 32-bit processes.)
Adding to what #jbsmith said in the comment, also check to make sure that the environment variable that PowerShell relies on to know where it's modules are is populated correctly when python starts the process.
%PSMODULEPATH% is the environment variable in question, and it works the same way the %PATH% variable does, multiple directories separated by ;. Based on what you say your observed behavior is, it seems that you are using PowerShell 3.0, and cmdlet autoloading is in effect.
The solution here: Run a powershell script from python that uses Web-Administration module got me the cmdlet I needed, however there are still missing cmdlets even when using this method. I'm still at a loss as to why some are loaded and others are not, but for the time being, my script does what I need it to and I can't spend any more time to figure it out.
For reference here is the code that worked for me
startPS = subprocess.Popen([r'C:\Windows\sysnative\cmd.exe', '/c', 'powershell',
'-ExecutionPolicy', 'Unrestricted', './getKey.ps1'], cwd=os.getcwd())
I had the same issue, and it was simply that the BitLocker feature was not installed, hence the module wasn't present.
I fixed it by installing the Bitlocker feature:
Windows Server:
Install-WindowsFeature BitLocker -IncludeAllSubFeature -IncludeManagementTools -Restart
Windows Desktop:
Enable-WindowsOptionalFeature -Online -FeatureName BitLocker -All

Categories

Resources