When I tried to debug some Deep Learning projects, I noticed that there are many projects use the shell script as the method of passing parameters. We can easily run these projects by bash run.sh in command line, but how to debug these kind of projects in IDEs like Pycharm?
I have tried to use some methods of python to call system commands like: os.system('bash run.sh') in debugger.py and simply debug the debugger.py. But the breakpoint set by pycharm in this method do not work, and will be directly ignored. What is the reason?
I also try to parse the passed parameters in the shell script and add these parameters in the debug configuration of pycharm like this:
debug configuration
but it seems too troublesome, especially when the incoming parameters are complex.
Is there any elegant solution to debug a python project with shell script as the entry?
Or is it not normative to write too complex parameter processing flow in shell script?
Related
I am trying to setup PyCharm to invoke a shell script, instead of python, as the run option. Is this possible? In the default options I only have Python, Python docs, and Python tests. With the professional edition I also have Django and others. However, they all have python as the interpreter in a combobox and they can't be changed as far as I can see.
If you want to see such a feature in PyCharm please vote on IDEA-112256
'Command Line' Run Configuration feature request.
Run/Debug Configurations section of Pycharm's online help lists all supported types of Run/Debug configurations and there's no support for shell scripts indeed.
However, you might be able to add such support by installing a plugin. For example, if you are interested in bash scripts there's BashSupport plugin which adds support for running shell scripts in Run/Debug configuration.
From plugins' home page:
BashSupports can directly run scripts within IntelliJ. You can create
a new run configuration for Bash scripts. Here you can set which
interpreter is used to run it. Whenever a script is executed the
output is logged. If Bash prints out syntax errors then the errorneous
lines are clickable to jump to the location of the error.
For Windows there's CmdSupport plugin which provides an action to run .cmd scripts. It seems it does not support running such scripts as Run/Debug configuration however.
As a workaround you can use Python run/debug configuration, specifying some dummy (empty) Python file to run and use Before launch option specifying External tool and specify path to the script when adding/configuring this external tool. See In IntelliJ IDEA, how can I create a key binding that executes a shell script with the current file as a parameter? for details.
As PyCharm is based on IntelliJ's IDEA platform the question IntelliJ IDEA: Running a shell script as a Run/Debug Configuration is very related.
Speaking of run/debug configurations you might be interested in the plugin Run Configuration as Action which
(...) provides a way to use run configurations as buttons on toolbar.
Or assign shortcuts to execute specific run configuration.
This is really a missing feature that normally should be considered as basic functionality. There are two options
First i tried to create a standard (empty) Python configuration and used the "Before launch"->External tool option, therefore you must create a new external tool definition with Tool Settings:
Program: cmd.exe
Parameters: /C your-batch-file.bat
Working directory $ProjectFileDir$ (In my case $ProjectPath$ was empty)
The annoying thing about this solution is, that the "external tool" standard output is redirected to an extra tab in the console log window which is immediately going into the background when the dummy Python Configuration is executed afterwards.
Second and better is to use python to execute the command. Normally i always use subprocess module but in this case os.system() is the nice and minimal solution. The python run configuration then looks like this
Script: (empty)
Parameters: -c "import os; os.system('your-batch-file')"
Working directory: (select project directory, unfortunately no macros here)
I think the easiest way is to just write a python script that calls the .bat file, and then run that:
from subprocess import Popen
p = Popen("batch.bat", cwd=r"C:\Path\to\batchfolder")
stdout, stderr = p.communicate()
Another hacky solution to this is altering the config/options/jdk.table.xml file in your PyCharm's configuration folder. You simple add another entry in your jdks list:
<jdk version="2">
<name value="Python 3.7 (docker)" />
<type value="Python SDK" />
<version value="Python 3.7.0" />
<homePath value="/path/to/your/shell.sh" />
<roots>
<classPath>
<root type="composite" />
</classPath>
<sourcePath>
<root type="composite" />
</sourcePath>
</roots>
<additional />
</jdk>
After that just select your interpreter for your project and you can use this shell as your interpreter. I used this solution when using interpreter inside docker's image.
I'm using Linux Eclipse (pydev) as IDE to develop python scripts that are launched by an application written in C++. I can debug the python script without problems in the IDE, but the environment is not real (the C++ program sends and receives messages through the stdin/stdout and it's a complex communication channel that I can't fully reproduce writing the messages by hand).
Until now I was using log messages to debug (poor man's debug) but it's getting too complex. When I do something similar in PHP I can just leave xdebug listening and add breakpoints in Netbeans. Very neat and easy. Is it possible to do something like that in Python 3.X (with Eclipse or other IDE)?
NOTE: I know there is a Pydev / Attach to Process functionality, but it doesn't work. Always fails to attach.
NOTE2: There is also a built-in "breakpoint()" in Python 3.7 but it links to a debugger and if also fails, the IDE never gets the control.
After some research, this is the best option I have found. Without any other solution provided, I post it just in case anyone has the same problem.
Python has an integrated debugger: pdb. It works as a module and it doesn't allow to use it if you don't have the window control (i.e. you launch the script).
To solve this there are some coders that have created modules that add a layer on pdb. I have tried some and the most easy and still visual interesting is rpudb (but have a look also to this).
To install it:
pip3 install https://github.com/msbrogli/rpudb/archive/master.zip
(if you install it using the pip3 install rpudb command it will install an old version only valid for python 2)
Then, you use it just adding an import and a function call:
import rpudb
.....
rpudb.set_trace('127.0.0.1', 4444)
.....
Launch the program and it will stop in the set_trace call. To debug it (and continue) open a terminal and launch a telnet like this:
telnet 127.0.0.1 4444
You will have a visual debugger in front of you with the advantage that you can not only debug local programs, but also remote (just change the IP).
I was able to attach PyCharm to a running python process and use break points using PyCharm attach to process
I created a bash script which exec a python script, should work the same with C++
I use debugger for Python in Visual Studio Code. It works great, but in many cases I want to debug scripts which use different CLI arguments. It's very cumbersome to create an entry in lauch.json for each CLI argument group I want to debug.
I'm looking for a way for VSC to ask for CLI arguments which will be passed to the script hen debugger starts. I would be even better is there would be some history of previous arguments.
PS. I know you can attach the debugger and run it via the command line, but I hope there is a clean solution :)
About your question:
I'm looking for a way for VSC to ask for CLI arguments which will be passed to the script hen debugger starts.
I'm sorry to tell you that the request can't be met. For now, there're two ways to debug showed in python-debug:
Add configurations in the launch.json, then press Run->Start Debugging, which will create a debug process automatically that provided by the python extension. During this debug process, all debug configurations depend on the content in launch.json:
Command line debugging. The debugger command line syntax you can refer to commandline-debug. Compared to changing configurations in launch.json every time, this is more convenient.
More information about debug in VS Code, please refer to Debugging
This question is very similar to this one but for PyCharm.
I need to use aws-vault to access AWS resources in my script, but this seems to be impossible to accomplish in PyCharm debugging mode. It gives ability to enter script path, parameters, environment variables and there is also external tools functionality, but neither of these work.
Here is the format that works in shell:
aws-vault exec ${AWS_PROFILE} -- script.py
I thought that I've almost arrived at a solution by using external tools and setting the program to "aws-vault" and its arguments to "exec your-profile -- $FilePath$", but it wants to run the script in $FilePath$, finish and only after completion run the debugged script in PyCharm (which is the same one as the one inserted by $FilePath$).
How it would work for my case is by running needed script in debug mode in conjunction with external tool, so the script would go into arguments of the external tool and run as one command.
There are ways to deal with this by launching PyCharm from command line with aws-vault as a prefix or editing its .desktop file and writing the prefix directly into the Exec field, but the app needs to be restarted when AWS profile has to be changed.
Any ideas would be appreciated, thanks.
I was able to do this by installing the envfile plugin in PyCharm. This plugin can read in a .env file when starting a process. Basically I did the following:
Create a script that generates a .env file, envfile.env and name the script generate.sh
This generate.sh script is a shell script that basically does: aws-vault exec $AWS_PROFILE -- env | grep AWS_ > envfile.env, so all the aws creds are in the envfile.env. Possibly add other environment variables if you need so.
Execute command above at least once.
Install the envfile plugin in pycharm.
In the Run configuration, a new tab appears with 'EnvFile'. In this tab, enable the EnvFile. Add the generated envfile.env (see previous).
Do Tool / External Tools and create an external tool for the generate.sh. This way you can execute the script from PyCharm.
Again in the Run configuration add a Before Launch that executes the External Tool generate.sh.
Warning, the temporary aws-creds are in the plaintext envfile.env.
how to set run arguments for python when using wing IDE?
D:\TEST\djgprj
├─.idea
├─testdjgprj.wpr
├─testdjgprj.wpu
└─testdjg
├─.idea
├─testdjg
├─manage.py
├─static
└─templates
I create a project using wingIDE, and save to folder D:\TEST\djgprj, but the models, manage.py, programs are in folder D:\TEST\djgprj\testdjg.
When I try to run manage.py using wingIDE, I dont know where to set arguments. I know there is one place which is Debug->Run Arguments, but it does not work well.
Right click on the file (either in editor or Project tool) and select Properties and set the run args under Debug > Run Arguments.
If you mean sending arguments to Python itself and not your code, this is the Debug > Python Options property instead (the default is -u for unbuffered output).