This question is very similar to this one but for PyCharm.
I need to use aws-vault to access AWS resources in my script, but this seems to be impossible to accomplish in PyCharm debugging mode. It gives ability to enter script path, parameters, environment variables and there is also external tools functionality, but neither of these work.
Here is the format that works in shell:
aws-vault exec ${AWS_PROFILE} -- script.py
I thought that I've almost arrived at a solution by using external tools and setting the program to "aws-vault" and its arguments to "exec your-profile -- $FilePath$", but it wants to run the script in $FilePath$, finish and only after completion run the debugged script in PyCharm (which is the same one as the one inserted by $FilePath$).
How it would work for my case is by running needed script in debug mode in conjunction with external tool, so the script would go into arguments of the external tool and run as one command.
There are ways to deal with this by launching PyCharm from command line with aws-vault as a prefix or editing its .desktop file and writing the prefix directly into the Exec field, but the app needs to be restarted when AWS profile has to be changed.
Any ideas would be appreciated, thanks.
I was able to do this by installing the envfile plugin in PyCharm. This plugin can read in a .env file when starting a process. Basically I did the following:
Create a script that generates a .env file, envfile.env and name the script generate.sh
This generate.sh script is a shell script that basically does: aws-vault exec $AWS_PROFILE -- env | grep AWS_ > envfile.env, so all the aws creds are in the envfile.env. Possibly add other environment variables if you need so.
Execute command above at least once.
Install the envfile plugin in pycharm.
In the Run configuration, a new tab appears with 'EnvFile'. In this tab, enable the EnvFile. Add the generated envfile.env (see previous).
Do Tool / External Tools and create an external tool for the generate.sh. This way you can execute the script from PyCharm.
Again in the Run configuration add a Before Launch that executes the External Tool generate.sh.
Warning, the temporary aws-creds are in the plaintext envfile.env.
Related
When I tried to debug some Deep Learning projects, I noticed that there are many projects use the shell script as the method of passing parameters. We can easily run these projects by bash run.sh in command line, but how to debug these kind of projects in IDEs like Pycharm?
I have tried to use some methods of python to call system commands like: os.system('bash run.sh') in debugger.py and simply debug the debugger.py. But the breakpoint set by pycharm in this method do not work, and will be directly ignored. What is the reason?
I also try to parse the passed parameters in the shell script and add these parameters in the debug configuration of pycharm like this:
debug configuration
but it seems too troublesome, especially when the incoming parameters are complex.
Is there any elegant solution to debug a python project with shell script as the entry?
Or is it not normative to write too complex parameter processing flow in shell script?
I am trying to setup PyCharm to invoke a shell script, instead of python, as the run option. Is this possible? In the default options I only have Python, Python docs, and Python tests. With the professional edition I also have Django and others. However, they all have python as the interpreter in a combobox and they can't be changed as far as I can see.
If you want to see such a feature in PyCharm please vote on IDEA-112256
'Command Line' Run Configuration feature request.
Run/Debug Configurations section of Pycharm's online help lists all supported types of Run/Debug configurations and there's no support for shell scripts indeed.
However, you might be able to add such support by installing a plugin. For example, if you are interested in bash scripts there's BashSupport plugin which adds support for running shell scripts in Run/Debug configuration.
From plugins' home page:
BashSupports can directly run scripts within IntelliJ. You can create
a new run configuration for Bash scripts. Here you can set which
interpreter is used to run it. Whenever a script is executed the
output is logged. If Bash prints out syntax errors then the errorneous
lines are clickable to jump to the location of the error.
For Windows there's CmdSupport plugin which provides an action to run .cmd scripts. It seems it does not support running such scripts as Run/Debug configuration however.
As a workaround you can use Python run/debug configuration, specifying some dummy (empty) Python file to run and use Before launch option specifying External tool and specify path to the script when adding/configuring this external tool. See In IntelliJ IDEA, how can I create a key binding that executes a shell script with the current file as a parameter? for details.
As PyCharm is based on IntelliJ's IDEA platform the question IntelliJ IDEA: Running a shell script as a Run/Debug Configuration is very related.
Speaking of run/debug configurations you might be interested in the plugin Run Configuration as Action which
(...) provides a way to use run configurations as buttons on toolbar.
Or assign shortcuts to execute specific run configuration.
This is really a missing feature that normally should be considered as basic functionality. There are two options
First i tried to create a standard (empty) Python configuration and used the "Before launch"->External tool option, therefore you must create a new external tool definition with Tool Settings:
Program: cmd.exe
Parameters: /C your-batch-file.bat
Working directory $ProjectFileDir$ (In my case $ProjectPath$ was empty)
The annoying thing about this solution is, that the "external tool" standard output is redirected to an extra tab in the console log window which is immediately going into the background when the dummy Python Configuration is executed afterwards.
Second and better is to use python to execute the command. Normally i always use subprocess module but in this case os.system() is the nice and minimal solution. The python run configuration then looks like this
Script: (empty)
Parameters: -c "import os; os.system('your-batch-file')"
Working directory: (select project directory, unfortunately no macros here)
I think the easiest way is to just write a python script that calls the .bat file, and then run that:
from subprocess import Popen
p = Popen("batch.bat", cwd=r"C:\Path\to\batchfolder")
stdout, stderr = p.communicate()
Another hacky solution to this is altering the config/options/jdk.table.xml file in your PyCharm's configuration folder. You simple add another entry in your jdks list:
<jdk version="2">
<name value="Python 3.7 (docker)" />
<type value="Python SDK" />
<version value="Python 3.7.0" />
<homePath value="/path/to/your/shell.sh" />
<roots>
<classPath>
<root type="composite" />
</classPath>
<sourcePath>
<root type="composite" />
</sourcePath>
</roots>
<additional />
</jdk>
After that just select your interpreter for your project and you can use this shell as your interpreter. I used this solution when using interpreter inside docker's image.
I am trying to run this GitHub project in python, but I could only run it using the Terminal of Pycharm IDE.
According to the guide from the GitHub repository, I removed the $ sign from the beginning of $ python train.py RGCN PPI and could run it there. What does $ mean here and how can I run a file like this in Python Console (for example after >>> sign)?
The '$' isn't part of Python's syntax, it's a visual cue in the documentation representing the command prompt.
To answer the question from the title of this post, I'll provide some
instructions first on how to load scripts into the Python console.
However, for your specific case, you don't need this. Scroll down to
the part about debugging in PyCharm.
There's two ways you can get your script into the console. One is to simply load it using the right version of the two lines I give right below, or you can load it as a module - even if it wasn't intended to be one.
In general, to execute a script in the Python shell on Python 2 you can do
>>> execfile(r"<path to script here>")
On Python 3 it's more verbose:
>>> exec(open(r"<path to script here>").read())
The effect this has is as if you cut-n-pasted the script into the console. The console's global scope will get all the functions, classes, and variables that are leftmost indented in the file. Also it might not run your if __name__ == '__main__': block. But you could hack that.
If you want the vars/classes/etc to be put in another scope than your console's global scope, then there are two additional parameters to the above commands. The first one is a dictionary for the globals , and one for the locals. You can get away with only supplying the globals parameter - it's just an ordinary dictionary object you need.
If the file you want to load is a module, you could import it as you would any other module by appending its home folder to the Python module search path, and using the import directive. You can load your script this way even if it wasn't intended to be module.
>>> import sys
>>> sys.path.append(r'/Users/todd/projects/mymodule_folder')
>>> import mymodule
If you make modifications to it and want to reload it:
>>> import importlib
>>> importlib.reload(mymodule)
Loading your script as a module avoids polluting your console's global scope. After it loads, just prefix the names of your script's functions and variables with the module name. The module name will be the name of the file without the .py extension.
If the script requires command line options, you could just hard code values for those into the script and disable lines of code that try and get values from the CLI. If it gets complicated, consider running it in an IDE as described in the next section.
So the above is how you can run your python scripts in whatever Python REPL console you want.
BUT loading your scripts into the Python console may not be at all
required for your purposes. You wanted to debug some scripts (train.py,
test.py) from this project:
https://github.com/microsoft/tf-gnn-samples).
Debugging Command Line Script With PyCharm
In many cases, a Python script is written to run from the OS shell and take command line options from the user. These kinds of script could be loaded into the Python console, but most require some minor hacks to run. However, if all you want to do is debug such a script, you don't need to muck with the console.
PyCharm supports running these as is (as does Eclipse and other IDEs) like any other script. It's just a matter of creating a run/debug configuration for the project. I just installed PyCharm and gave it a try in order to record the details. Easy task.
Just open the project in PyCharm, and above the editor pane, on the toolbar, there's a menu option for Edit Configurations. Click that to open the Run/Debug Configurations dialog and click the + to add a configuration. A small dialog will appear with predefined templates - select Python as your template and accept.
Then in the main dialog, fill in Script path: with the path to train.py (or another script), then click the checkbox, [x] Emulate terminal in output console. Also, you can add command line options in the Parameters: text box (I put in the text: mymodel mytask just to satisfy the script's need for two parameters). Click OK at the bottom to accept the configuration and shut the dialog.
Now you should see a green bug icon on the toolbar.Set a breakpoint in the __main__ block of the script and click the debug icon to start debugging the script. That should do it!
Debugging Python Command Line Script with PDB
PDB - the Python Debugger can be run without an IDE. This is another way to debug a script of any sort. If it requires command line parameters, provide them from the OS shell when you start the debugger:
$ pdb myscript.py mymodel mytask
That's really all there is to starting a debug session. PDB requires some knowledge of its text based commands. After starting a session, you can get a listing of code near the current line of execution by entering l. Enter help to see a listing of the commands.
To step one line of execution, enter 's' for step, or enter 'step'. To set a breakpoint, enter break <line-number>, or set a breakpoint on an expression. A reference on the commands available can be found here: https://docs.python.org/2/library/pdb.html . There are also plenty of versions of pdb cheatsheets available online - just google "pdb cheatsheet" and select one.
I have a Python script, and I want it to be autostarted at every login. It's in a linux system. I followed a guide that explains that is enough to create a .desktop file in ~/.config/autostart/*.desktop and write:
[Desktop Entry]
Name=MyApp
Type=Application
Exec=python3 ~/.myapp/myapp
Terminal=false
I tried several times to reboot but the program doesn't execute, even if it seems to be active in the list of application of startup in my lxde environment.
Often things like the ~ (tilde) aren't evaluated when placed in a config file. Try using the complete path instead (/home/user/… instead of ~/…) and see if this works. If this works, you can try to use $HOME instead ($HOME/…) to make this more portable and abstract.
If you want to run your script on terminal login, place it to /etc/profile.d/
For KDE (at least, KDE 5) you could add applications to autorun in System Settings > Startup and Shutdown > Autostart (either *.desktop files or scripts), it adds links to ~/.config/autostart.
You can achieve this by adding this line python /home/user/program.py in your .bashrc file. It will be invoked each time when you login to your system.
I am starting to learn python with sublime and ipython. They are cool tools and but I want a way to connect them.
I normally have a sublime and a IPython console open. Is there any command that I can run in sublime just send:
runfile('~\someExample.py', wdir='~\myDir')
to the running IPython console?
Thanks!
I edit in geany and use, in ipthon:
run myfile
to load and run myfile.py. ls and cd and pwd are available to check and change the directory.
I save the file from the editor, but I control the run from the ipython console.
If I just need to run the script, I could do a python myfile.py in the editor's terminal window, or maybe via the editor's execute shortcut. But the value in running the code in an existing ipython console is that I can interact with the newly loaded code and variables. I can examine variables, rerun functions with new values, etc.
There is a plugin to do this called SendCode; you can install it directly through the package manager.
As another answer mentioned, you can run some set of shell commands from within ipython, including execution of a file with run python_file.py, as well as other shell commands including cd, mkdir, rm, mv, etc.