I am using the Command Prompt in Windows 8 to perform some analytical tasks with Python.
I'm using a few external libraries, some .py files with functions I need, and some set up commands (like setting certain variables based on databases I need to load).
In all there are about 20 statements. The problem is, that each time I want to work on it, I have to manually enter (copy/paste) all of these commands into the Shell, which adds a few minutes each time.
Is there any way I can save all of these commands somewhere and automatically load them into the prompt when needed?
Yes,
Save your commands in any file in your home directory. Then set your PYSTARTUP environment variable to point to that file. Every time you start the python interpreter it will run the commands from the startup file
Here is the link to example of such file and more detailed explanation
If you need to have different start up files for different projects. Make a set of shell scripts one per project. The scripts should look like this:
#!/bin/bash
export PYTHONSTARTUP=~/proj1Settings.py
python
And so on. Or you can simply change the value of PYTHONSTARTUP variable before you start working on a particular project. Personally, I use MacOS with Iterm2, so I set-up multiple profiles for different projects. When I need to work on a particular project I simply launch a tab with the profile configured for the project.
Related
My use case is I want to initialize some functions in a file and then start up IPython with those functions defined. Is there a way to do something like this?
ipython --run_script=myscript.py
In recent versions of IPython, you do need to add the -i option to get into the interactive environment afterwards. Without the -i it just runs the code in myfile.py and returns to the prompt.
ipython -i myfile.py
Per the docs, it's trivial:
You start IPython with the command:
$ ipython [options] files
If invoked with no options, it
executes all the files listed in
sequence and drops you into the
interpreter while still acknowledging
any options you may have set in your
ipythonrc file. This behavior is
different from standard Python, which
when called as python -i will only
execute one file and ignore your
configuration setup.
So, just use ipython myfile.py... and there you are!-)
You can use IPython profiles to define startup scripts that will run every time you start IPython. A full description of profiles, is given here. You can create multiple profiles with different startup files.
Assuming you only need one profile, and always want the same startup files every time you start IPython, you can simply modify the default profile. To do this, first find out where your IPython configuration directory is in an ipython session.:
In [1]: import IPython
In [2]: IPython.paths.get_ipython_dir() # As of IPython v4.0
In [3]: print(ipython_config_dir)
/home/johndoe/.config/ipython
For this example, I am using Ubuntu Linux, and the configuration directory is in /home/johndoe/.config/ipython, where johndoe is the username.
The default_profile is in the profile_default subdirectory. Put any starting scripts in profile_default/startup. In the example here, the full path would be /home/johndoe/.config/ipython/profile_default/startup.
Nowadays, you can use the startup folder of IPython, which is located in your home directory (C:\users\[username]\.ipython on Windows). Go into the default profile and you'll see a startup folder with a README file. Just put any Python scripts in there, or if you want IPython commands, put them in a file with an .ipy extension.
You seem to be looking for IPython's %run magic command.
By typing in IPython:
%run hello_world.py
you'll run the hello.py program saved in your home directory. The functions and variables defined in that script will be accessible to you too.
The following is for the case when you want your startup scripts to automatically be run whenever you use IPython (instead of having a script that you must specify each time you run IPython).
For recent versions (i.e., 5.1.0) of IPython, place one or more python scripts you wish to have executed in the IPYTHON_CONFIG_DIR/profile_PROFILENAME/startup folder.
On Linux, for example, you could put your Python startup code into a file named ~/.ipython/profile_default/startup/10-mystartupstuff.py if you want it to run when no IPython profile is specified.
Information on creating and using IPython profiles is available here.
Update to Caleb's answer for Python 3.5 in Ubuntu 14.04 (Trusty Tahr): I made this answer self-contained by copying relevant parts of Caleb's answer.
You can use IPython profiles to define startup scripts that will run every time you start IPython. A full description of profiles, is given here. You can create multiple profiles with different startup files.
Assuming you only need one profile, and always want the same startup files every time you start IPython, you can simply modify the default profile. To do this, first find out where your IPython configuration directory is in an IPython session:
Input:
import IPython
ipython_config_dir = IPython.paths.get_ipython_dir()
print(ipython_cofig_dir)
Output:
/home/johndoe/.ipython
For this example, johndoe is the username.
Inside the /.ipython folder, the default_profile is in the profile_default subdirectory. Put any starting scripts in profile_default/startup. In the example here, the full path would be
/home/johndoe/.ipython/profile_default/startup
I have separate preprocessing and training Python scripts. I would like to track my experiments using mlflow.
Because my scripts are separate, I am using a Powershell script (think of it as a shell script, but on Windows) to trigger the Python scripts with the same configuration and to make sure that all the scripts operate with the same parameters and data.
How can I track across scripts into the same mlflow run?
I am passing the same experiment name to the scripts. To make sure the same run is picked up, I thought to generate a (16 bit hex) run ID in my Powershell script and pass it to all the Python scripts.
This doesn't work as when a run ID is given to mlflow_start_run(), mlflow expects that run ID to already exist and it fails with mlflow.exceptions.MlflowException: Run 'dfa31595f0b84a1e0d1343eedc81ee75' not found.
If I pass a run name, each of the scripts gets logged to a different run anyways (which is expected).
I cannot use mlflow.last_active_run() in subsequent scripts, because I need to preserve the ability to run and track/log each script separately.
Obviously, the same program is run, and the code is the same. Why do two of them appear the same
When you execute scripts with the same name but different file locations or run configurations this can happen.
It can be useful in a variety of situations; where you want to run the same script in multiple different versions of python. Each Run configuration can be setup to use a separate interpreter. Though, it would be wise to rename the run configuration.
In this case, pycharm simply took the name of the script and appended (1) when it saw another run configuration of that same name already existed.
I have a Python script, and I want it to be autostarted at every login. It's in a linux system. I followed a guide that explains that is enough to create a .desktop file in ~/.config/autostart/*.desktop and write:
[Desktop Entry]
Name=MyApp
Type=Application
Exec=python3 ~/.myapp/myapp
Terminal=false
I tried several times to reboot but the program doesn't execute, even if it seems to be active in the list of application of startup in my lxde environment.
Often things like the ~ (tilde) aren't evaluated when placed in a config file. Try using the complete path instead (/home/user/… instead of ~/…) and see if this works. If this works, you can try to use $HOME instead ($HOME/…) to make this more portable and abstract.
If you want to run your script on terminal login, place it to /etc/profile.d/
For KDE (at least, KDE 5) you could add applications to autorun in System Settings > Startup and Shutdown > Autostart (either *.desktop files or scripts), it adds links to ~/.config/autostart.
You can achieve this by adding this line python /home/user/program.py in your .bashrc file. It will be invoked each time when you login to your system.
I was trying to run Python 3.3 off of my flash drive. I already tried Portable Python, but most of what it had wouldn't open, and it crashed my laptop.
So, I deleted it, and installed regular Python. Then, I wanted to start adding my favorite modules. And, I needed a way to start IDLE without the original shortcut.
To install modules, I added my Python install to my PATH variable.
To make all this easier, I made a batch file, using some code I found on this question.
So far, I have this. It also asks for the drive letter, because that changes from computer to computer.
#echo off
echo This only works if your Python install is added to the system-wide PATH variable
set /p Path="Enter the Drive Letter on this computer. No Symbols, Just the Letter, Capital"
cd %Path%:\Program Files\Python33
echo type this when python comes up...
echo import idlelib.PyShell
echo idlelib.PyShell.main()
echo.
echo.
echo.
echo.
python
It outputs this:
If you go on and follow the instructions and type what it says, it brings up IDLE.
I couldn't figure out how to get the batch file to actually type into the Python prompt, so I told it to tell the user to type what needed to be typed.
What I need to know is, how can I change the PATH variable from within the batch file. Also, how to I remove it when I'm done (this isn't as important, and could even be in a separate batch file).
Or, alternatively, is there a way just to shortcut to IDLE?
Also, is there a way to run .py files without the command line, with the Python install on my flash drive?
Thanks!
You can invoke Python with this command line:
python -c"import idlelib.PyShell;idlelib.PyShell.main()"
and it will bring-up IDLE shell without the need for the user to type anything.
Edit: BTW, Are you sure you really need to change the global path settings. Try to see if the following script can start Python the way you want. You must put it in the root of the USB drive where you have your Python installation.
#echo off
setlocal
set SCRIPT_DIR=%~dp0
:: Removes trailing backslash (for readability in the following)
set SCRIPT_DIR=%SCRIPT_DIR:~0,-1%
set PYTHON_HOME=%SCRIPT_DIR%\Program Files\Python33
set PATH=%PYTHON_HOME%;%PATH%
"%PYTHON_HOME%\python.exe" -c"import idlelib.PyShell;idlelib.PyShell.main()"
Edit: Every process has an associated environment, which is a set of name-value pairs called environment variables. When a process is started it gets a copy of the environment of its parent process. The global OS settings for environment variables are used for processes started directly from the OS (GUI or command line) shell. The set command in batch files sets or modifies an environment variable in the environment of the current process (not globally).
All the set commands you see in the above script change only the environment of the current process. These changes will be seen by the process created by the last line (python.exe) because it is a child process of the command shell (cmd.exe) process that is executing the batch-file.
The line
set PATH=%PYTHON_HOME%;%PATH%
prepends the content of the PYTHON_HOME variable to the PATH variable of the current process. For example, if PATH were c:\foo\bar;d:\aaa\bbb and PYTHON_HOME were c:\python then the new value of PATH will be c:\python;c:\foo\bar;d:\aaa\bbb
It can not be guaranteed this is possible unless you have high enough system privileges that you can change the global path. There is really no way around this on most computers that you do not own, which I imagine is the main purpose. In those cases when you have enough privileges (it is worth a try some systems still allow this for regular users but many others do not) you can use:
setx PATH "%path%;yourpath"
edit and ps:
You can figure out the drive letter without input, if you know the disk label, with something like this:
#echo off
set label=DRIVENAME
set cmd=WMIC logicaldisk WHERE volumename^^="%label%" GET caption
FOR /F "tokens=1" %%G IN ('%cmd% ^| find ":"')DO set pydrive=%%G
echo %pydrive%\pathtopython
rem do your stuff here
the idle started inside the batch will inherit path but other instances will not. Hard to test conclusively tough.
Explanation of the batch script above. The command wmic is short for windows management instrumentation commandline. One can use WMI to do many things one of them is to issue WQL (SQL for WMI) queries as if windows would be a a database. Databases contain many tables in this case the computer is instructed to fetch the table named logicaldisk. Table logicaldisk has 38 columns and one row for each disk connected to the system. This is way to much data for this purpose. So the data is filtered. WHERE causes the database only to spit out rows that contain some specific value in this case its only interested in rows where column volumename ins equal to DRIVENAME, likewise you could use serial number size or any other criteria. Lastly GET is used to limit the columns you get back as results since your only interested in the letter name of the drive that's what you ask. That is called a caption in the table so that what you ask.
Since the command is a bit long, so i put the command inside a variable (not the result) this shortens the for line so it fits stack overflow. Since = need to be escaped i need to use the escape sequence ^ too times so it still es capable in the for loop.
The for loop is used to capture the return value of the wmic command. Since the answer has many lines i filter for only the lines which contain a colon character. And put it to variable pydrive.