I run a cronjob-script as root user. The script is located in /root/scripts/myscript.py
In this python script I write to a file like this
log = open(os.path.basename(__file__) + ".log", "w+")
print("some text", file=log)
This is supposed to create a file with the name myscript.py.log and it does. But this file is located in /root/myscript.py.log. Why is that? I expected it to be created in the script directory.
I use Debian Squeeze and add the job via crontab -e as root user.
This is because it depends on the cwd when the cron task was started, cron could change directory too so Always use absolute pathnames for jobs run by cron, or pass in an absolute stub from the command line.
In your case crontab -e as root, cron will run your script as root and therefore use the home directory for that user. ie /root
Related
Whenever I use snakemake with singularity and would like create a directory or file, I run into read-only errors if I try to write outside of the snakemake directory.
For example, if my rule contains the following:
container:
docker://some_image
shell:
"touch ~/hello.txt"
then everything runs fine. hello.txt is created inside the snakemake directory. However if my touch command tries to create a file outside of the snakemake directory:
container:
docker://some_image
shell:
"touch /home/user/hello.txt"
Then I get the following error:
touch: cannot touch '/home/user/hello.txt': Read-only file system
Is there anyway to give snakemake the ability to create files anywhere it wants when using singularity?
Singularity by default mounts certain directories including user's home directory. In your first command (touch ~/hello.txt), file gets written to home directory, where singularity has read/write permissions. However in your second command (touch /home/user/hello.txt), singularity doesn't have read/write access to /home/user/; you will need to bind that path manually using singularity's --bind and supply it to snakemake via --singularity-args.
So the snakemake command would look something like
snakemake --use-singularity --singularity-args "--bind /home/user/"
This is basically a duplicate of Change the current directory from a Bash script, except that I'm on windows and want to be able to change my command line directory from a powershell script.
I'm actually running a python script from the powershell file, parsing the output directory from that python script, and then using that output as the dir to change to, so if there's a way to do this from within the python script instead of the powershell file extra points!
To be 100% clear, yes I know about "cd" and "os.chdir", and no that's not what I want - that only changes the current directory for the script or batch file, NOT the command line that you are running from!
Code follows (batch file was my first attempt, I want it to be PS1 script):
sw.bat - parent dir added to sys $PATH
#echo off
set _PY_CMD_="python C:\code\switch\switch.py %*"
for /f "tokens=*" %%f in ('%_PY_CMD_%') do (
cd /d %%f
)
switch.py
import os
import argparse
LOCAL = r'C:\code\local'
def parse_args():
parser = argparse.ArgumentParser(description='Workspace manager tool.')
parser.add_argument('command', type=str, help='The command to execute, or workspace to switch to.')
return parser.parse_args()
def execute_command(command):
if command in ['l', 'local']:
print(LOCAL)
def main():
args = parse_args()
execute_command(args.command)
if __name__ == '__main__':
main()
If I got your question right, a simplistic approach would involve using [SS64]: FOR /F.
script.py:
target_dir = r"C:\Program Files"
print(target_dir)
script.bat:
#echo off
set _PY_CMD="e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe" script.py
for /f "tokens=*" %%f in ('%_PY_CMD%') do (
pushd "%%f"
)
Output:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057062514]> ver
Microsoft Windows [Version 10.0.18362.239]
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057062514]> script.bat
[cfati#CFATI-5510-0:C:\Program Files]>
[cfati#CFATI-5510-0:C:\Program Files]> popd
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q057062514]>
Notes:
I used pushd (instead of cd) because simple popd would return to the original directory. The drawback is that one has to remember (the stack) how many times they called pushd in order to match the correspondent popds. If want to switch to cd, use it like: cd /d %%f
#EDIT0:
At the beginning, the question was for batch only, after that the PowerShell (PS) requirement (which can qualify as a different question) was added. Anyway, here's a simple (dummy) script that does the trick (I am not a PS expert).
script.ps1:
$PyCmdOutput = & 'e:\Work\Dev\VEnvs\py_064_03.07.03_test0\Scripts\python.exe' 'script.py'
Set-Location $PyCmdOutput
Output:
PS E:\Work\Dev\StackOverflow\q057062514> .\script.ps1
PS C:\Program Files>
Try os.chdir in Python:
os.chdir(path)
Change the current working directory to path. Availability: Unix, Windows.
Here's what I would do if you want to do the directory changes purely in batch:
Create a home directory Var of the Batch file you are calling your Python from.
SET homeDir=%cd%
Then use cd to go into the desired directory. Once the program is in the directory, create a new command prompt using start "Title here (optional)" cmd.exe
Finally, return to the home directory with
cd "%homeDir%"
The script might look something like this:
#echo off
REM using cd with no arguments returns the current directory path
SET homeDir=%cd%
REM set home Directory
cd "%desiredDirectory%"
REM moved directory
start "thisIsAName" cmd.exe
REM created new "shell" if you will
cd "%homeDir%"
REM returned to home directory
pause
With the help of CristiFati, I got this working in a batch file. That solution is posted in the question. After some more googling, I've translated the batch file solution to a Powershell Script solution, which is below:
$command = "& python C:\code\switch\switch.py $args"
Invoke-Expression $command | Tee-Object -Variable out | Out-Null
set-location $out
No changes required to the Python file.
Note - this answer is provided for those who want to take in any number of arguments to the python script and suppress stdout when calling the python script.
I have a python script that queries a database. I run it from the terminal with python3 myscript.py
I've added a cron task for it in my crontab file
*/30 9-17 * * 1-5 python3 /path/to/my/python/script\ directory\ space/myscript.py
The script imports a function in the same directory that parses login info for a database located in database.ini in the same directory. The database.ini is:
[postgresql]
host=my-db-host-1-link.11.thedatabase.com
database=dbname
user=username
password=password
port=10898
But currently cron outputs to the file in my mail folder:
Section postgresql not found in the database.ini file
The section is clearly present in the database.ini file, so what am I missing here?
Instead of running "python3 myscript.py" in the directory where it is present, try running it from some other directory (like home directory). Most likely you will see the same issue.
Note that cron's current-working-directory is different on different systems. So, the safest method is to explicitly switch to the directory where your script is and run the command there:
cd /path/to/my/python/script\ directory\ space/ && python3 myscript.py
Try this:
import os
...
change --> filename=database.ini
for --------> filename=os.path.dirname(__file__)+'/database.ini'
So, i need to run this run.sh file and i could not with windows default CMD.
So i installed Cygwin64 Terminal and it acctualy reads the file, but at the end of the reading, it spams an error:
$ /cygdrive/c/Python27/Scripts/./run.sh
Starting scraper
Scrape complete, checking movies with imdb
C:\python27\python.exe: can't open file 'check_imdb.py': [Errno 2] No such file or directory
Inside run.sh:
#!/bin/bash
echo "Starting scraper"
scrapy runspider cinema_scraper.py -t json --nolog -o - > "movies.json"
echo "Scrape complete, checking movies with imdb"
python check_imdb.py movies.json
check_imdb.py is inside run.sh folder.
The file is referenced inside the script as a relative path.
python check_imdb.py movies.json
Relative means that it does not specify the whole path (starting with /), and is interpreted relative to the current directory, which you can find out with :
pwd
A path starting with / is said to be absolute.
The important thing is to remember a script interprets paths relative to the current directory, not the directory where the script is located.
You could change to the directory of the script before running it, with :
cd /cygdrive/c/Python27/Scripts
But if you do that, then you will need to provide an absolute path on the command line to your movies.json file.
Better yet, modify the script to have an absolute path:
python /cygdrive/c/Python27/Scripts/check_imdb.py movies.json
I am trying to run a python file that takes in one argument (CSV file). It works when i run the script in the terminal but it gives errors when i run it in cron.
This is the line that i run in the terminal:
python nb2.py my_csv_file.csv
And here is my code that i am trying to run in cron:
42 13 * * * /usr/local/bin/python2.7 ~/nb/Development/code/nb2.py ~/nb/Development/code/my_csv_file.csv &> /tmp/June_QB_cat.log
The error says that it cannot find a sqlite table file which is already in the code folder.
Note that when you run the command in the terminal, you're in the ~/nb/Development/code directory already and so your current working directory is that; when you run in it cron, it is not. I would suggest either doing (in your cron job) cd ~/nb/Development/code && python nb2.py my_csv_file.csv &> /tmp/logfile.txt or doing an os.chdir("~/nb/Development/code") as the first step in your code. (Also, I'd suggest doing /home/username instead of ~ just in case you aren't running cronjob as your username at some point, but given the error you get, that sounds like it's not an issue)
You can get the path of a file relative to the current script with
import os.path
relative_path = os.path.join( os.path.dirname(__file__), "sqlitetable" )