Every 4 hours files are updated with new information if needed - i.e. if any new information has been processed for that particular file (files correspond to people).
I'm running this command to convert my .stp files (those being updated every 4 hours) to .xml files.
rule convert_waveform_stp:
input: '/data01/stpfiles/{file}.Stp'
output: '/data01/workspace/bm_data/xmlfiles/{file}.xml'
shell:
'''
mono /data01/workspace/bm_software/convert.exe {input} -o {output}
'''
My script is in Snakemake (python based) but I'm running the convert.exe through a shell command.
I'm getting an error on the ones already processed using convert.exe. They are saved by convert.exe as write-protected and there is no option to bypass this within the executable itself.
Error Message:
ProtectedOutputException in line 14 of /home/Snakefile:
Write-protected output files for rule convert_waveform_stp:
/data01/workspace/bm_data/xmlfiles/PID_1234567.xml
I'd still like them to be write-protected but would also like to be able to update them as needed.
Is there something I can add to my shell command to write over the write protected files?
take a look at the os standard library package:
https://docs.python.org/3.5/library/os.html?highlight=chmod#os.chmod
It allows for chmod with the following caveat:
Although Windows supports chmod(), you can only set the file’s read-only flag with it (via the stat.S_IWRITE and stat.S_IREAD constants or a corresponding integer value). All other bits are ignored.
#VickiT05, I thought you wanted it in python. Try this:
Check the original file permission with
ls -l [your file name]
stat -c %a [your file name]
Change the protection to with
chmod 777 [your file name]
change back to original file mode or whatever mode you want
chmod [original file protection mode] [your file name]
Related
I'm running a Jenkins job that has to compile and package a bunch of java files into a jar file. The Jenkins job is calling a Python script that has the steps to be automated. I've iteratively verified that all pervious steps have worked successfully. But my python code executes
jar cvf jarFile.jar .
which is the last step. The code brakes with the following error
Error occurred while deleting java files....
Usage: jar {ctxui}[vfmn0PMe] [jar-file] [manifest-file] [entry-point] [-C dir] files ...
Options:
-c create new archive
-t list table of contents for archive
-x extract named (or all) files from archive
-u update existing archive
-v generate verbose output on standard output
-f specify archive file name
-m include manifest information from specified manifest file
-n perform Pack200 normalization after creating a new archive
-e specify application entry point for stand-alone application
bundled into an executable jar file
-0 store only; use no ZIP compression
-P preserve leading '/' (absolute path) and ".." (parent directory) components from file names
-M do not create a manifest file for the entries
-i generate index information for the specified jar files
-C change to the specified directory and include the following file
If any file is a directory then it is processed recursively.
The manifest file name, the archive file name and the entry point name are
specified in the same order as the 'm', 'f' and 'e' flags.
Example 1: to archive two class files into an archive called classes.jar:
jar cvf classes.jar Foo.class Bar.class
Example 2: use an existing manifest file 'mymanifest' and archive all the
files in the foo/ directory into 'classes.jar':
jar cvfm classes.jar mymanifest -C foo/ .
My best guess is because the underlying OS is Linux its facing some sort of permission issue OR that a previous process is still occupied with a file that the java subprocess is trying to delete.
Whenever I use snakemake with singularity and would like create a directory or file, I run into read-only errors if I try to write outside of the snakemake directory.
For example, if my rule contains the following:
container:
docker://some_image
shell:
"touch ~/hello.txt"
then everything runs fine. hello.txt is created inside the snakemake directory. However if my touch command tries to create a file outside of the snakemake directory:
container:
docker://some_image
shell:
"touch /home/user/hello.txt"
Then I get the following error:
touch: cannot touch '/home/user/hello.txt': Read-only file system
Is there anyway to give snakemake the ability to create files anywhere it wants when using singularity?
Singularity by default mounts certain directories including user's home directory. In your first command (touch ~/hello.txt), file gets written to home directory, where singularity has read/write permissions. However in your second command (touch /home/user/hello.txt), singularity doesn't have read/write access to /home/user/; you will need to bind that path manually using singularity's --bind and supply it to snakemake via --singularity-args.
So the snakemake command would look something like
snakemake --use-singularity --singularity-args "--bind /home/user/"
I have EV3 Lego Mindstorms and I instaled on it ev3dev operating system. I set the connection with the PC via SSH and using PuTTY I started to "programming". I used the cat > test2.py and wrote this code:
#!/usr/bin/env python3
import ev3dev.ev3 as ev3
motor = ev3.LargeMotor('outA')
motor.run_timed(time_sp = 1000, speed_sp = 500)
I saved the file and initialized it using ./test2.py. I got this output:
-bash: ./test2.py: Persmission denied
What caused it and what should I change?
try this:
sudo python3 test2.py
that will allows you to open almost anything in linux
Use ls -la ./test2.py in order to see the file permissions.
Look at the beginning of the output, you'll see something like this:
-rw-rw-r--
The first - means if is a directory or a file. In this case means that is a file.
Now If you observe the remaining chars there are 3 sets of 3 chars with means the permissions for the owner of the file, the owner group and the last set is for the rest of the users.
We have permissions to read, write and execute and in the example I showed there are read and write permissions for the owner user and the owner group but non permissions for the other users.
As Is said above you can just use sudo every time you execute the script but to run it with root privileges. However I would recommend you change your file permissions and using chmod
sudo chmod +x ./test2.py
This will let you execute the script. Take a look at chmod documentation to learn more: https://help.ubuntu.com/community/FilePermissions
I have the following bash wget command
wget -rHncp --cut-dirs=1 -A .txt -e robots=off -l1 -i ./ids.txt -B 'http://archive.org/download/'
I would like to be able to run the exact same command in Python. However, this python file will have to be compatible to running on a windows machine without bash or the wget command. Would I be able to use the wget python package for this? I've tried, but I haven't figured out how to pass it flags yet.
Edit: March 31, 2017
The purpose of the command is to download multiple files from archive.org's website. Given a filetype (in this instance a .txt) and given a file with a list of identifiers (in this instance, the file is call ids.txt), this command will download every txt file associated with the given identifiers.
One such identifier is aeneid_391 and the resulting file from this identifier is virgiletext95anide10_djvu.txt.
I trying to setup virtualenvwrapper in GitBash (Windows 7). I write the next lines:
1 $ export WORKON_HOME=$HOME/.virtualenvs
2 $ export MSYS_HOME=/c/msys/1.0
3 $ source /usr/local/bin/virtualenvwrapper.sh
And the last line give me an error:
source /usr/local/bin/virtualenvwrapper.sh
sh.exe: /usr/local/bin/virtualenvwrapper.sh: No such file or directory
I find, where on my drive is virtualenvwrapper.sh and change directory name. On my computer it's /c/Python27/Scripts/virtualenvwrapper.sh. When I again run command
$source /c/Python27/Scripts/virtualenvwrapper.sh
I get the next ERROR message:
sh.exe":mktemp:command not found ERROR: virtualenvwrapper could not create a temporary file name
I check my enviroment variable: C:\python27\;C:\python27\scripts\;C:\python27\scripts\virtualenvwrapper.sh\;C:\msys;C:\Program Files (x86)\Git\cmd;C:\Program Files (x86)\Git\bin\
I don't know where i made a mistake
The error is saying that sh.exe (the shell) can't find a command matching mktemp, which means it's not present in GitBash, at least not in your environment.
One option is you could download a Windows version of mktemp, such as http://gnuwin32.sourceforge.net/packages/mktemp.htm and then place it in the C:\Program Files (x86)\Git\bin directory. The shell should then be able to match the mktemp command and be able to proceed.
I've found a fix for this problem on a Windows 8 machine using GitBash.
TL;DR:
Get mktemp for windows, put it somewhere that can be used by GitBash, then edit virtualenvwrapper.sh and on line 202, add a touch command with the file created. It should look like this:
file="$(virtualenvwrapper_mktemp -t virtualenvwrapper-$suffix-XXXXXXXXXX)"
touch $file # this is the new line
if [ $? -ne 0 ] || [ -z "$file" ] || [ ! -f "$file" ]
FULL ANSWER:
As khampson mentioned, you do have to download mktemp and place it where your Git\bin (C:\Program Files (x86)\Git\bin usually) directory is. After that, running the virtualenvwrapper.sh file would cause an error saying:
path = C:/Users/User/AppData/Local/Temp/virtualenvwrapper-initialize-hook-XXXXXX XXXX
lpPathBuffer = C:\Users\User\AppData\Local\Temp\
szTempName = C:\Users\User\AppData\Local\Temp\tmp23A9.tmp
path = C:\Users\User\AppData\Local\Temp\tmp23A9.tmp
fd = 3
ERROR: virtualenvwrapper could not create a temporary file name.
On line 202(source), you see a function call to virtualenvwrapper_mktemp (which is just a wrapper function to call mktemp) and this is supposed to create the new temp file, but apparently it doesn't on windows.
Going through the manual for mktemp, on the examples section, you see that they are always sending something to that new file handle which forces the file to be created.
So instead of sending an empty string using echo like the manual, just add a touch command to the virtualenvwrapper.sh:
file="$(virtualenvwrapper_mktemp -t virtualenvwrapper-$suffix-XXXXXXXXXX)"
touch $file # new command here
This should force windows to create the temp file. I can't post the rest of the links due to low rep but I hope this still helps someone.
EDIT
I created a pull request on the virtualenvwrapper repo and it got approved. You can see the touch command I suggested added here.