Develop Reference

Translate youtube_dl command line into python .py file

It's about how to get a list of URLs by using youtube_dl. Although I have been trying all day long, I couldn't work it out. Thus I would like to ask help to translate the following command lines (partially in Linux) into Python codes. I mean in a .py file.
To get JSON data, use command line: youtube-dl -j --flat-playlist 'https://www.youtube.com/c/3blue1brown/videos'
To parser use command line in Linux: youtube-dl -j --flat-playlist 'https://www.youtube.com/c/3blue1brown/videos' | jq -r '.id' | sed 's_^_https://youtube.com/v/_'
The codes above are from: https://web.archive.org/web/20180309061900/https://archive.zhimingwang.org/blog/2014-11-05-list-youtube-playlist-with-youtube-dl.html (The youtube link there was removed so I replaced the youtube link above)

You can use the same command to run inside a .py file using os as follows:
import os
os.system("youtube-dl -j --flat-playlist 'https://www.youtube.com/c/3blue1brown/videos'")
You can pipe the output of the above commands to a file and then process your json file in python.

Empty file when output is written to a file (python gsutil run from cmd)

I use gsutil for uploading a file to Google cloud storage. I would like to write the output to a file.
I have created a shortcut with this command in it
%windir%\system32\cmd.exe /k python2 c:\gsutil\gsutil -m rsync -r -n -d "XX" gs://xx/XX > C:\myoutput.txt
I run the cmd as admin. The output.txt file is created but it's empty after the script exits.
Any idea how I solve this?
Old question:
I have tried adding /myoutput.txt cf here after gs://xx/XX it doesn't works : I get a Access is denied. message.

I guess output went to errorstream, to merge with normal output append 2>&1
To redirect to a file and see on screen you need a tee or t-pipe tool.
There is one contained in GNU utilities for Win32 or one from Bill Stewart's Site
So your command could look like (untested)
%windir%\system32\cmd.exe /k python2 c:\gsutil\gsutil -m rsync -r -n -d "XX" gs://xx/XX 2>&1|tee "%USERPROFILE%\Desktop\myoutput.txt"

try to write to a different place or give the cmd admin, that would be my guess. Hope this helps)

Re-write write-protected file

Every 4 hours files are updated with new information if needed - i.e. if any new information has been processed for that particular file (files correspond to people).
I'm running this command to convert my .stp files (those being updated every 4 hours) to .xml files.
rule convert_waveform_stp:
input: '/data01/stpfiles/{file}.Stp'
output: '/data01/workspace/bm_data/xmlfiles/{file}.xml'
shell:
'''
mono /data01/workspace/bm_software/convert.exe {input} -o {output}
'''
My script is in Snakemake (python based) but I'm running the convert.exe through a shell command.
I'm getting an error on the ones already processed using convert.exe. They are saved by convert.exe as write-protected and there is no option to bypass this within the executable itself.
Error Message:
ProtectedOutputException in line 14 of /home/Snakefile:
Write-protected output files for rule convert_waveform_stp:
/data01/workspace/bm_data/xmlfiles/PID_1234567.xml
I'd still like them to be write-protected but would also like to be able to update them as needed.
Is there something I can add to my shell command to write over the write protected files?

take a look at the os standard library package:
https://docs.python.org/3.5/library/os.html?highlight=chmod#os.chmod
It allows for chmod with the following caveat:
Although Windows supports chmod(), you can only set the file’s read-only flag with it (via the stat.S_IWRITE and stat.S_IREAD constants or a corresponding integer value). All other bits are ignored.
#VickiT05, I thought you wanted it in python. Try this:
Check the original file permission with
ls -l [your file name]
stat -c %a [your file name]
Change the protection to with
chmod 777 [your file name]
change back to original file mode or whatever mode you want
chmod [original file protection mode] [your file name]

How to prevent command prompt ( windows ) from opening file when typing command?

I download source code of my app from google appengine using this command in cmd:
appcfg.py download_app -A <your_app_id> -V <your_app_version> <output-dir>
But, instead of running the command, it opens the file "appcfg.py". So I don't know what to do now.

Sounds like python files are associated with an editor instead of with the python interpreter.
If so you'll have to change the associations for .py files (found in folder options), or call the python interpreter:
C:\path\to\python appcfg.py download_app -A -V

cannot compile cpp files under the python pyc script file

I am trying to compile & execute my hw cpp file under the python script file which we are given by lecturer. the how-to-manual.pdf he sent us it says use:
c:\>python ./submit.pyc problemID -u username -p password -b //submit.pyc is already given to us
and here is the manifest.txt we are given:
[main]
problem = gc
build =
g++ main.cpp -o solver
run =
./solver %f
my cpp file works normally like this:
./solver input_file
However, I am trying (I have to) to do this under the windows OS. I have Python 2.7.x installed and python.exe is in the Command PATH. I can't run it under the linux ssh sytem because there is 2.4.x python installed and I can't touch it (school's system).
Anyway, when I execute the line above, it returns me:
Command execution failed:
g++ solver.cpp -o solver
I think I told everything I can. So, any idea that what I have to do else? except asking to lecturer:)

For the above to work it needs to be able to find g++ so you need to add the directory that it resides in to the PATH environment variable. This can be done from within your python script or on the command line with:
path=Where\g++\lives;%path%
This will only apply within the current DOS session.
Or you can add it permanenty through system settings->advanced settings->environmental variables
You could also look at using a python virtual environments on the schools linux system.