I'm stuck with a little problem:
I have a website to manage videofiles for different users. Each user can upload videos to a personal folder which I don't want to change because I don't want to mix up files from different users. After uploading the video file I call a subprocess which should create a thumbnail. The subprocess fails because of an error in ffmpeg, seeming to be related to missing writing permissions. The uploaded file and the containing folder belong to www-data.
The code:
command = ("ffmpeg -ss 00:00:10 -i %s -dframes 1 %s -y" % (video_path, image_path)).split()
subprocess.call(command)
FFMPEG seems to be run as a different user because it only works if the target-folder has 777-permissions. Otherwise it fails with this message:
av_interleaved_write_frame(): I/O error occurred
Usually that means that input file is truncated and/or corrupted.
If I touch the image-file instead of creating it via ffmpeg it doesn't matter if the folder has 775 or 777. The resulting file then also belongs to www-data, which means that the subprocess itself is run as www-data, doesn't it?
I thought about creating a subfolder which has 777-permissions but I don't like it for two reasons: This folder had to be created dynamically because I want to be able to create new users (and resulting new subfolders in my uploads-folder). 777-permissions are no nice solution anyway.
Do you have any suggestions what I have to change so ffmpeg can write to the folder without opening security leaks and without having to touch anything when creating a new user/folder?
I found it!
It was not a permission-problem but something strange in error handling: If the code is run from the webserver the resulting image-file is dismissed when the error occurs. If it is run from command line the resulting file remains in the folder.
So basically I changed my command that no error message appears any more by using -vframes instead of -dframes (which only worked fine in windows):
command = ("ffmpeg -ss 00:00:10 -i %s -vframes 1 %s -y" % (video_path, image_path)).split()
Try specifying -vframes 1 as described here
However, for ffmpeg 0.9 and later both dframes and vframes are aliases for frames, so if you use newer version of ffmpeg, problem is somewhere in other place.
You could run conversion process asynchronously with Celery. Your worker process might be invoked with required permissions, and apache just needs permissions to access communication channel, such as RabbitMQ for example
Related
I am attempting to write a python script that calls rsync to synchronize code between my vm and my local machine. I had done this successfully using subprocess.call to write from my local machine to my vm, but when I tried to write a command that would sync from my vm to my local machine, subprocess.call seems to erroneously duplicate text or modify the command in a way that doesn't make sense to me.
I've seen a number of posts about subprocess relating to whether a shell is expected / being used, but I tried running the command with both shell=True and shell=False` and saw the same weird command modification.
Here is what I am running:
command = ['rsync', '-avP', f'{user}#{host}:/home/{user}/Workspace/{remote_dir}', os.getcwd()]) # attempting to sync from a dir on my remote machine to my current directory on my local machine
print('command: ' + ' '.join(command))
subprocess.call(command)
Here is the output (censored):
command: rsync -avP benjieg#[hostname]:/home/benjieg/Workspace/maige[dirname] /Users/benjieg/Workspace
building file list ...
rsync: link_stat "/Users/benjieg/Workspace/benjieg#[hostname]:/home/benjieg/Workspace/[dirname]" failed: No such file or directory (2)
8383 files to consider
sent 181429 bytes received 20 bytes 362898.00 bytes/sec
total size is 217164673 speedup is 1196.84
rsync error: some files could not be transferred (code 23) at /AppleInternal/Library/BuildRoots/810eba08-405a-11ed-86e9-6af958a02716/Library/Caches/com.apple.xbs/Sources/rsync/rsync/main.c(996) [sender=2.6.9]
If I run the command directly in my terminal, it works just fine. The mysterious thing that i'm seeing is that subprocess appears to append some extra path before my user name in the command. You can see this in the 'link stat' line in the output. I'm guessing this is why it's unable to find the directory. But I really can't figure out why it would do that.
Further, I tried moving out of the 'Workspace' directory on my local machine to home, and even more weird behavior ensued. It seemed to begin looking into all of the files on my local machine, even prompting dropbox to ask me if iterm2 had permission to access it. Why would this happen? Why would whatever subprocess is doing cause a search of my whole local file system? I'm really mystified by this seemingly random behavior.
I'm on MacOS 13.0.1 (22A400) using python 3.10.4
Been trying to get this working all day, and i just can't figure out why its not working.
Trying to implement a simple trigger to run when the user submitted a file.
example in the .tmp file:
hello_trigger change-submit //testDepot/... "python F:/triggers/hello_trigger.py"
when i try to submit a file i get this:
Submit validation failed -- fix problems then use 'p4 submit -c 10199'.
'hello_trigger' validation failed: python: can't open file 'F:/triggers/hello_trigger.py': [Errno 2] No such file or directory
File exists and can be read, so its not a python issue.. same error with a .txt or .bat file.
From what i can gather the problem seems to be coming from the depot line in the trigger.
//testDepot/... fails
//depot/... doesnt fail, but the script is never fired off.
Any suggestions are greatly appreciated.
also testDepot is a stream not sure if that matters.
python: can't open file 'F:/triggers/hello_trigger.py': [Errno 2] No such file or directory
seems pretty clear that the file doesn't exist, at least from the point of view of this trigger command. Some things to double check:
This is running on the server machine, i.e. the place where the p4d service is running. If you have the script on your client machine the Python executable on the server isn't going to be able to find it!
Similarly, this is being run by whatever user is running p4d (on Windows this is usually the SYSTEM user, which may have limited permissions). Does that user have permission to read this path?
Could it be that your version of Python on Windows doesn't know how to handle paths with Unix-style forward slashes? (Many tools will normalize these for you but you shouldn't depend on it!) Try using a valid Windows path, i.e. F:\triggers\hello_trigger.py.
I'm trying to distribute a python script through Pypi. This script takes input from a user and stores it in a text file. However, because the script is creating/writing to a text file, it requires a "sudo" every time it runs. I.e.:
$ my_script
Permission error
$ sudo my_script
Success
I've run into this problem while working on another script and solved it by chmod-ing a newly created file. This way, sudo was required only once to create a file with lowered permissions (that could be written to w/o extra privileges). However, I can't believe that this is the best answer to such a problem— requiring users to give privileges to a no-name script seems awfully suspicious. Is there really not a cleaner way to handle recording data when distributing through Pypi?
I am writing a Network File System, and I have started by trying to understand the xmp.py provided with python-fuse.
Well basically I'm sending all the calls over the network, executing it on the server-side and sending the result to the client and returning it to the user. These parts are working perfectly fine.
The problem I am facing is that, the xmp.py is not able to create a file and hence the problem is showing up in my Network File System too.
The steps I have followed are:
Installed fuse-python by:
sudo apt-get install python-fuse
Then I go into the directory where xmp.py is located:
cd /usr/share/doc/python-fuse/examples/
I ran the xmp.py program with the following command-line arguments
python xmp.py -o root=/home/user /tmp/fuse
where /home/user will be replicated or mounted on /tmp/fuse
Then I cd into the /tmp/fuse directory and try to make a file with cat, like:
cd /tmp/fuse
cat > temp.txt
I get the error
bash: temp.txt: Invalid argument
After a lot of poking around, I believe that this is due to the function
def getattr(self, path):
return os.lstat("." + path)
When I do a cat > temp.txt, os.lstat("./temp.txt") is called and an error is thrown as the file /home/user/temp.txt is not found. After this the program gets stuck for a while and slows down my computer.
Please help me figure out what I'm doing wrong.
Thanks.
I think i have figured it out, the problem that i saw was once os.lstat("./temp.txt") fails, the file class's __init__ method is called and then immediately truncate is called and the program returns.
One solution that i followed was to not use the file class at all, and to implement the Fuse class' open/create to make the file.
This approach worked for me. If anyone has a better suggestion please let me know. I would like to use the object-oriented approach and use the file class.
I am attempting to call ffmpeg to create an image from a frame in a video, I am using python to do this with subprocess.Popen on a mac, eventually this will move to a unix server.
I can successfully create a video from the command line with this line
ffmpeg -i /Users/bimemployee/Movies/ski\ commute.m4v -r .5 -vframes 1 -ss 00:01:14 /Users/bimemployee/Movies/untitled\ folder/image-%d.jpeg
I then turn this into a python iterable and passed it Popen
s=["ffmpeg","-i","Users/bimemployee/Movies/ski\ commute.m4v","-r","1","-vframes","1","-ss","00:01:14","/Users/bimemployee/Movies/untitled\ folder/image-%d.jpeg"]
subprocess.Popen(s)
When I do so I get the standard info screen from ffmpeg and an error that says Users/bimemployee/Movies/ski\ commute.m4v: No such file or directory
Why would this path work ok from the command line but not from python?
Secondly is their a better library for handling this, the ones I could find don't seem to be active projects or don't work with straight python but require things like cython.
Thanks,
CG
You're missing the opening forward slash:
/Users/bimemployee/Movies/ski_commute.m4v
is not the same as
Users/bimemployee/Movies/ski_commute.m4v