Python program Airnef stuck while downloading images - python
I am using Airnef to download pictures from my Canon DSLR camera through python.
I can download one picture without problems so the whole setup seems to work. However, as soon as I want to download another image the software hangs. The code to me looks quite complex.
Two months ago I did post a thread on TestCams.com. Since I haven't gotten a response, I post this as a python-related question here.
The thread
I start airnef from the command line.
python airnefcmd.py --ipaddress 192.168.188.84 --action getfiles --realtimedownload only --downloadexec open #pf# --transferorder newestfirst --outputdir "/Users/besi/Desktop"
I connect the camera and I’m shown some information about my connection:
Connection established to 192.168.188.84:15740
Camera Model “Canon EOS 200D”, S/N “XXXXXXXXX”
Now airnef tells me:
Waiting for realtime photos from camera to download.
Press to exit |
I take a picture and it downloads it as expected:
Downloading “IMG_0084.JPG”: 96%
Airnef then shows some more information about this image:
/Users/besi/Desktop/IMG_0084.JPG [size = 4,602,357] in 1.94 seconds (2.26 MB/s)
I take some more pictures but they’re not downloaded and the software is stuck at the prompt:
Waiting for realtime photos from camera to download. Press to exit \
Source code
The source code is available at the Airnef Website. I created a github repository for tackling this issue: https://github.com/besi/airnef
The place where the code is stuck is at airnefcmd.py:3203
Update: Forum Post
Here is the link to the forum post on testcams.com
Update: Debugging
The first image called IMG_0182 was downloaded successfully.
In the debug output I can see a new picture being taken, but the download is skipped because the prior image was already downloaded:
See airnef.log:433:
filename = DCIM\100CANON\IMG_0183.JPG
captureDateSt = 20180926T071759
modificationDateStr= 20180926T071758
A new image called IMG_0183.JPG was found.
Skipping IMG_0182.JPG - already downloaded this session
The old downloaded image seems to block the further processing of the current image.
Skipping 100CANON - object is not file - MTP_OBJFORMAT_Assocation (0x3001)
Skipping DCIM - object is not file - MTP_OBJFORMAT_Assocation (0x3001)
Waiting for realtime photos from camera to download. Press <ctrl-c> to exit -execMtpOp: MTP_OP_GetObjectHandles - CmdReq payload:
Now we end up again in the loop waiting for more pictures.
When a new picture is taken, the same procedure happens again.
I don't have a compatible camera, so I'm basing my answer solely on the logs (in Debug mode) posted on the forum. Also it was a trial and error suggestion in one of the comments, so it's not the "scientific" approach (where the cause is being identified, and then fixed).
A team (#Besi and I) effort was required in order to come up with this answer (and the credit should be split accordingly).
According to logs, there is a difference between how the 2 files are handled:
...
filename = DCIM\100CANON\IMG_0182.JPG
captureDateSt = 20180926T071747
modificationDateStr= 20180926T071748
Download history file “/Users/besi/Library/Application Support/airnef/appdata/Canon EOS 200D-SN59074c1578e347a3bf1f6f85e8dec624-downloadhist” loaded – 53 entries
>> MTP_OP_GetObject
Downloading “IMG_0182.JPG”: 0%IMG_0182.JPG – downloading next piece, offset=0x0, count=0x100000
...
filename = DCIM\100CANON\IMG_0183.JPG
captureDateSt = 20180926T071759
modificationDateStr= 20180926T071758
Skipping IMG_0182.JPG – already downloaded this session
Skipping 100CANON – object is not file – MTP_OBJFORMAT_Assocation (0x3001)
Skipping DCIM – object is not file – MTP_OBJFORMAT_Assocation (0x3001)
Waiting for realtime photos from camera to download. Press <ctrl-c> to exit -execMtpOp: MTP_OP_GetObjectHandles – CmdReq payload:
...
As seen when handling the 2nd file (IMG_0183.JPG), the existence of the 1st one (IMG_0182.JPG), triggers everything to be abandoned.
Browsing [TestCams]: airnef - Wireless download from your Nikon Camera!, one of the command line argument (actually, there were more that I suggested) caught my eye: --rtd_mtppollingmethod_newobjdetection, and I suggested specifying numobjs (and thus, overriding the default). Apparently, this was the (main) problem. The other part was the presence of --transferorder newestfirst. By default, in Realtime Download mode, it's set to oldestfirst (see below). Removing it (or redundantly specifying --transferorder oldestfirst) did the trick.
Conclusion
In order to fix the problem, 2 things were necessary (in terms of cmdline args for airnefcmd.py):
Specify --rtd_mtppollingmethod_newobjdetection numobjs
Remove --transferorder newestfirst
According to [GitHub]: besi/airnef - (master) airnef/airnefcmd.py#3403:
g.args['transferorder'] = 'oldestfirst' # so that downloadMtpFileObjects() will properly enumerate through multiple realtime images as we add them
I consider this a bug on airnef's side (regarding --transferorder). It's located in either
Code: --transferorder should be ignored when in Realtime mode
Doc: Specify that --transferorder newestfirst is not compatible with Realtime mode
Related
Ros2 RViz: Can't update static costmap layer, no map received
I have the following project, a 4 wheeled robot based on ROS2 official documentation and some additional Internet sources. The link points to a ZIP, simply unpacking it inside "src" of a ROS2 workspace should be sufficient. Link: https://drive.google.com/file/d/1ZmeCwVyIyx-YxXgc8N7IfW2KdkjQGAxS/view?usp=sharing In a zip, the lidar is commented out, just so it doesn't clog the picture. However, before that, I had it enabled, navigated the robot around and got a map (it is in a "maps" folder now). The next step - one I failed - was to load the map into RViz. What I do: Terminal 1: ros2 launch navigation_bot_05 main_simulation_launch.py world:=src/worlds/maze.sdf map:='src/maps/map.yaml' slam:=False This brings up RViz and Gazebo, but RViz doesn't show the robot yet, as it is configured to show map, which is not there yet. To fix it: Terminal 2: ros2 launch slam_toolbox online_async_launch.py Now we have a robot and (if we enable lidar) it can drive around and scan. ... and I am getting "[global_costmap.global_costmap]: Can't update static costmap layer, no map received" Could someone please tell me how to fix this package and what is wrong in it? Thank you.
Saving Video to MJPEG Format Locking Up Application
I am using a RPi camera to capture video in a MJPEG format to file. However, when I attempt to do so, the application freezes up and requires me to manually kill it. I am not sure why it is freezing up, but any ideas to resolve this problem would appreciated. This program entails the following code in accord to the documentation from the following website under the section "start_recording": https://picamera.readthedocs.org/en/release-1.10/api_camera.html#picamera.camera.PiCamera.start_recording The actual program is as listed below: start_recording('/home/pi/testVideo.mjpeg', format = 'mjpeg') camera.wait_recording(5) stop_recording()
Questions regarding Splinter for file download
I have written a simple program to mimic logging into a web page, clicking on a few options and the final step is to click on a link to generate a report. Everything seems to be running fine on my PC and the report actually get downloaded into the default download directory(I am using Chrome). However, when my colleague tried it, the file download was cut off and the browser process terminates itself running the same code and I can't seem to replicate what he saw. My code looks like this: browser = splinter.Browser('chrome') browser.visit('https://village-us.albourne.com/castle/') browser.fill('username','xxx') browser.fill('password','xxxxx') browser.find_by_name('signinform') button = browser.find_by_name('submit_0') button.click() browser.visit('https://village-us.albourne.com/castle/hf/listingoptions') rptButton = browser.find_by_name('buildReport') rptButton.click() browser.find_by_name('checkbox_0').click() browser.find_by_name('checkbox_1').click() excelButton = browser.find_by_id('excelReport').first excelButton.click() So my questions are: 1. Is the excelButton.click() response supposed to be synchronized to the browser's response(per default timeout of course)? 2. Is there a way to change the default timeout period? 3. Since the browser is getting opened, if we run this process as a batch process, will this pose any issue when the screen lock is on? I have read about using the zope-testbrowser which seems like a good alternative for this purpose, but not sure if zope-testbrowser supports file downloads as well.
How can I script video playback with output to multiple screens?
Background I'm attempting to craft a simple video playback script for a small cinema that automates the playing of videos and control of the projector, sound and lighting systems. I have two video outputs, one goes to a monitor in the projection booth, and the other directly to the projector. I desire to play video (and only video) fullscreen to the projector while putting controls and a small (~1/4 screen) preview on the monitor. This will allow the projectionist to view the video being output and control the playback from the monitor in the booth while all the audience ever sees is the video output. Problem I am currently using Python to control VLC player (with libvlc Python bindings) to playback videos. I have everything working fine except that I can't figure out how to get a preview (direct copy) of the video being played fullscreen on the projector output into my GUI. I have tried using the clone filter, but I cant get the cloned window to automagically appear full screen nor in my GUI. The clone filter seems like the logical choice but it seems to be VERY inflexible when it comes to specifying destination screens, fullscreen, etc. I must be able to open video windows full screen on the projector monitor. Professionalism is key and it would look bad if the projectionist had to drag a window over and double click on it when the movie started. Currently Using: Debian Linux Python 2.7 wxPython libvlc I would like to continue using Python as I already have the code for controlling the projector, sound processor, lighting and curtain written and tested. I chose VLC because it really seems bulletproof when it comes to video playback but am not committed to it's continued use. I also chose wxWidgets for my GUI as a result of past experience but I am not stuck on that either.
This describes the direct solution and does not concentrate on any alternative or the overall design of your application. As Your Application and VLC media player are separate processes, you will not be able to get what you want directly because there is no "shared memory" between those 2 applications. The best shot to "copy" the decoded frames from VLC will be to e.g. send a RAW Video .mts stream (ts is usually used for this kind of usecase) and send e.g. to udp://localhost:1234. In your application, you will need to be able to receive the ts stream, "decode" it and display at the spot of interest. For start, i would try if you are able to do this using 2 vlc players that you control manually. When you achieved that the first VLC streams to udp and outputs on the main display at the same time, and the other VLC player receives and plays the udp stream you can go on: Find a player library that you can use directly in your wxpython application and check if it can receive the udp stream as well E.g. https://wxpython.org/Phoenix/docs/html/wx.media.MediaCtrl.html This player lib for example requires gstreamer as a base. As a result, main display and the picture in your applicatoin might have a latency of some seconds. To come around this latency, the best way that i currently know is using WebRTC but this is a lot more complex setup than the above. https://www.sipwise.org/news/technical/tv-over-webrt/ Sure in case you do some "encoding" for WebRTC or even for UDP, you would need to utilize some hardware encoder, e.g. Nvidia NVENC in order to be able to guarantee the needed resources are always there.
Duplicate device input events on unix (/dev/input/event)
I'm using linux/ubuntu, and I would like to play a little bit with my touchpad. I'm trying to use python-evdev to read events from /dev/input/events, for now just printing them: import evdev dev = evdev.InputDevice('/dev/input/event6') import time while True: try: for event in dev.read(): print event except: print " ~ " time.sleep(.5) If I do run that script in with root privilege in a virtual console (outside X, pressing ctrl+alt+F1), the script does print events when I touch the touchpad. Yet, if X is on screen and I run this in a gnome-terminal console, nothing is printed; I somehow guess this is normal, the inputs are intercepted by X. Yet I would like to get them anyway. Is there a way to duplicate whatever comes from /dev/input/event6 so thqt both X and my script can read all events ?
sorry, a bit late on the answer here. Up until version 1.8, the xorg synaptics driver used the EVIOCGRAB ioctl to prevent events to be delivered to other clients. That's disabled now by default, you can still use the GrabEventDevice option to disable it on your machine for older versions (see man synaptics). In short, nothing wrong with your script, it's the synaptics driver that's the problem here. You'll find that your script will work on other devices just fine (though the xorg wacom driver did also grab the device until recently). upstream commit in synaptics: http://cgit.freedesktop.org/xorg/driver/xf86-input-synaptics/commit/?id=f1948e08ee9894864254a18098e4f4fceb6e322f
So, your idea is, X got the data from your touchpad, so that your python code is blocked from receiving touchpad signal, right? Or, may I repeat your words as: at least for some specific kind of device, an application can't get /dev/input/event*, when another is reading from that device? Theoritically, since linux make all devices as a file, you are accessing a file as read-only, while X is also read-only accessing the file. I just did another experiment as: I have a infrared reciver on my archlinux, and I connected to the system in two ssh consoles. I use two ways to access IR, that is, two applications to read the file of /dev/input/event0 (event0 is the SF on my arch): 1, a piece of python code, with evdev; 2, a shell command as: sudo cat /dev/input/event0 | hexdump You can look on the 2nd as working as your X. If you were reasoning correctly, they both should not receive data from the IR (event0) on the same time, when I sending signal from a IR remote controller, right? But, I really got date on the two ssh consoles(I wish I could post image, but I am new with too low reputation to do so). So, I think it should not be the reason. I guess it might be because of your touchpad itself. You know, some devices can only work on a single application. e.g., keyboard can only enter characters on the active application, and some input method just make itself as active over other applications, and redirect after it processed. Also, say, if you had a VM running on your system, and you use only one mouse, what would happen if you are moving pointer on the host desktop? Will the pointer in VM move? Or vice versa? So I need more info about your touchpad. If your TP works only with a single active applicatio, I am afraid you need somethink like a hook to get touchpad signal ahead of X, and redirec it to X and your python code, which might be beyond evdev.
You could create a kernel input handler based on evdev so that the device input is distributed both to the normal /dev/input/eventN and, let's say, /dev/input/copied_eventN X would read from /dev/input/eventN but you would still be able to read from /dev/input/copied_eventN Actually you could very easily create a kernel module by copy-pasting the code in drivers/input/evdev.c