Difference between revisions of "Explorer/RaspberryPi/Visualprocessing/Motionvector/Run"

From PaparazziUAV
Jump to navigation Jump to search
 
(15 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
  rm /tmp/camera*;
  rm /tmp/camera*
  /home/pi/RaspiCV/build/raspicv -t 0 -w 640 -h 480 -fps 30/1 -b 500000 -cd H264 -n -a ENAC -ae 22 -x /dev/null -r /dev/null -rf gray -o - \
  /home/pi/RaspiCV/build/raspicv -v -w 640 -h 480 -fps 30 -t 0 -o /dev/null -x /dev/null -r /dev/null -rf gray
  | gst-launch-1.0 fdsrc \
 
    ! h264parse \
  gst-rtsp-server-1.14.4/examples/test-launch  "shmsrc socket-path=/tmp/camera3 do-timestamp=true ! video/x-raw, format=I420, width=640, height=480, framerate=30/1 ! omxh264enc ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96 config-interval=1" ""
    ! video/x-h264,stream-format=byte-stream \
    ! udpsink host=127.0.0.1 port=5100
  gst-rtsp-server-1.14.4/examples/test-launch  \
  "udpsrc port=5100 do-timestamp=true ! video/x-h264,stream-format=byte-stream,alignment=au ! rtph264pay name=pay0 pt=96 config-interval=1" \
  "shmsrc socket-path=/tmp/camera3 do-timestamp=true ! video/x-raw, format=I420, width=640, height=480, framerate=30/1 ! omxh264enc ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96 config-interval=1"


client<br />
client<br />
gst-launch-1.0 rtspsrc location=rtsp://RASPBERRYPI_IP:8554/test ! rtph264depay ! avdec_h264 !  xvimagesink sync=false
gst-launch-1.0 rtspsrc location=rtsp://RASPBERRYPI_IP:8554/test ! rtph264depay ! avdec_h264 !  xvimagesink sync=false<br />
 
gst-launch-1.0 rtspsrc location=rtsp://RASPBERRYPI_IP:8554/test2 ! rtph264depay ! avdec_h264 !  xvimagesink sync=false<br />
 
___________________
 
Adam Heinrich thesis
An Optical Flow Odometry Sensor Based
on the Raspberry Pi Computer
 
The VideoCore uses two hardware motion estimation blocks for video encoding. A coarse motion estimation (CME) block estimates displacement
in pixel resolution and a subsequent fine motion estimation (FME) block is able to estimate the displacement in a sub-pixel resolution
 
The motion estimation block uses a block matching method to estimate the displacement (∆u, ∆v). For each macroblock in the current frame, the closest match is found in the previous frame within a given range. Vectors from the CME block can be obtained directly from the encoder while vectors from the FME block are encoded in the final H.264 bitstream
 
For each P-frame, the encoder provides a buffer which contains a single 32-bit value for each 16 × 16 px macroblock [Upt14, Hol14]. The most significant 16 bits represent a Sum of Absolute Differences (SAD) value. The SAD value is a measure of the estimated motion’s quality: the lower the SAD, the better match has been found. The other 16 bits represent motion in horizontal and vertical directions (8-bit signed integer per direction)


The number of macroblocks provided by the CME is constant for each frame.
airpi.sh
Moreover, the analysis shows that the CME, in fact, estimates motion in two-pixel resolution (i.e. only even values are present).
...
/home/pi/wifibroadcast-svpcom/wfb_tx -K /home/pi/wifibroadcast-svpcom/drone.key -p 6 -u 5610 wlan1 &
/home/pi/wifibroadcast-svpcom/wfb_tx -K /home/pi/wifibroadcast-svpcom/drone.key -p 1 -u 5600 wlan1 &
...
/home/pi/RaspiCV/build/raspicv -t 0 -w 640 -h 480 -fps 30/1 -b 500000 -cd H264 -n -a ENAC -ae 22 -x /dev/null -r /dev/null -rf gray -o - \
| gst-launch-1.0 fdsrc \
  ! h264parse \
  ! video/x-h264,stream-format=byte-stream,alignment=au \
  ! rtph264pay name=pay0 pt=96 config-interval=1 \
  ! udpsink host=127.0.0.1 port=5610 &
sleep 10
gst-launch-1.0 shmsrc socket-path=/tmp/camera3 do-timestamp=true \
  ! video/x-raw, format=BGR, width=640, height=480, framerate=30/1, colorimetry=1:1:5:1  \
  ! v4l2h264enc extra-controls="controls,video_bitrate=3000000;" \
  ! rtph264pay name=pay0 pt=96 config-interval=1 \
  ! udpsink host=127.0.0.1 port=5600 &
...


A video_splitter component which has the ability to split video streams to multiple outputs. The video_splitter performs format conversion to grayscale so it is not necessary to configure the format at the camera’s output (the camera’s output format is optimized for the most efficient encoding)
groundpi.sh
both encoder_buffer_callback() and splitter_buffer_callback() contain a single line code which passes buffers to the main application for further processing.
...
/home/pi/wifibroadcast-svpcom/wfb_rx -K /home/pi/wifibroadcast-svpcom/drone.key -p 6 -c 127.0.0.1 -u 5600 wlan1 &
/home/pi/wifibroadcast-svpcom/wfb_rx -K /home/pi/wifibroadcast-svpcom/drone.key -p 1 -c 127.0.0.1 -u 5610 wlan1 &
...
gst-launch-1.0 udpsrc port=5600 \
  ! tee name=streams \
  ! queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 \
  ! udpsink host=127.0.0.1 port=5620 streams. \
  ! queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 \
  ! application/x-rtp, encoding-name=H264,payload=96 \
  ! rtph264depay \
  ! video/x-h264,stream-format=byte-stream \
  ! fdsink | /home/pi/wifibroadcast_osd/fpv_video/fpv_video &
...
/home/pi/gst-rtsp-server-1.14.4/examples/.libs/test-launch \
  "udpsrc port=5610 do-timestamp=true ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! video/x-h264,stream-format=byte-stream,alignment=au ! rtph264pay name=pay0 pt=96 config-interval=1" \
  "udpsrc port=5600 do-timestamp=true ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! video/x-h264,stream-format=byte-stream,alignment=au ! rtph264pay name=pay0 pt=96 config-interval=1" &
...


Currently the cv.cpp is limited to 640x480 grayscale image. This can be easily modified (see function cv_init()).
[[Explorer/RaspberryPi/Visualprocessing]]

Latest revision as of 06:19, 29 September 2020

rm /tmp/camera*;
/home/pi/RaspiCV/build/raspicv -t 0 -w 640 -h 480 -fps 30/1 -b 500000 -cd H264 -n -a ENAC -ae 22 -x /dev/null -r /dev/null -rf gray -o - \
  | gst-launch-1.0 fdsrc \
    ! h264parse \
    ! video/x-h264,stream-format=byte-stream \
    ! udpsink host=127.0.0.1 port=5100
gst-rtsp-server-1.14.4/examples/test-launch  \
  "udpsrc port=5100 do-timestamp=true ! video/x-h264,stream-format=byte-stream,alignment=au ! rtph264pay name=pay0 pt=96 config-interval=1" \
  "shmsrc socket-path=/tmp/camera3 do-timestamp=true ! video/x-raw, format=I420, width=640, height=480, framerate=30/1 ! omxh264enc ! video/x-h264,profile=high ! rtph264pay name=pay0 pt=96 config-interval=1"

client
gst-launch-1.0 rtspsrc location=rtsp://RASPBERRYPI_IP:8554/test ! rtph264depay ! avdec_h264 ! xvimagesink sync=false
gst-launch-1.0 rtspsrc location=rtsp://RASPBERRYPI_IP:8554/test2 ! rtph264depay ! avdec_h264 ! xvimagesink sync=false
___________________

airpi.sh 

...
/home/pi/wifibroadcast-svpcom/wfb_tx -K /home/pi/wifibroadcast-svpcom/drone.key -p 6 -u 5610 wlan1 &
/home/pi/wifibroadcast-svpcom/wfb_tx -K /home/pi/wifibroadcast-svpcom/drone.key -p 1 -u 5600 wlan1 &
...
/home/pi/RaspiCV/build/raspicv -t 0 -w 640 -h 480 -fps 30/1 -b 500000 -cd H264 -n -a ENAC -ae 22 -x /dev/null -r /dev/null -rf gray -o - \
| gst-launch-1.0 fdsrc \
  ! h264parse \
  ! video/x-h264,stream-format=byte-stream,alignment=au \
  ! rtph264pay name=pay0 pt=96 config-interval=1 \
  ! udpsink host=127.0.0.1 port=5610 &
sleep 10
gst-launch-1.0 shmsrc socket-path=/tmp/camera3 do-timestamp=true \
 ! video/x-raw, format=BGR, width=640, height=480, framerate=30/1, colorimetry=1:1:5:1  \
 ! v4l2h264enc extra-controls="controls,video_bitrate=3000000;" \
 ! rtph264pay name=pay0 pt=96 config-interval=1 \
 ! udpsink host=127.0.0.1 port=5600 &
...

groundpi.sh 

...
/home/pi/wifibroadcast-svpcom/wfb_rx -K /home/pi/wifibroadcast-svpcom/drone.key -p 6 -c 127.0.0.1 -u 5600 wlan1 &
/home/pi/wifibroadcast-svpcom/wfb_rx -K /home/pi/wifibroadcast-svpcom/drone.key -p 1 -c 127.0.0.1 -u 5610 wlan1 &
...
gst-launch-1.0 udpsrc port=5600 \
 ! tee name=streams \
 ! queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 \
 ! udpsink host=127.0.0.1 port=5620 streams. \
 ! queue max-size-buffers=0 max-size-time=0 max-size-bytes=0 \
 ! application/x-rtp, encoding-name=H264,payload=96 \
 ! rtph264depay \
 ! video/x-h264,stream-format=byte-stream \
 ! fdsink | /home/pi/wifibroadcast_osd/fpv_video/fpv_video &
...
/home/pi/gst-rtsp-server-1.14.4/examples/.libs/test-launch \
 "udpsrc port=5610 do-timestamp=true ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! video/x-h264,stream-format=byte-stream,alignment=au ! rtph264pay name=pay0 pt=96 config-interval=1" \
 "udpsrc port=5600 do-timestamp=true ! application/x-rtp, encoding-name=H264,payload=96 ! rtph264depay ! video/x-h264,stream-format=byte-stream,alignment=au ! rtph264pay name=pay0 pt=96 config-interval=1" &
...

Explorer/RaspberryPi/Visualprocessing

Retrieved from "http://wiki.paparazziuav.org/w/index.php?title=Explorer/RaspberryPi/Visualprocessing/Motionvector/Run&oldid=25189"