Tank Day 14: Video Streaming

Today’s job on the Raspberry Tank build diary: getting that webcam up and running. Despite a few annoying diversions, this was remarkably easy!

Step 1: Get the Drivers

The Debian image that was originally offered on the Raspberry Pi website was great, but it was missing a few things in the name of keeping it lightweight – including, unfortunately, the uvcvideo driver that is required to talk to the Logitech C200 webcam I chose for the tank. (It was the cheapest option at the time on the list of confirmed working devices.)

After a while (okay, actually quite a long while) of trying to build my own version of the uvcvideo driver on the Raspberry Pi, I noticed that the Raspberry Pi team had added the driver to newer builds of the kernel. Rather than download and flash a whole new OS image to my SD card, I took advantage of Hexxeh’s rpi-update utility which handles updates automatically. This doesn’t come as part of the Raspberry Pi builds (yet?), so it has to be downloaded separately. (Instructions are here.) After exposing my Pi to the internet for the first time, rpi-update found and installed a new kernel for me. I rebooted and had a working webcam device at /dev/video0!

…but broken WiFi.

Step 2: Fix the WiFi

Unfortunately, the WiFi drivers that we downloaded and installed back on Day 10 were specific to the version of the kernel that came with the default Debian image – and didn’t work at all with the new one.

Luckily, the maintainer of the WiFi driver build had a new version available within a few hours, so repeating the instructions from day 10 resulted in downloading a new, working driver. However, there were a few issues along the way – notably that the downloaded “8192cu-latest.tar.gz” had to be renamed “8192cu.tar.gz” for the build script to find it, and that running the script for the second time adds duplicate entries in /etc/network/interfaces which must be removed for the configuration to be valid. I also found that I had to reboot to get the WiFi dongle working this time – it did not start working immediately during the install process as it had done first time around.

Step 3: Stream the Video

Raspberry Tank with (Semi-Working) Webcam Image

With both webcam and WiFi working, we could finally look at the camera imagery. I took the advice of Miklós from Day 13’s comment thread, and used mjpg-streamer for this.

mjpg-streamer doesn’t provide pre-built binaries for the Raspberry Pi’s ARM architecture, so I had to build my own. Fortunately, this was pretty easy. The source code is available on SourceForge – this was downloaded and unpacked on the Raspberry Pi. To build, mjpg-streamer requires the development libraries for libjpeg. I downloaded libjpeg8-dev for ARM manually from the Debian package repositories, but it could equally be downloaded using apt-get from an internet-connected Pi. With that installed, building mjpg-streamer was as simple as running make clean all in the directory to which it was unpacked.

With that done, mjpg-streamer could be run by issuing the following commands (as root):

export LD_LIBRARY_PATH=.
./mjpg_streamer -o "output_http.so -w ./www"

This runs a simple built-in web server on port 8080, to which we could connect and stream video:

Step 4: Automate It

Once we had a working webcam and streaming software, we needed to make it run on boot-up. This is achieved fairly simply on Debian by creating an “init script” in the right location and linking it appropriately. My script looks like this (bear in mind that if you’re recreating this build, your path to mjpg_streamer may be different to mine):

#! /bin/sh
# /etc/init.d/webcam

# Carry out specific functions when asked to by the system
case "$1" in
  start)
    echo "Starting webcam script"
    export LD_LIBRARY_PATH=/home/pi/webcam/mjpg-streamer-r63
    /home/pi/webcam/mjpg-streamer-r63/mjpg_streamer -o "output_http.so -w /home/pi/webcam/mjpg-streamer-r63/www" &
    ;;
  stop)
    echo "Stopping webcam script"
    killall mjpg_streamer
    ;;
  *)
    echo "Usage: /etc/init.d/webcam {start|stop}"
    exit 1
    ;;
esac

exit 0

This file was placed in /etc/init.d/. It was then made executable and set to run on startup by executing the following commands (as root):

chmod 755 /etc/init.d/webcam
update-rc.d webcam defaults

One restart later, we had a Raspberry Pi that would stream its video by default from a known URL.

Now that this has been achieved, we can embark on the next step of the build – embedding the webcam video on a webpage that also allows for proper remote control of the vehicle. Follow our first steps towards that goal here on the Raspberry Tank Build Diary!

Comments

That's the first time I've seen a debian init script. To my gentoo-befuddled eyes it looks really odd. Why are there close braces after the case definitions? Is that just a bash thing I don't know about?

I'm still really enjoying reading about the project.

I assumed it was just bash's weird case syntax, I don't think there's anything special about init scripts. All the Debian ones I've seen are like that, so I just copy that style whenever I write a new one.

Glad you're enjoying it!

Nice work!

I am having some issues with mjpg_streamer when running under Raspbian. Namely, as soon as I start it, the raspi starts crawling on its knees. The cpu load is around 30%, so that seems fine, but everything is super slow, typing ls takes 10 sec.

Any ideas?

thanks
Daniel

I'm not sure I'm afraid, as I haven't tried Raspbian yet. When I get a chance I'll see what mjpg-streamer's CPU load is on my Debian Squeeze install. (I don't notice any slow-down at the terminal.)

Mour&#227o 31 July 2012

Nice work.
It's really cool if you could get the stress load in your raspberry from the video stream.

Best regards, M. Mour&#227o

Hi again,

Turned out it was a power problem. I used an el-cheapo powered hub that I order from a chinese company on ebay. After opening it up I realised that they added a bunch of 0 Ohm resistors and skipped all capacitors completely...

cheers
Daniel

I used your instructions, i also have logitech c200 webcam , it is recognized, driver is loaded ;
root@raspberrypi: lsmod
gspcazc3xx 38803 0
gspca
main 19788 1 gspcazc3xx
videodev 88790 2 gspca
main

mjpg-streamer starts but it shows errors when i open pi's 8080 http in the browser :

404: Not Found!
Invalid input plugin number

Could You please help me with this ?

Marek: Do you definitely have a Logitech C200? Mine uses the uvcvideo driver, which doesn't appear in your lsmod output. Try running sudo modprobe uvcvideo, then (with your cam connected) see if uvcvideo then appears in your lsmod output.

It would also help if you could run mjpg_streamer manually from the command-line and post the output it produces, this should make it easier to diagnose the problem.

yeah, there was my fault, i use logitech e2500, it looks almost like c200 so that's why i mistaken it. Anyway, i got it running with mplayer (mjpg-streamer, streamer from xawtv , and few other tools failed) using this line :

mplayer tv:// -tv driver=v4l2 device=/dev/video1

It shows many many "artifacts" when displaying a stream, but generally it works,
anyway, thanks for Your reply and good luck with Your projects.

Simon Vans-Colina (@simonvc) 23 September 2012

Hi Thanks for the help, im building a little tank myself. One thing:

I had to

ln -s /usr/include/linux/videodev.h -> videodev2.h

to get it to compile on my Rpi. Thanks again.

Wow, not even a little bit outdated but a whole 101 revisions outdated! I saw a bug report relating to r163 and thought it must have been a typo of r63, but clearly not. Thanks for finding that!

Mour&#227o: Sorry I didn't notice your comment there for some reason! The CPU load from the video stream isn't significant -- the top output below shows it using significantly less CPU than top itself. This is under Raspbian (armhf architecture) rather than armel Debian.

top - 15:12:05 up 5 min,  1 user,  load average: 0.50, 0.50, 0.25
Tasks: 67 total, 1 running, 66 sleeping, 0 stopped, 0 zombie
%Cpu(s): 5.1 us, 11.6 sy, 0.0 ni, 69.8 id, 13.0 wa, 0.3 hi, 0.1 si, 0.0 st
KiB Mem: 189104 total, 61096 used, 128008 free, 9164 buffers
KiB Swap: 102396 total, 0 used, 102396 free, 29472 cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1975 root 20 0 51048 680 524 S 11.1 0.4 0:24.72 rt_http
2237 root 20 0 4648 1396 1044 R 1.6 0.7 0:00.15 top
2081 root 20 0 21576 2020 1832 S 0.3 1.1 0:00.43 mjpg_streamer
1 root 20 0 2136 732 624 S 0.0 0.4 0:01.64 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 0:00.03 ksoftirqd/0
4 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kworker/0:0
5 root 20 0 0 0 0 S 0.0 0.0 0:00.08 kworker/u:0
6 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 khelper
perenecabuto 27 September 2012

If you are getting something like:
404: Not Found!
Invalid input plugin number

Try this:
./mjpgstreamer -i "./inputuvc.so -r 320x240 -y" -o "./output_http.so -w ./www"

I hope this thread is still being monitored. I am new to Linux and the Raspberry Pi. I bit off a bit more then I can chew and agreed to do a project with my son for a special school science event. The initial instructions from Jeremy's Blog seem straight forward so I "assumed" I could simply follow the steps and make it work too.

Nope, not quite.

I have the latest Arm image (and v2 of the Pi) and its been updated. Tried jjstream and can get the Pi to run it, I see its grabbing and outputting from the camera from my Putty session. However, I can't get either the browser or VLC to show the stream. Very frustrating to say the least.

One of the most frustrating steps in this process was trying to find the proper command to start jjstreamer. Jeremy seemed to skip that step :(

I just found this blog discussion and am willing to give it a whorl. Only have a week and a half to get this working and the presentation put together, but also have a federal audit going on at work smack dab in the middle of it all. grrrr.....

Any advice would be much appreciated. Be kind....

Yeah, I still get an email when someone posts a comment! :)

I haven't heard of jjstreamer before so I can't offer much help there, but if you run into trouble getting mjpg_streamer working let me know and I'll help if I can!

I'm still getting notifications too :)

Anyway, if everything fails try mplayer to make a stream - it was the only streamer that worked for my logitech e2500 webcam. You can redirect your stream from mplayer to Pi's web server for example to make it online.

Well I was able to get it going last night but had to switch to Motion and use a different Webcam. I had a MS HD-5000, MS HD-5001, Logitect Desktop Pro (laptop model), and an old Logitect Pro something-or-other camera to work with. I had to switch out the USB hub for a different one. Some of the ports seemed to not work on the older one.

In the end the new usb hub and the MS HD-5001 camera worked clean and clear with no errors, not even a single one.

Tuning Motion is the next step to improve the quality of the video capture. Currently using 640x480 @ 5FPS. Still seems sluggish considering I am running the Pi v2 model and running Arch without a mouse, keyboard, or monitor. The network is hardwired and the Pi configured to use a static address.

Audio capture has not yet been tried. Not sure if I have the guts to try to include that in this project.

So I guess the next step is to try and find a CPU/Memory monitor that can run from within terminal. I intend to use a second Putty session for this. I want to see how high I can set Motion's frame rate before hitting the devices performance ceiling. I need the ability to monitor both sessions and tweek as things progress. I briefly installed Conky but it seems to be focused on using a desktop to display the results, which I will not be using in this project. Have not tried TOP yet.

Suggestions? ( w/configuration settings and command to start it please :) )

Thanks again!

Trying your setup without the X server (the bit that provides a graphical environment) is definitely a good idea. I've never really used a graphical environment on my RPi so I don't know how much of a hit on the system resources it is, but it's probably significant. If you can turn that off - not just a matter of unplugging the monitor but making sure the software itself isn't running - your video processing will probably run faster. If your graphical environment runs automatically on startup, experienced Arch users should be able to advise on the best way to disable it. (The distro I use starts up in a terminal, and you have to run X manually.)

Conky uses a reasonable amount of processing power itself if you're running it with settings such as a transparent background and 1 sec refresh interval. Using top is very easy - you just run it from the console and it starts showing you the top 20-or-so processes ordered by CPU usage, which is probably what you want. From within top, hit q to quit and ? for help if you need to sort by something other than CPU load.

If you wanted to avoid two PuTTY sessions you could use a terminal multiplexer such as screen or tmux - but there's not really anything wrong with running two PuTTY windows at the same time.

I was able to get top working easier then expected. It came preinstalled so it was simple to start and provided me what I needed. I still have very, very slow FPS and no noticible improvements when adjusting the motion.conf settings. I am getting about a 5 sec delay between actual movement and output in VNC. That seems just too slow to be of practicle use. I have the v2 Pi model and have overclocked it to 900mhz. No effect at all on the FPS. Must be a physical limitation at the USB level is my guess.

Is the setup good enough for the original intention? Yes, this is good enough for my sons project. More to learn and that's really what the Pi is all about... learning.

Thanks for letting me bounce this project off of you.

DC

Not a problem. Out of interest, was motion (or anything camera-related) using up significant amounts of your CPU? If so, it's likely that you're being limited by the Pi's processing power - if not, something else is artificially limiting the throughput.

No, nothing over 1.5% CPU usage, with Motion taking an average of 45% with spikes upward of 96% at times. Nothing else is running besides TOP and it was the 1.5% CPU consumer. Memory consumption was negligible across the board. As mentioned in my previous post, I suspect there is a bandwidth limitation on the USB bus that is hampering the throughput significantly. Not sure if different cameras will make a difference if the limitation is on the RPi.

chmm there shouldn't be any USB bandwidth limit if it's 2.0 hub, You can read /sys/bus/usb/devices/usb?/speed - it'll give you the bus speed of the root hub(s) in Mbps: either 1.5, 12 or 480. The first two indicate USB1 (low speed or full speed) and the third USB2

Speed says 480. The path was /sys/bus/usb/devices/usb1. I will dump my motion.conf to my public skydrive folder if anyone would care to look at it. It currently has the increased settings I was trying to use to improve FPS. As stated earlier, they had no effect. Skydrive: https://skydrive.live.com/redir?resid=AB6AF3F2D12F07DD!130

Also, just to be sure I connected the camera to the Pi directly. No change in FPS.

I'm a bit confused that you say nothing on your top output is above 1.5%, but that motion is 45%-96%. If motion is regularly maxing out your CPU that does suggest that your RPi is the limiting factor, not USB.

Sorry for the confusion. Let me try to explain again. Of the top two CPU processes Motion was spiking as high as 90+% but averaged about 40% cpu load. The next highest cpu user was TOP at about 1.5% cpu usage. I lowered the camera resolution to 320x240 and the cpu load for Motion is now idling around 19%. The FPS is the same, very slow. I may try to completely rebuild the image. My early experiments with the other webcam driver may have created this issue.

Is anyone able to explain why I only get a fairly smooth stream, if I select the supported max resolution (MPEG, 1280x720)? Any other supported resolution (except below 320x240) works bad (first picture after 20-30s, ~0,1fps).
With the highest resolution, my connection (WLAN/internet/mobile) is the bottleneck. So I would like to get a slightly lower resolution working like 800x600.

Ahmed T Hawas 03 March 2013

Iam having troubles in Setting up the MJPG Streamer i cant download the Package Libthread and and also i got an error of Failed to connect , kindly if u can support me with the OS name u used and how u get that package libthread

Ahmed T Hawas 03 March 2013

Hello sir
Kindly need your Help in my Gradutaion project i ve check ur TANK project and im using the RPI in my project but iam using it in other porpuse iam making something like Security system depending on Camera so i ve tried to get the MJPG stream but i got Error in lunchin it the error says FAILED TO START i dont knw wat is the reason for that error also i ve seen in ur toturial when u moved to RASPIAN u downloaded a libthread which is no longer availbe to download it i make apt-get install libpthreads-dev i get unable to locate this package , Sorry for disturbance and Thanks

Ahmed Taha

Ahmed: If I recall correctly, for Raspbian you will need to install the package libpthread-stubs0-dev rather than libpthread-dev.

I have not encountered a "Failed to start" error before -- could you post the entire output that you get when running mjpg-streamer? There are a number of people with a similar problem in this thread, hopefully some of their suggestions may help:

http://www.raspberrypi.org/phpBB3/viewtopic.php?f=38&t=19661&p=191825

Ahmed T Hawas 08 March 2013

Thank You , You are the man , btw is there any way i can make the Streaming go Online so that anyone one can see my streaming not only Local network ?

That's up to whatever network your Raspberry Pi is on. If you want it to be accessible from the internet, and the local network it's on includes an internet-connected router, you'll want to open a port on the router and point it at the Raspberry Pi.

For example, if your RPi had IP 192.168.0.100, and your mjpg-streamer web interface was running on port 5000, you could set up a port forwarding rule in your router to do something like:

Incoming, port 80 ---> 192.168.0.100:5000

If you then find your router's public IP address (e.g. using http://whatismyip.com), someone else on the internet could type your IP address into their web browser and they would see the mjpg-streamer web interface.

tejonbikeronbiker 10 March 2013

Hi, to get more FPS you can lower the resolution, with a resolution of 320x240 you can get very fluid images, it's true, the resolution lowers, but trust me, the result of more FPS it's better.

This is how I exec mjpg-streamer:

mjpgstreamer -i "/usr/lib/inputuvc.so -d /dev/video0 -r 320x240 -f 28" -o "/usr/lib/outputhttp.so -p 8090 -w /var/www/mjpgstreamer"

Greetings

Prof Shadoko 31 May 2013

Hello,

I would like to know if you have any lag in the video streaming? I have a similar project underway with a Tiger RC tank, but with Arduino for controlling the tank, and a Raspberry Pi for video streaming. With motion I have about 1 second of lag between the camera and the viewing computer. This make drinving a bit complicated :)

Best regards

Prof Shadoko 31 May 2013

Just for info here is a picture of the finished tank :) You can see the Raspberry Pi-powered camera on top of the turret and the Arduino-driven targeting system: a laser designator corrected for distance with an infrared rangefinder.

Charduino

There is some lag in the video - I haven't done any accurate testing but it feels like about half a second. It's alright if you drive slowly and cautiously, but that's about it. If you're after faster streaming, you may be able to achieve that using something like VLC talking straight to mjpg-streamer, rather than using a web page.

Anonymous 06 June 2013

Ian wrote:

There is some lag in the video - I haven't done any accurate testing but it feels like about half a second. It's alright if you drive slowly and cautiously, but that's about it. If you're after faster streaming, you may be able to achieve that using something like VLC talking straight to mjpg-streamer, rather than using a web page.

Thank ou for the advice - I get from 1/2 to 1s too. That will do for the moment, the Android smartphone I use as a receptor is too old to run VLC, and the tests I made on a newer one didn't gime me a much shorter delay.

Your project is truly awesome.

what should be the extension of the init script...I use
"cd mjpg-streamer/mjpg-streamer & mjpgstreamer -i "/usr/lib/inputuvc.so -d /dev/video0 -r 320x240 -f 28" -o "/usr/lib/outputhttp.so -p 8080 -w /var/www/mjpgstreamer" to start streamer..where sould i edit the given script

akshay, an init script is not quite the same as a normal bash script -- if you want to create one, follow my example above and replace the line beginning '/home/...' with yours. (Yours currently runs cd with a relative path, so you will need to modify your line slightly to make that an absolute path e.g. /home//mjpg-streamer/mjpg-streamer</code>.)</p>

init scripts don't need a file extension, they are files with any name you like that live in /etc/init.d — for example, mine is /etc/init.d/webcam. So long as it's in the right place, the update-rc.d script will find it.

Note that this is for Debian and Debian-derived distros such as Ubuntu, Raspbian etc. If you use a different distro it may have a different way of writing init scripts.

Late 2014 update for all your streaming needs. Linux projects has made a pretty stable, but still work in progress MMAL interfaced UV4L driver, which can stream raw h264 or MJPEG. http://www.linux-projects.org/modules/sections/index.php?op=viewarticle&artid=14

I have tested this and can conclude that there is still the 500ms lag but I can stream 800x600 @ 9-12FPS which has a really sharp "video" This lag seems to decrease slightly at 640x480 and you can get up to 20fps. It works on iPhone safari, Android and almost everything else.

Unfortunately, because the h264 stream is raw, you cannot embed it for HTML5 complaint browsers, yet (it will be possible soon). But using VLC and the --dumux h264 option, we can now watch 1080p at 25-30FPS! There is still that annoying 500-1000ms lag.

I wonder what is causing this lag though. I managed to get 200ms using nginx-rtmp and standard raspciam, piping data - but viewing the video was a nightmare on mobiles and most PCs anyway.

All my testing done on LAN.

Very interesting project and comments here :)

Add a Comment