3D Camera software development

$2,150 of $20k goal

Raised by 41 people in 27 months
Daniel Patrick Johnson  PORTLAND, OR
I have a rare opportunity to develop open source software, and tutorials for hardware that hasn't been released yet. Considering that google currently has zero results for using the new intel Real Sense 3D Camera with Linux I'm possibly the only person working on it. If you were to send me money I could potentially have working software ready by the time you can buy. My development time is limited, so I can only spend my time on it if you contribute now.

I’d like to build an app that gets the data into a point cloud, and has basic OpenGL visualization of the data as it’s recorded. I definitely want something useable with Point Cloud Library, and Meshlab. I would like to use that as the basis of a tutorial on how to use the camera to do useful things. I would also like to write a websocket app to make it easy to get the data into other applications such as a web browser.

Blog
intel RealSense camera on Linux 

Part 2
3d Camera Controls

Part3
RealSense 3d camera on Linux Mac OS X. infrared, calibration, point clouds

Software developed so far is available on github  https://github.com/teknotus/depthview
+ Read More
Update 27
Posted by Daniel Johnson
6 months ago
   Share
I wrote a new blog post. It's a status of the 3d camera tech post. I'm going to get back to in depth tech in the next post. Also Razer just started shipping a new RealSense SR300 based camera today. It is the first stand alone intel camera for consumers. I signed up for an affiliate account so I think if you buy one via the link on my blog I get a tiny cut. I don't get anything for a click, but I think it sets a cookie in case you buy something from them later.

http://solsticlipse.com/2016/09/26/long-road-to-ubiquitous-3d-cameras.html
+ Read More
Update 26
Posted by Daniel Johnson
8 months ago
   Share
A little over a year after I started this project Intel released cross platform open source support. They basically used my blog posts as a design guide while reimplementing almost all of it from scratch. Intel has been reasonably good at managing the code as an open source project, but they are missing some critical things. For example they only did the minimum amount of kernel work to make it functional, and only attempted to upstream half of that.

https://github.com/IntelRealSense/librealsense/issues/48

Besides two cameras having no support sent upstream there hasn't been work on calibration, or a serious kernel issue that currently requires a camera reset to work around. I've become very interested in the kernel side of things, and spent a huge amount of time learning kernel development so that I can work on these types of problems.

So far I wrote a driver from scratch that can fetch data from the camera. It needs a lot of work before it would be seriously considered upstream, but the steepest part of the learning curve is done.

As it is a different software stack is needed for each 3d camera on the market. When working to get code upstreamed I plan to address this as well by trying to get standard methods for working with 3d cameras. Doing this would be the basis of allowing programs to be written the same no matter what vendor built the camera. Interestingly many of the problems are the same for 360 degree video so this may solve more than one problem. Here is a link to my driver in progress.

https://github.com/teknotus/iclcam

Next up is a blog post detailing this and other 3d camera issues that don't seem to be actively developed. I increased my funding goal considerably in the hopes that I can afford to work on this full time.
+ Read More
Update 25
Posted by Daniel Johnson
15 months ago
   Share
I got my laptop with a builtin R200 about a month ago, and since then I've worked almost every day on figuring out hardware of that machine. Besides the obvious R200 there are two other color cameras (5 total), GPS, accelerometer, gyro, and magnetometer. You can imagine why I would want to get that other hardware working for 3d mapping projects. I created several tools for figuring it out. For example I added UVC support to strace so that I could have someone with the ROS library working figure out how it was retrieving the factory calibration. I nearly had the R200 stuff figured out, and then today Intel finally made good on their promise to provide open source support for RealSense quite a few months after announcing it would be available soon. It's just support for raw data, calibration, and controls at this point. No gesture stuff, but it was a huge amount of work for them to put that together.

https://github.com/IntelRealSense/librealsense

This lets me shift my focus to application stuff, but there are a few things I learned in my investigations that aren't represented in the new library so I may still do some driver type work.

Also today an open source app for RealSense on OSX was released. The email I got for that preceded the one for librealsense by a few hours.

http://www.cmanders.com/2016/01/the-intel-realsense-camera-working-in.html

+ Read More
Update 24
Posted by Daniel Johnson
16 months ago
   Share
I got my intel 3d camera a year ago today. I've made a lot of progress, but still quite a ways away from the dream. I haven't been able to work on it as consistently as I would like because of things life threw at me, but I've almost always answered questions within a few days. The last few weeks I've been actively working on the project quite a bit. The only public bit of that work so far is a kernel package for Unbuntu 15.10 that combines my patch for the F200 with the patch intel recently released for the R200. Although intel has now released some free software for one of their cameras it's only enough to get raw data with no calibration, and no access to the camera controls. There is still much work to be done even to get the basics working much less gestures. Announcing release schedules has a funny way of jinxing them so I'm not going to tell you my current focus until I have code to share. The project just hit $2k which I've spent several times over in the last year, but every bit helps. Thank you so much. Actually I spent that much in the last week. Mostly to replace my years old slowly failing laptop with one that has an integrated RealSense camera. I've had multiple offers to fund my work with conditions that would cede too much control, and possibly prevent me from sharing what I learned about the cameras. I keep turning them down, and instead working for feature bounties from smaller companies, and everyone who has backed my project.
+ Read More
Read a Previous Update
Daniel Patrick Johnson
16 months ago

The forums indicate that they will have another batch to ship out in January. I just got invited to an R200 hackathon at the end of February where they will be giving away cameras. The F200 is getting a replacement with a new name SR300. The R200 is similar enough to the F200 that I've managed to figure out a lot just from emailing back and forth with people lucky enough to have one. Hopefully the same will be true of the SR300. Probably will for the same reason that Windows has ancient code in it, "the code works, don't fix it".

+ Read More
Steve Baker
16 months ago

FYI: Intel have stopped making the F200 and R200 cameras - their web site shows them perpetually "Out of Stock". Similar cameras have appeared in some laptops - and eventually, we'll start to see them showing up in more phones and tablets. But perhaps we've seen the end of the USB devices.

+ Read More

$2,150 of $20k goal

Raised by 41 people in 27 months
Created January 9, 2015
Daniel Patrick Johnson  
PG
$50
Paul Gregoire
2 months ago
$100
Anonymous
15 months ago
GP
$80
George Profenza
16 months ago

Looking forward to try it ^_^ Good luck!

RM
$40
Richard Moats
18 months ago
$50
Alexander Gorban
18 months ago

Thanks a lot for your work!

$150
Imad M'Hamdi
19 months ago

Thanks very much for your work !

$50
Anonymous
19 months ago
RE
$10
Robert Eckstein
20 months ago
$10
Anonymous
20 months ago
$25
Anonymous
21 months ago
Daniel Patrick Johnson
16 months ago

The forums indicate that they will have another batch to ship out in January. I just got invited to an R200 hackathon at the end of February where they will be giving away cameras. The F200 is getting a replacement with a new name SR300. The R200 is similar enough to the F200 that I've managed to figure out a lot just from emailing back and forth with people lucky enough to have one. Hopefully the same will be true of the SR300. Probably will for the same reason that Windows has ancient code in it, "the code works, don't fix it".

+ Read More
Steve Baker
16 months ago

FYI: Intel have stopped making the F200 and R200 cameras - their web site shows them perpetually "Out of Stock". Similar cameras have appeared in some laptops - and eventually, we'll start to see them showing up in more phones and tablets. But perhaps we've seen the end of the USB devices.

+ Read More
or
Or, use your email…
Use My Email Address
By continuing, you agree with the GoFundMe
terms and privacy policy
There's an issue with this Campaign Organizer's account. Our team has contacted them with the solution! Please ask them to sign in to GoFundMe and check their account. Return to Campaign

Are you ready for the next step?
Even a $5 donation can help!
Donate Now Not now

This campaign needs your support

Any amount helps!
Donate Now
Connect on Facebook to keep track of how many donations your share brings.
We will never post on Facebook without your permission.