OpenNI Cookbook Learn how to write NIUI-based applications and motion-controlled games Soroush Falahati BIRMINGHAM - MUMBAI OpenNI Cookbook Copyright © 2013 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing, and its dealers and distributors will be held liable for any damages caused or alleged to be caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. First published: July 2013 Production Reference: 1190713 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 978-1-84951-846-8 www.packtpub.com Cover Image by Ramin Gharouni ([email protected]) Credits Author Project Coordinator Soroush Falahati Leena Purkait Reviewers Proofreader Vinícius Godoy Stephen Copestake Li Yang Ku Liza Roumani Indexer Monica Ajmera Mehta Acquisition Editor Usha Iyer Graphics Ronak Dhruv Abhinash Sahu Lead Technical Editor Amey Varangaonkar Production Coordinator Shantanu Zagade Technical Editors Aparna Chand Athira Laji Cover Work Shantanu Zagade Dominic Pereira Copy Editors Insiya Morbiwala Aditya Nair Alfida Paiva Laxmi Subramanian About the Author Soroush Falahati is a Microsoft MCPD certificated C# developer of Web and Windows applications, now preparing for a new MCSD certification from Microsoft. He started programming at the age of 13 with VB5 and then continued to VB.Net, C#, C++, C for microcontrollers, as well as scripting languages such as PHP and JavaScript. He is currently the owner of an e-commerce company that uses web applications and smart phone apps as primary advantages over other competitors. As a hobby, Soroush supports robotic teams by voluntarily training them on how to program microcontrollers. I would like to thank my family, who supported me at the time of writing of this book with their patience, just as they always have been patient through the rest of my life! Also, I want to thank PrimeSense, which gave me access to confidential material and helped me through the writing of this book. I would like to especially thank Eddie Cohen, Software Team Leader, who answered my many questions and Jeremie Kletzkine, the Director of Business Development. About the Reviewers Vinícius Godoy is a computer graphics university professor at PUCPR. He is also an IT manager of an Electronic Content Management (ECM) company in Brazil, called Sinax. His former experience also includes building games and applications for Positivo Informática— including building an augmented reality educational game exposed at CEBIT—and network libraries for Siemens Enterprise Communications. In his research, he used Kinect, OpenNI, and OpenCV to recognize Brazilian sign language gestures. He is also a game development fan, having a popular website entirely dedicated to the field, called Ponto V (http://www.pontov.com.br). He is mainly proficient with the C++ and Java languages and his field of interest includes graphics synthesis, image processing, image recognition, design patterns, Internet, and multithreading applications. Li Yang Ku is a Computer Vision scientist and the main author of the Serious Computer Vision Blog (http://computervisionblog.wordpress.com), one of the foremost Computer Vision blogs. He is also the founder of EatPaper (http://www.eatpaper.org), a free web tool for organizing publications visually. He has worked as a researcher in HRL Laboratories, Malibu, California from 2011 to 2013. He did AI research on multiple humanoid robots and designed one of the vision systems for NASA's humanoid space robot, Robonaut 2, at NASA JSC, Houston. He also has broad experience on RGBD sensor applications, such as object recognition, object tracking, human activity classification, SLAM, and quadrotor navigation. Li Yang Ku received his MS degree in CS from University of California, Los Angeles, and has a BS degree in EE from National Chiao Tung University, Taiwan. He is now pursuing a Ph.D. degree at the University of Massachusetts, Amherst. Liza Roumani was born in Paris in 1989. After passing the French scientific Baccalaureate, she decided to move to Israel. After one year in Jerusalem University, she joined the Technion Institute of Technology of Haifa, where she obtained a BSC degree in Electrical Engineering. Liza Roumani is currently working at PrimeSense Company, the worldwide leader in 3D sensors technology. www.PacktPub.com Support files, eBooks, discount offers, and more You might want to visit www.PacktPub.com for support files and downloads related to your book. Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.PacktPub.com and, as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.PacktPub.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks. TM http://PacktLib.PacktPub.com Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book library. Here, you can access, read and search across Packt's entire library of books. Why Subscribe? f Fully searchable across every book published by Packt f Copy and paste, print and bookmark content f On demand and accessible via web browser Free Access for Packt account holders If you have an account with Packt at www.PacktPub.com, you can use this to access PacktLib today and view nine entirely free books. Simply use your login credentials for immediate access. Table of Contents Preface 1 Chapter 1: Getting Started 5 Introduction 5 Downloading and installing OpenNI 11 Downloading and installing NiTE 15 Downloading and installing the Microsoft Kinect SDK 18 Connecting Asus Xtion and PrimeSense sensors 21 Connecting Microsoft Kinect 24 Chapter 2: OpenNI and C++ 27 Introduction 27 Creating a project in Visual Studio 2010 28 OpenNI class and error handling 43 Enumerating a list of connected devices 47 Accessing video streams (depth/IR/RGB) and configuring them 53 Retrieving a list of supported video modes for depth stream 61 Selecting a specific device for accessing depth stream 68 Listening to the device connect and disconnect events 75 Opening an already recorded file (ONI file) instead of a device 81 Chapter 3: Using Low-level Data 85 Introduction 85 Configuring Visual Studio 2010 to use OpenGL 90 Initializing and preparing OpenGL 100 Reading and showing a frame from the image sensor (color/IR) 106 Reading and showing a frame from the depth sensor 124 Controlling the player when opening a device from file 150 Recording streams to file (ONI file) 161 Event-based reading of data 165