App – Reel Interaction Final Presentation

Course: Fall 2012 – CGT 411: Contemporary Problems in Applied Computer Graphics (Senior Design)

Role: Lead Developer/Architecture Designer in team of five

Introduction:

Reel Interaction is a CGT 411/450 capstone group at Purdue University researching Augmented Reality in film production. For film production companies who need to visualize an interaction between actors and graphics, Reel Interaction is a product that allows users the ability to visualize graphical interaction in real-time. Unlike current methods that rely on physical placeholders, our product utilizes virtual placeholders which more accurately represent the final vision.

We built a Augmented Reality Unity 3D stage which connected to four Microsoft Kinect sensor servers under Open Sound Control protocol. I act as lead developer/programmer in the team.

Early Demo:

Video (Please jump to 25:50 for demo):

Sample OSC Server Code:

                            bool somethingTracked = false;
                            foreach (Joint joint in data.Joints)
                            {
                                if (joint.TrackingState == JointTrackingState.Tracked)
                                    somethingTracked = true;
                                // prepare the message to send
                                OscElement message = new OscElement("/tracker/" + data.TrackingId + "/" + joint.JointType.ToString(), joint.Position.X, joint.Position.Y, joint.Position.Z, joint.TrackingState.ToString());

                                // add joint message to bundle
                                bundle.AddElement(message);
                            }
                            if (data.TrackingState == SkeletonTrackingState.Tracked)
                            {
                                somethingTracked = true;
                                foreach (BoneOrientation orientation in data.BoneOrientations)
                                {
                                    OscElement message = new OscElement("/orientation/" + data.TrackingId + "/" + orientation.StartJoint.ToString(), orientation.AbsoluteRotation.Quaternion.X, orientation.AbsoluteRotation.Quaternion.Y, orientation.AbsoluteRotation.Quaternion.Z, orientation.AbsoluteRotation.Quaternion.W);
                                    // add orientation message to bundle
                                    bundle.AddElement(message);
                                }
                            }
                            if (somethingTracked)
                            {
                                if (OSCSender.logBlock != null)
                                    OSCSender.logBlock.Text = "Sending data of skeleton #" + data.TrackingId;
                                //send bundle
                                bundle.DateTime = DateTime.Now;
                                OSCSender.oscSender.Send(bundle);
                            }

Leave a comment