App – Reel Interaction Final Presentation
Course: Fall 2012 – CGT 411: Contemporary Problems in Applied Computer Graphics (Senior Design)
Role: Lead Developer/Architecture Designer in team of five
Introduction:
Reel Interaction is a CGT 411/450 capstone group at Purdue University researching Augmented Reality in film production. For film production companies who need to visualize an interaction between actors and graphics, Reel Interaction is a product that allows users the ability to visualize graphical interaction in real-time. Unlike current methods that rely on physical placeholders, our product utilizes virtual placeholders which more accurately represent the final vision.
We built a Augmented Reality Unity 3D stage which connected to four Microsoft Kinect sensor servers under Open Sound Control protocol. I act as lead developer/programmer in the team.
Early Demo:
Video (Please jump to 25:50 for demo):
Sample OSC Server Code:
bool somethingTracked = false; foreach (Joint joint in data.Joints) { if (joint.TrackingState == JointTrackingState.Tracked) somethingTracked = true; // prepare the message to send OscElement message = new OscElement("/tracker/" + data.TrackingId + "/" + joint.JointType.ToString(), joint.Position.X, joint.Position.Y, joint.Position.Z, joint.TrackingState.ToString()); // add joint message to bundle bundle.AddElement(message); } if (data.TrackingState == SkeletonTrackingState.Tracked) { somethingTracked = true; foreach (BoneOrientation orientation in data.BoneOrientations) { OscElement message = new OscElement("/orientation/" + data.TrackingId + "/" + orientation.StartJoint.ToString(), orientation.AbsoluteRotation.Quaternion.X, orientation.AbsoluteRotation.Quaternion.Y, orientation.AbsoluteRotation.Quaternion.Z, orientation.AbsoluteRotation.Quaternion.W); // add orientation message to bundle bundle.AddElement(message); } } if (somethingTracked) { if (OSCSender.logBlock != null) OSCSender.logBlock.Text = "Sending data of skeleton #" + data.TrackingId; //send bundle bundle.DateTime = DateTime.Now; OSCSender.oscSender.Send(bundle); }