App – Reel Interaction Final Presentation
Course: Fall 2012 – CGT 411: Contemporary Problems in Applied Computer Graphics (Senior Design)
Role: Lead Developer/Architecture Designer in team of five
Introduction:
Reel Interaction is a CGT 411/450 capstone group at Purdue University researching Augmented Reality in film production. For film production companies who need to visualize an interaction between actors and graphics, Reel Interaction is a product that allows users the ability to visualize graphical interaction in real-time. Unlike current methods that rely on physical placeholders, our product utilizes virtual placeholders which more accurately represent the final vision.
We built a Augmented Reality Unity 3D stage which connected to four Microsoft Kinect sensor servers under Open Sound Control protocol. I act as lead developer/programmer in the team.
Early Demo:
Video (Please jump to 25:50 for demo):
Sample OSC Server Code:
bool somethingTracked = false;
foreach (Joint joint in data.Joints)
{
if (joint.TrackingState == JointTrackingState.Tracked)
somethingTracked = true;
// prepare the message to send
OscElement message = new OscElement("/tracker/" + data.TrackingId + "/" + joint.JointType.ToString(), joint.Position.X, joint.Position.Y, joint.Position.Z, joint.TrackingState.ToString());
// add joint message to bundle
bundle.AddElement(message);
}
if (data.TrackingState == SkeletonTrackingState.Tracked)
{
somethingTracked = true;
foreach (BoneOrientation orientation in data.BoneOrientations)
{
OscElement message = new OscElement("/orientation/" + data.TrackingId + "/" + orientation.StartJoint.ToString(), orientation.AbsoluteRotation.Quaternion.X, orientation.AbsoluteRotation.Quaternion.Y, orientation.AbsoluteRotation.Quaternion.Z, orientation.AbsoluteRotation.Quaternion.W);
// add orientation message to bundle
bundle.AddElement(message);
}
}
if (somethingTracked)
{
if (OSCSender.logBlock != null)
OSCSender.logBlock.Text = "Sending data of skeleton #" + data.TrackingId;
//send bundle
bundle.DateTime = DateTime.Now;
OSCSender.oscSender.Send(bundle);
}