Microsoft has made big changes to improve the accuracy and potential applications of Kinect. In a talk at GDC Next titled “The Next Generation of Kinect,” Microsoft senior program manager Ben Lower detailed a few of the new features that Microsoft’s new Kinect sensor is capable of on both Xbox One and Kinect for Windows.
“I hope people are inspired by the future of Kinect and the direction that we’re going in as a company, the investments that we’re making and how that’s unlocking new scenarios for games, but also new scenarious outside of gaming. Outside of the living room,” Lower said.
Lower explained that the new Kinect camera has significantly higher fidelity than Xbox 360’s, now able to detect a room of six people, versus the original which could only detect one or two. He explained that this opens up “amazing opportunities for building richer and more engaging experiences,” plus explained that Microsoft sees Kinect’s Natural User Interaction as “a paradigm shift that allows us to go beyond only manual input to communicating with our technology in a more natural way.”
Throughout the session, Lower described potential scenarios for Kinect and also showed live demos of how developers can take advantage of new features. He explained that the new Kinect now has full skeletal tracking, which allows for far more precise detection of movement, and can also approximate where a user’s body ends even if it’s partially obscured or turned away from the camera. Like on Xbox One, the newest version of Kinect for Windows allows for full color, 1080p images to be captured, which allows for a “new level of realism.”
At one point, Lower explained that Kinect can now so accurately detect the user that it can completely strip away the background, which he called “green screen without a screen.” He showed off a homemade application that allowed him to “photobomb your friends on Facebook” and demonstrated how in just a few seconds he could choose a friend’s Facebook photo, stand the appropriate distance from Kinect to appear as though he was standing next to someone in the picture, capture a still, then post a version in which he appeared in the shot on his friend’s wall.
He also discussed other potential applications for Kinect, showing an image of a sensor being used in a retail store to try on clothing, and described*Playoke Dance, a “virtual dance experience” in European health clubs that uses up to 36 Kinects mounted on a ceiling to track users in dance classes in real time, analyze how correct their movements are and compare them to others.
Lower captured a 3D image of himself and showed how he could do a 360-degree fly-around of the full model, plus demonstrated how Kinect’s microphone can strip out background noise, how it tracks specific hand movements and gestures, and even how it can detect how much force a user is exerting for things like yoga poses or throwing punches in a fighting game. He also showed how the tech can recognize facial expressions including whether he was happy or neutral, plus whether his eyes were open, whether he was wearing glasses, and whether he was looking at the camera.
Developers can now also build custom gestures that Kinect can read, so specific movements can be fine-tuned for new kinds of input. Gesture tracking has also been improved to detect “full joint orientation and rotation,” so even if players are leaning at strange angles or away from the lens, Kinect won’t lose track of them.
The new Kinect for Windows will fully launch next year, and Lower said the team is hoping that Kinect will be able to interact with Windows apps in the future. While Xbox One only has one Kinect port, Kinect for Windows will support multiple Kinects, with up to four Kinect sensors working on one PC “out of the box” (as long as each has a separate USB controller) and the potential for significantly more, such as in the Playoke example above.
During a Q&A session, Lower also said more accurate face tracking is possible, though generating an HD face could take significant processing resources that might make it hard to use in the middle of a graphically-intensive game. When asked, Lower also hinted that Kinect “might” be able to read lips, but couldn’t confirm for sure.
Microsoft recently released version 1.8 of the Kinect for Windows Software Development Kit and will begin shipping new sensors to select developers this month.
Andrew Goldfarb is IGN’s news editor. Keep up with pictures of the latest food he’s been eating by following @garfep on Twitter or garfep on IGN.
More...
“I hope people are inspired by the future of Kinect and the direction that we’re going in as a company, the investments that we’re making and how that’s unlocking new scenarios for games, but also new scenarious outside of gaming. Outside of the living room,” Lower said.
Lower explained that the new Kinect camera has significantly higher fidelity than Xbox 360’s, now able to detect a room of six people, versus the original which could only detect one or two. He explained that this opens up “amazing opportunities for building richer and more engaging experiences,” plus explained that Microsoft sees Kinect’s Natural User Interaction as “a paradigm shift that allows us to go beyond only manual input to communicating with our technology in a more natural way.”
Throughout the session, Lower described potential scenarios for Kinect and also showed live demos of how developers can take advantage of new features. He explained that the new Kinect now has full skeletal tracking, which allows for far more precise detection of movement, and can also approximate where a user’s body ends even if it’s partially obscured or turned away from the camera. Like on Xbox One, the newest version of Kinect for Windows allows for full color, 1080p images to be captured, which allows for a “new level of realism.”
At one point, Lower explained that Kinect can now so accurately detect the user that it can completely strip away the background, which he called “green screen without a screen.” He showed off a homemade application that allowed him to “photobomb your friends on Facebook” and demonstrated how in just a few seconds he could choose a friend’s Facebook photo, stand the appropriate distance from Kinect to appear as though he was standing next to someone in the picture, capture a still, then post a version in which he appeared in the shot on his friend’s wall.
He also discussed other potential applications for Kinect, showing an image of a sensor being used in a retail store to try on clothing, and described*Playoke Dance, a “virtual dance experience” in European health clubs that uses up to 36 Kinects mounted on a ceiling to track users in dance classes in real time, analyze how correct their movements are and compare them to others.
Lower captured a 3D image of himself and showed how he could do a 360-degree fly-around of the full model, plus demonstrated how Kinect’s microphone can strip out background noise, how it tracks specific hand movements and gestures, and even how it can detect how much force a user is exerting for things like yoga poses or throwing punches in a fighting game. He also showed how the tech can recognize facial expressions including whether he was happy or neutral, plus whether his eyes were open, whether he was wearing glasses, and whether he was looking at the camera.
Developers can now also build custom gestures that Kinect can read, so specific movements can be fine-tuned for new kinds of input. Gesture tracking has also been improved to detect “full joint orientation and rotation,” so even if players are leaning at strange angles or away from the lens, Kinect won’t lose track of them.
The new Kinect for Windows will fully launch next year, and Lower said the team is hoping that Kinect will be able to interact with Windows apps in the future. While Xbox One only has one Kinect port, Kinect for Windows will support multiple Kinects, with up to four Kinect sensors working on one PC “out of the box” (as long as each has a separate USB controller) and the potential for significantly more, such as in the Playoke example above.
During a Q&A session, Lower also said more accurate face tracking is possible, though generating an HD face could take significant processing resources that might make it hard to use in the middle of a graphically-intensive game. When asked, Lower also hinted that Kinect “might” be able to read lips, but couldn’t confirm for sure.
Microsoft recently released version 1.8 of the Kinect for Windows Software Development Kit and will begin shipping new sensors to select developers this month.
Andrew Goldfarb is IGN’s news editor. Keep up with pictures of the latest food he’s been eating by following @garfep on Twitter or garfep on IGN.
More...
