Having been around for over 40 years, here at Sesame Workshop we understand that there are some time-tested ways children learn and play – there’s no need to reinvent the playset. But we believe that, as new technologies emerge, there are ways to enhance and support the tried and true ways children use their imagination to make sense of the world.
That’s why we have partnered with Qualcomm to explore how augmented reality technology can encourage learning and emotional growth in young children. Our CEO Mel Ming, Innovation Lab team member David Glauber and Grover demonstrated the Vuforia™ augmented reality platform at the Consumer Electronics Show in Las Vegas on January 10th. By merely holding up a tablet to a traditional playset, children can interact with their favorite Sesame Street characters while developing socially, emotionally and cognitively.
Quite simply, when a child points a tablet or smart phone at these toys, the camera recognizes the objects and they come to life on screen. The camera on the tablet recognizes 3-D objects – in this instance specially designed versions of Bert and Ernie as well as a television, a bathtub, a racecar and other toys for Bert and Ernie to play with. (The characters are designed with special code-infused clothing so the camera can identify them. But developers at Sesame Street’s Innovation Lab are confident that, someday soon, the camera will be able to recognize any of the plush Sesame Street characters children own.) T-shirts, books and countless other items also have the potential to “come to life” when viewed through the app.
When Ernie is placed onto the playset, the camera recognizes the floor and triggers a response on screen, creating “walls” around him. In the prototypical version of the technology presented at CES – the app is not yet available for purchase – Ernie says hello to the user and asks for another Sesame Street character to play with. But in future, more developed versions of the technology, Ernie and the other Sesame Street characters will have a wide range of reactions to any given scenario. This will allow for both a more guided form of pretend play, as well as child-directed experiences.
Both kinds of play – guided and child-directed – are important for fostering social confidence and a children’s ability to manage their own behavior and emotions. As children move toys in and out of the playset, they choose what kinds of social situations they would like to experiment with. Meanwhile, the app provides the structure necessary for them to learn more advanced forms of narrative construction, such as telling stories with a beginning, middle and end.
It was important to our Innovation Lab team that, in the words of team member Jason Milligan, the use of the augmented reality technology not be “gimmicky.” Milligan and the rest of the team wanted it to genuinely support and enhance the well-established ways children already play with their toys. So they reimagined the ways information can be input into a digital tool like a tablet.
For instance, when children play, they physically move their toys in and out of the playset. That’s why, instead of using a mouse or touch screen as an input device, the toys themselves are the input device. It’s also why all it takes to “activate” the toy’s digital rendering is to point the tablet at it. Directing the camera at the jukebox causes music to play; directing it at the TV turns it on, and causes whatever Sesame Street scene is playing to fill the tablet screen.
Because it comes in the form of an app, the technology is very malleable. As new characters are created and new storylines for them imagined, the software can be automatically updated like any other app. This is just the first generation of a new technology that has almost limitless possibilities.
The future is a fun place to play.
