our blog

January 23, 2012

share this +

Augmented Reality Technology Brings Sesame Street Characters to Life

By Graydon Gordian

Having been around for over 40 years, here at Sesame Workshop we understand that there are some time-tested ways children learn and play – there’s no need to reinvent the playset. But we believe that, as new technologies emerge, there are ways to enhance and support the tried and true ways children use their imagination to make sense of the world.

That’s why we have partnered with Qualcomm to explore how augmented reality technology can encourage learning and emotional growth in young children. Our CEO Mel Ming, Innovation Lab team member David Glauber and Grover demonstrated the Vuforia augmented reality platform at the Consumer Electronics Show in Las Vegas on January 10th. By merely holding up a tablet to a traditional playset, children can interact with their favorite Sesame Street characters while developing socially, emotionally and cognitively.

Quite simply, when a child points a tablet or smart phone at these toys, the camera recognizes the objects and they come to life on screen. The camera on the tablet recognizes 3-D objects – in this instance specially designed versions of Bert and Ernie as well as a television, a bathtub, a racecar and other toys for Bert and Ernie to play with. (The characters are designed with special code-infused clothing so the camera can identify them. But developers at Sesame Street’s Innovation Lab are confident that, someday soon, the camera will be able to recognize any of the plush Sesame Street characters children own.) T-shirts, books and countless other items also have the potential to “come to life” when viewed through the app.

When Ernie is placed onto the playset, the camera recognizes the floor and triggers a response on screen, creating “walls” around him.  In the prototypical version of the technology presented at CES – the app is not yet available for purchase – Ernie says hello to the user and asks for another Sesame Street character to play with. But in future, more developed versions of the technology, Ernie and the other Sesame Street characters will have a wide range of reactions to any given scenario. This will allow for both a more guided form of pretend play, as well as child-directed experiences.


Both kinds of play – guided and child-directed – are important for fostering social confidence and a children’s ability to manage their own behavior and emotions. As children move toys in and out of the playset, they choose what kinds of social situations they would like to experiment with. Meanwhile, the app provides the structure necessary for them to learn more advanced forms of narrative construction, such as telling stories with a beginning, middle and end.

It was important to our Innovation Lab team that, in the words of team member Jason Milligan, the use of the augmented reality technology not be “gimmicky.” Milligan and the rest of the team wanted it to genuinely support and enhance the well-established ways children already play with their toys. So they reimagined the ways information can be input into a digital tool like a tablet.

For instance, when children play, they physically move their toys in and out of the playset. That’s why, instead of using a mouse or touch screen as an input device, the toys themselves are the input device. It’s also why all it takes to “activate” the toy’s digital rendering is to point the tablet at it. Directing the camera at the jukebox causes music to play; directing it at the TV turns it on, and causes whatever Sesame Street scene is playing to fill the tablet screen.

Because it comes in the form of an app, the technology is very malleable. As new characters are created and new storylines for them imagined, the software can be automatically updated like any other app. This is just the first generation of a new technology that has almost limitless possibilities.

The future is a fun place to play.

share this +

January 12, 2012

share this +

Sesame Workshop and Qualcomm Unveil Next Step in Children’s Education

By Beatrice Chow

Qualcomm Vuforia and Sesame Street

Qualcomm's Vuforia In Action with Sesame Street Characters

“Much of Sesame Workshop’s success can be attributed to our collaborative, research-intensive approach to the development of programs and activities. Qualcomm’s Vuforia platform offers a new dimension to mobile experiences. We think it can bring enhanced entertainment and educational benefits to children.”
– Terry Fitzpatrick, chief content and distribution officer, Sesame Workshop.

Sesame Workshop has been a longtime advocate of embracing cutting-edge technologies to enrich children’s early learning experiences. At the International CES (Consumer Electronics Show), Qualcomm and Sesame Workshop unveiled the result of their latest collaboration, a prototype playset that brings physical toys to life.

Using a tablet and a traditional playset, children engage with their toys to make playtime both fun and educational. The prototype playset includes elements such as common household objects, as well as figurines of classic Sesame Street characters Cookie Monster, Bert and Ernie. Children interact with this playset using a tablet and an application that features Qualcomm’s newly branded Vuforia augmented reality platform. When the tablet is pointed at the playset, the pieces and the play environment come alive through the tablet’s camera, transforming the playset into an interactive experience.

“In the past, the only place toys came to life was on TV and in movies. Today, we are bringing that magic one step closer to reality,” said Jay Wright, senior director of business development at Qualcomm. “With the ability to recognize 3D objects, Qualcomm’s Vuforia platform will transform the play experience. Our collaboration with the Sesame Workshop is helping us demonstrate the power of augmented reality to enrich children’s lives.”

Qualcomm’s award-winning Vuforia platform transforms real-world objects into interactive experiences for use in gaming, interactive media and instructional applications.  Sesame Workshop seized the opportunity to combine their educational research with cutting-edge technology to address children’s developmental needs. Augmented reality is another major step in proving that technology can be a useful tool in children’s education.

For more information, read the full press release here.

News Coverage:
A Game-Changer for Television? Sesame Street Will Be First Interactive Show

CES: Qualcomm Teams with Sesame Street, Microsoft
The Street

Educational Applications of Augmented Reality
Raman Media Network

CES Moment of Zen: A Fuzzy Friend

share this +