Jason Milligan is the Creative Director of Sesame Workshop’s Innovation Lab.

“Kids won’t know what that is!”

It wasn’t the first time I heard Sesame Workshop Curriculum Specialist Sue Scheiner say that, but this time it threw me a bit. We were reviewing Elmo’s World episodes to include in Season 2 of Kinect Sesame Street TV. Sue was referring to a camera. The camera was an old fashioned black box with a huge flashbulb attached. And one old fashioned camera in Mr. Noodle’s hands wouldn’t have mattered so much if any of the cameras in the piece looked and worked like current cameras do. But they didn’t. They were clunky film cameras and video cameras with tapes. There was a scene in which a kid takes film to a store to have it “developed.”

Not one person in the entire episode took a picture with a phone, or was able to immediately show Elmo his image on the back. The way today’s kids experience digital photography (often on smart phones) is completely, utterly, totally different than it was only a few years ago, apparently when this episode of Elmo’s World was made in 2005. Seriously. I checked the air date. It freaked me out a little. The same way it freaked me out when my niece pointed to a phone booth in a video and asked my sister what it was. Or when I explained to my kids how television used to show programs at certain times of day and you couldn’t pause or rewind or even decide which show you wanted to watch right now. Sue is right. Kids don’t know what those things are. Some Sesame content will always be relevant. Ernie will always be able to sing about the joys of bathing with his Duckie. C will always be for Cookie. But not this.

We couldn’t use Elmo’s World: Cameras. It was simply out of date.

The Sesame Workshop Blog

With that legacy of innovation in mind, we are proud to announce a new collaboration with Qualcomm centered on researching and developing new ways to educate children using mobile devices and applications. By bringing together Qualcomm’s cutting-edge mobile technologies and Sesame Workshop’s expertise in educating young children, Qualcomm and Sesame hope once again to revolutionize early childhood education.

The partnership was announced on Tuesday at the Consumer Electronics Show in Las Vegas, Nevada.

One of the first educational products created by this collaboration will be Big Bird’s Words, an app that will feature Qualcomm’s Vuforia augmented reality platform. The app will make teaching literacy an interactive experience by encouraging children to use a virtual “wordscope” to search for and find new words in their surrounding environment.

And American pre-schoolers won’t be the only children whose educational experience will be improved by this partnership. Qualcomm and Sesame Workshop will continue collaborating in China and India through Qualcomm’s Wireless Reach program. In India, Sesame Workhop’s Galli Galli Sim Sim, the Indian version of Sesame Street, is using 3G mobile technology to distribute radio content, videos and fun lessons about healthy living, hygiene, literacy and math. This program is critical to reaching children in rural areas where televisions are not commonly owned.

In China, a project launching later this year will provide tips, activities and other tools to help families with young children prepare for emergencies. These are just two examples of ways mobile devices can positively impact young children around the world.

With each passing day mobile devices and applications are reshaping the way we engage with one another and the rest of the globe. With the help of Sesame Workshop and Qualcomm, it will also reshape the way we educate our children.

The Sesame Workshop Blog

CA Technologies Chief Marketing Officer Andrew Wittman and Super Grover 2.0

Students in the United States are falling behind their peers globally in the subjects of science, technology, engineering and math, known as STEM. That is why Sesame Workshop and CA Technologies, a leading IT management solutions company, have partnered to encourage young children to focus on STEM learning even before they reach kindergarten.

The initiative will include a STEM hub on SesameStreet.org that will provide educational resources for preschool children. Resources will include original videos that teach STEM concepts, interactive games, hands-on lesson plans and activities. It’s all part of Sesame Workshop and CA Technologies joint effort to make math and science fun for young children.

“CA Technologies is proud to partner with Sesame Workshop and support the development of creative and interactive programs that engage children in STEM learning,” said CA Technologies Chief Marking Officer Andrew Wittman. “Through this initiative, we are hoping to help young people discover an interest in technology.”

While CA Technologies and Sesame Workshop’s partnership is new, both have focused on STEM education for some time. The organizations are participants in the 100Kin10 effort, a presidentially led effort to prepare 100,000 new STEM teachers over the next decade. CA Technologies and Sesame Worskhop are also members of the Clinton Global Initiative’s STEM education and Early Childhood Development working groups.

“We’re grateful to CA Technologies for supporting our work on STEM education and helping us reach more children with this valuable curriculum,” said Sherrie Westin, executive vice president and chief marketing officer for Sesame Workshop. “Through early STEM education we strive to help children develop strong critical thinking skills and a better understanding of how things work to help prepare them for future learning.”

To learn more about Sesame Workshop’s effort to support STEM education, click here.

The Sesame Workshop Blog

Sesame Workshop, whose mission is to help children reach their highest potential, is learning as much as we can about these media platforms so that we can use them to best meet children’s educational and developmental needs. We scour academic journals and policy-based reports; we consult experts in the field, and we also spend as much time as we can with children and parents observing and talking to them while they use touch screen devices.

Surprisingly, there are very few resources that are publically available to help guide developers who make educational apps for young children. Much like when Sesame Street was created in the 1960s and little was known at the time about how to best develop educational television, now too there seems to be little standardization for ensuring the best conditions under which children can learn from assets on these new touch screen devices. While understanding learning theories and how children process information through older media can lend some support in these endeavors, we quickly realized that these new technologies were raising additional questions about usability and navigation that could best be answered by experimentation.

Since jumping into the world of touch devices, we’ve learned many things to which we feel can benefit the industry and parents alike. That’s why, after conducting over 60 studies on the ways children interact with tablet devices, we have released our best practices, or “lessons learned.” This document is by no means “final.” Our practices are ever evolving as we learn from our research.

Just as researchers have documented the “formal features” of television, which are the rules, syntax, and indicators of audio and visuals cues designed to help children understand a television show, we are beginning to understand the formal features of touch screen devices. We now know, for example, that hotspots (triggers that take children to new locations within the app) in the lower right and left hand corners of the screen are precisely where children rest their wrists, which inevitably means that children accidentally exited the activity they were in. We have since recommended repositioning hotspots to the top of the screen so that children will not accidentally exit the focal activity. We know that children need both audio and visual clues to help support their play patterns. We have also learned that more complex gestures such as pinching are not as intuitive and easy for the developing dexterity of a preschooler.

As with everything we do, we want to ensure that children learn from their digital experiences. To improve comprehension and learning from games, we provide a three-tiered educational scaffold (three opportunities with increasing support so that children can advance in a game) when children do not know the answers to challenging questions. For book apps and ebooks, we have learned that making the story narration uninterruptable increases story comprehension. We know that the “bells and whistles” of interactivity can, at times, detract from learning. To mitigate distraction, after the page text is read, we then allow a child to interact with the hotspots. We also believe in utilizing a word-by-word highlighting as the text is read out loud in order to support early literacy skills.

In addition to developing best practices for preschoolers, we also take into consideration the parent and how to support co-play between children and adults. In the majority of our apps, we provide parent tips on how to extend the learning experience, and also ways to enrich the digital experience. For example, we have learned that parents want tips to be quick and easy to read. They also want the ability to customize and control the experience for their child by being able to turn the interactivity and audio on and off. Parents also like being able to record and narrate a storybook app or ebook in their own voice.

We hope that our best practices report will serve as a guide to those designing educational experiences for children through touch devices. To read the entire report, click here.

The Sesame Workshop Blog

J Milligan, the creative director of the Content Innovation Lab at Sesame Workshop, spoke at this year’s PSFK Conference in London. In his presentation J explains how Sesame Street Kinect TV, the Workshop’s newest interactive educational platform, is just the latest step in a decades long exploration of how technology can enhance early education.

The Sesame Workshop Blog

How would you update a classic? Take a treasured story from one era and spruce it up for a new century’s readers?

Sesame Workshop has produced over 1200 books in a variety of formats since the early 1970s. Part of the philosophy of our publishing group is the willingness to tell stories in whatever formats will attract and reach preschoolers. Animated book apps and e-books are the most recent formats in which Sesame Street characters have come to life. For the Workshop, an eagerness to create books in emerging digital formats is tempered by the need to balance innovation with our mission of education. It is a delicate balancing act, but one that the Workshop’s publishing group has pulled off time and time again.

In September, Sesame Workshop partnered with Callaway Digital Arts to produce an app version of the classic storybook The Great Cookie Thief. The story of The Great Cookie Thief originally aired as a Muppet segment on Sesame Street in 1971. Later the script became the basis for a storybook published in 1977 by Western Publishing. The story takes place in an Old West watering hole where residents talk about their collective problem: a thief who has been stealing the town’s cookies. Townfolk compare the attributes of a suspicious character (Cookie Monster) with an image on a wanted poster to determine if he is indeed the Great Cookie Thief.

In the process of transforming the show segment into a book, editors needed to capture the spirit of the story while meeting the curriculum goals of literacy and relational concepts (comparing objects).

Work on a book involves a lot of give and take. There are complex negotiations between illustrators, authors and editors to balance the creative vision for the book with what is appropriate and educational for young readers. When I pulled files from Sesame’s archive, illustrator Michael Smollin’s early sketches revealed notes from the book’s editors, asking him to rethink scenes with cowboys gambling, remove a cowboy’s pistol, and make sure only milk was served at the bar.

Illustrations also needed to remain consistent throughout. The fringe on a cowboy’s shirt, stitching on boots and the placement of the wanted posters needed to look the same from spread to spread.

The most notable request from the editors was for stronger female representation. When The Great Cookie Thief aired on Sesame Street, no female Muppets were written into the script – only cowboys and Cookie Monster. In a memo to Workshop editor Anna Jane Hayes, the team at Western Publishing was passionate about filling that gap: “We are very much concerned with avoiding sexism in our books and hope to include nothing that might seem to be demeaning to women. Therefore, we ask that you take special care in your presentation of the “dancing girls” here. … It is important also, that at least one more woman be included in a position of importance – as a cowgirl with a speaking part.”

The use of speech balloons was an effective way to use dialogue originally written for the show. Yet the placement of speech balloons affects how well young readers follow and understand the story. Editors repeatedly questioned: Do the balloons respond to each other and make sense? The positioning of the tail on a speech balloon could make all the difference in understanding who was talking. Use of overlapping balloons helped make it obvious when two sentences needed to be read in sequence.

When the Great Cookie Thief was adapted for an app, characters were re-drawn digitally, animated, and given voices. New technologies meant more interactivity between the reader and the story, so additional lines of dialogue were written to prompt the reader to participate, not just passively watch the action unfold. To create the app, editors at Sesame Workshop worked with programmers and designers at Callaway Digital Arts to update the book for the new digital format while still maintaining the educational themes of the book.

The interactivity of the app pulls young readers in by asking them to help the characters in the book. It’s the reader who makes comparisons between the wanted poster and Cookie Monster. To encourage this participation, it was important for children to relate to the characters acting as models for action in the story. Editors decided to make a child character already present in the book as gender-neutral as possible. In extra lines written for the app, gender-specific “he” and “him” terms were replaced with “the kid.” Puppeteer Fran Brill was asked to provide the voice, since she’s proven adept at performing both male and female characters on Sesame Street. She was able to create a voice that kids could interpret as either a girl or a boy – whichever helped them engage more with the story.

Callaway’s designers and programmers pored over 100 books from the Sesame Street library before choosing The Great Cookie Thief to adapt as their most recent project. The story won their vote because of its Cookie Monster star (a favorite character among the programmers) and the potential for interaction suggested by the illustrations and dialogue. There was also the hope that the story’s humor and the vintage look would appeal to adults, helping to promote a shared family experience.

The original plan was to adapt just the storybook into a simple story app. But as designers worked with new technologies they couldn’t resist adding a virtual photo booth where parents and kids can create their own wanted posters by adding props, googly eyes, and mustaches to self-portraits and Muppet characters. Editors and artists added wry humor to additional dialog and animations. When a cowboy hunts for his glasses to examine the wanted poster, he first pulls out an array of odd props from his jacket, including a sly homage to Steve Jobs. These additions help with the flow of the story and reward readers for lingering longer on a page.

For Senior Editor Betsy Loredo, working on book apps has required a tectonic shift in her thought process as an editor. In The Monster at the End of This Book, Grover desperately tries to keep the reader from turning the pages of his story so they will not reach the end of the book. When the story was developed into an animated app, it was difficult to imagine how to adapt it digitally when the whole concept revolved around physically turning pages. By accepting the idea that terms like “book”, “page” and “turn” were metamorphosing into something different for kids growing up reading books digitally, the possibility for what a book might become seemed limitless.

Understanding the evolving media children are using to read and having a presence in that media means that Sesame Workshop will continue to use books to extend Sesame Street’s curricular goals, no matter how kids turn the pages.

The Sesame Workshop Blog

Starting today “Kinect Sesame Street TV,” a groundbreaking 2-way television, Xbox-based experience that has plenty of educational potential, is now available for purchase.“It’s an opportunity to interact with our content,” said Sesame Workshop’s Vice President of Education and Research Rosemarie Truglio, PhD. “There’s a physical action to go with each concept. It’s truly interactive. That’s the beauty of this.”

Using a video camera that captures the image and movements of children as well as their parents and siblings (who often enjoy participating in the activities as much as the 2- to 4-year-olds Sesame Street’s educational content traditionally targets), “Kinect Sesame Street TV” intends to use a child’s physicality to help them learn fundamental concepts like spatial relations, enumeration and letter sounds.

For example, in one episode Grover spills coconuts all over the floor, and needs you to throw a specific number of them back to him. As the child makes a throwing motion with his arm – throwing, jumping, waving and clapping are the four basic actions children are asked to execute — the coconuts appear back on the screen. This interaction may offer the child an opportunity to more deeply engage with the concept than 1-way television traditionally has.

“We’re breaking the fourth wall,” said Sesame Workshop Senior Producer Todd Slepian. “Our characters can ask you something and you can respond.”

Over the course of the eight 40-plus minute episodes, children can appear on screen in an augmented reality version of Elmo’s World, help find hidden objects in a layered interactive street story, or be led by your furry monster friends through original interactive videos.

Here at Sesame Workshop we’ve been working on “Kinect Sesame Street TV” for a long time. We believe it has lots of potential, and we’re excited to be able finally to show the world.

The Sesame Workshop Blog

“It was very obvious that the things they were most interested in were various gadgets and computers,” she said. But she felt that there weren’t enough children’s books available digitally. Instead of waiting for them to come along, Hoh took the initiative and in 2010 founded MeeGenius, of which she now serves as CEO.

Two years later, MeeGenius and Sesame Workshop are happy to announce that we’ve formed a partnership that will bring six Sesame Street e-book titles to web, iOS and Android platforms. Beginning today with Celebrate School: First Day, the new titles will debut every Wednesday throughout the month of September.

“Think of the app as a bookshelf, bookreader and bookstore all rolled into one,” Hoh said. “The other thing is, it’s not just an app. We’re available on multiple platforms. The web. Google TV. Simply by downloading MeeGenius onto one of your platforms, you get all of those options.”

Each title features lively audio narration and text highlighting. An exclusive set of fun questions and activities at the end of each story extends the learning beyond the book and also helps build reading comprehension skills. Each e-book can be downloaded to the user’s MeeGenius library for $3.99-$4.99.

Hoh said that working with Sesame Workshop was a significant milestone for MeeGenius, but added that the partnership is also personally exciting, given how much her three girls love Sesame Street. “All six of these books are hardback books that we own on our bookshelf. I’ve read them to my kids hundreds of times.”

In addition to Celebrate School: First Day, the other books available on MeeGenius are Count to 10, Elmo and Abby’s Wacky Weather Day, Get Ready for School: Colors, It’s Check-up Time, and Elmo and The ABCs of Cookies.

As Hoh put it, Sesame Workshop is “really looking out for where kids are going next and making sure their beloved characters are there.” She’s absolutely right, and that is exactly why we team up with innovative, education-minded organizations like MeeGenius.

The Sesame Workshop Blog

The challenges are all spoofs of iconic video games: Pac-man, Guitar Hero and Super Mario Brothers, among others, inspired the levels Elmo and his friends must traverse. The animation was created by Magnetic Dreams, an animation company Sesame Street has been working with for almost a decade.“Being animators we’re all fans of games,” said Mike Halsey, president of Magnetic Dreams. “Getting to do an animation that was based on the games we played growing up – a combination of animation, games and Sesame Street – you can’t beat that.”

John Hamm, who has been working in animation for over 15 years, directed the animation. Hamm explained to me how Magnetic Dreams goes about creating the animation. The first step in the process is to design the characters.

“We didn’t want them to look exactly like the puppets themselves,” he said. “We wanted them to look like they were in video games.” In each level, Elmo, Abby and Telly have a different look. For instance, in the Pac-Man spoof, Elmo looks like an 8-bit, two dimensional figure.

From there they designed the environments, storyboarded the animation and blocked the characters movements. The setting of the 7 video games being spoofed — Super Mario Brothers, Angry Birds, Pac-Man, Just Dance, Guitar Hero, Mario Kart and World of Warcraft – formed the basis for their designs.

Most of the sections of the story were designed like all the other animation they work on, but in one instance – the Just Dance spoof – Magnetic Dreams used motion capture technology to design the characters and craft their dance moves. “Elmo’s Alphabet Challenge” is the first project on which Magnetic Dreams used its motion capture technology.

“The Just Dance game, from the animator’s standpoint, is very clearly a motion capture game,” said Halsey. “We weren’t sure whether we could get someone in a suit to pull of Elmo dancing, so we got a highly trained dancer. We got one of the animators in the suit to play the less graceful C-more. Their positions are exaggerated. Cartoons don’t move exactly like real people.”

You can get a copy of “Elmo’s Alphabet Challenge” and see all of the wonderful animation the Magnetic Dreams team created for Sesame Street at Sesame Street’s online store.

The Sesame Workshop Blog

Millions of families in India are cut off from information that can help children grow up healthy, happy, and ready to learn. To get around the barriers that marginalize these families, Sesame Workshop India is using phones to make educational media an integral part of the community. The poor and deeply conservative village of Nagina in the Mewat district if Haryana does not have electricity. Children here have never seen a radio or TV before, let alone a Bollywood movie.

Yet there is one media source that’s breaking through in Nagina. In the evenings, children and parents gather around a mobile phone to tune in Radio Mewat, a nearby community radio station. What are they hearing? Laughter mixed with learning, as characters from Galli Galli Sim Sim talk about literacy and math lessons, as well as good nutrition and healthy habits like always washing hands before you eat.

This audacious children’s health initiative, known as RadioPhone, is the first of its kind ever in communities like Nagina. The need for health lessons is critical here, where basic knowledge of nutrition and hygiene is scarce. Each year throughout India, approximately 1.9 million children under age five die from preventable diseases.

The pilot program began in 2011 with 31 episodes of Galli Galli Sim Sim broadcast by Gurgaon Ki Awaaz (GKA), a community station in Gurgaon, accessible via FM radio and on cell phones. So far Galli Galli Sim Sim RadioPhone broadcast in Gurgaon has reached over 200,000 listeners.

In separate subprojects funded by Qualcomm and HSBC, families and classrooms use 3G-equipped phones to stream Galli Galli Sim Sim radio shows on demand and access related videos and print materials that reinforce healthy messages. Community radio broadcasts are typically limited to a radius of 10 kilometers. But streaming content enables a far larger audience and consistent access for migrant families constantly on the move.

Given the great diversity of culture and language in India, RadioPhone relies on local relevance to break through. GKA in Gurgaon broadcasts in the language of its listeners and episodes include segments featuring Chamki, the beloved star of Galli Galli Sim Sim, interviewing neighbors in the local broadcast area.

By becoming part of the community experience, RadioPhone is giving families a voice. GKA reports an enthusiastic response from listeners, who call in to offer feedback and ask questions of health experts on air. More amazingly, listeners are using the program as a forum to share community concerns such as water access issues. Women are calling in to the show as well, a sign we’re making in-roads where other media cannot.

To understand where our efforts can have the biggest impact, we’re also bringing the program to life in 10 schools with “Healthy Habits” educational materials and curriculum guidance for teachers, with support from HSBC Bank.

Next for RadioPhone: nine more community stations, anticipated to reach over 1.5 million new listeners, and an expanded curriculum, with 60 new episodes on literacy and math now in the works. By combining local radio, mobile technology and engaging Sesame education, we’re delivering critical information to families whose health may depend on it.

Partners

Schwab Charitable Trust, Qualcomm, The Restoring Force (HSBC)

The Sesame Workshop Blog