Events

We are creative storytellers, tool-builders, and social beings, and since at least the first cave-paintings, people have invented new techniques to build shared understandings and communicate ideas using combinations of visuals and language. Meanwhile, human-computer interaction (HCI) challenges us to design interfaces that prioritize human experiences over just the development of technology. Ideally, interfaces should extend human creativity, cognition, and social interaction, rather than intrude on them. 

Can future intelligent systems increase our range of human expression in spontaneous creative processes, in harmony with our natural interactions, without taking away our control, and while respecting our creative agency? Interface design could take inspiration from our natural drawing, talking, gestural, and social interactions. Systems could help us think, learn, design, play, communicate, and tell stories in increasingly expressive ways together, with the support of computational capability such as explorable interactive graphics and simulations.

Towards this, I'll introduce an interaction concept inspired by storytelling and teaching at a whiteboard, called "DrawTalking," which I prototyped as a multi-touch tablet interface. The user sketches and narrates to explain concepts or tell stories at a whiteboard, but through their interactions, they also simultaneously direct interactive scenes with their own drawings and logic under their creative control.

DrawTalking is a starting-point for exploring how to augment longer-form creative processes in more complex everyday contexts. For example: giving animators, programmers, and storytellers the ability to program and iterate on stories and logic collaboratively just by talking and gesturing in 3D space, without needing to know how to program. Students and teachers across disciplines, too, could learn by creating and experiencing interactive visualizations they can directly manipulate. Natural interactions and intelligent systems open-up these new opportunities for future communication and learning, as technology takes a greater role in our social context.

 

Bio: Karl Rosenberg is a recent computer science Ph.D. graduate from New York University, where he focused on human-computer interaction (HCI) under Ken Perlin at the Future Reality Lab. Karl is especially interested in interactions and systems design for creativity and thought. His research explores how technology can extend natural human capabilities for creativity and communication in ways that are unobtrusive, controllable, and natural in everyday contexts such as teaching, telling stories, sketching, or collaborating with others. He's especially excited to find ways to support people's creative processes and social interactions. Karl designs and prototypes using multimodal user interactions (such as drawing and talking) and emerging technologies such as augmented reality and intelligent sketching systems. He appreciates cross-disciplinary collaborations and perspectives.

          Karl has collaborated with industry research, e.g. Adobe Research, as well as the Solomon R. Guggenheim Time-Based Media Lab in NYC. Most recently, he's been practicing teaching as a visiting professor at NYU. Outside academia, as a local NYC resident he deeply appreciates animated films and jazz music, and he also records his own demos.