Many writers and academics swear by the old methods. They prefer pen and paper and physical books over tablets or screens. Even digital artists and designers often draft offline, on paper or with raw materials where they can be more expressive and flexible.
François V. Guimbretière, Information Science, says it doesn’t have to be this way—one method or the other, old versus new. “In our work, we help you focus on the work or the design but also benefit from all the affordances of the digital world,” Guimbretière says.
From interconnected tablets to interactive 3D printing, Guimbretière designs technologies that combine digital functionality with more organic or expressive methods. That means he works on both hardware and software, with the goal of having his technologies recede into the background, so the user can focus on the task at hand and not the technology itself.
Guimbretière’s adviser at Stanford likened the ideal use of technology to driving a stick shift—there’s effort in learning the method, but once learned, it becomes unconscious. “It becomes so well integrated into your notion of driving that it almost never reaches the surface of consciousness,” he says. “That’s the kind of interface I’m always aiming at. For me the magic is when the technology can completely disappear.”
Digital Paper—Better Than Paper?
From early on, one strand of Guimbretière’s research has been trying to make an interface for digital paper that is as simple as real paper. In a project called United Slates, Guimbretière and his team, led by then-student Nicholas Chen, designed a lightweight slate you could mark with a pen (this was just before the iPad). A key innovation allowed users to easily transfer material from one slate to another, so users could employ multiple slates at once.
When Guimbretière and his team tested the system with a group of graduate students, the students used the slates as they would paper. They opened an article on one slate and the footnotes on another; they copied text from one slate and pasted into another. They set slates beside their computers, and even stacked and shuffled the slates.
“There was a book, The Myth of the Paperless Office [MIT Press, 2002], that pointed out that people like paper very much because they can have as many displays as they want,” Guimbretière says. “If I provide a system with many displays, you should really see people reacting positively to it, and it’s true.”
“That’s the kind of interface I’m always aiming at. For me the magic is when the technology can completely disappear.”
Along this research track, Guimbretière’s team also developed a software program, called RichReview, that added digital functionality to reviewing and annotating papers. Led by PhD student Dongwook Yoon (University of British Columbia, Vancouver), RichReview allows users to tear the text at any point, changing the text layout and creating more space to comment. They can also record audio comments as they navigate through the document, hovering over areas of text with a highlighter as they speak.
All of this is captured by the system, so a student or peer can play the comments back and see the gestures of the reviewers as they comment. Self-conscious reviewers can even edit a transcript of their audio and have the correct version play back. “Faculty loved it because it made it easy to create high-level and structural comments,” Guimbretière says. “Students loved it because it was like office hours but better. If they didn’t understand something, they could just replay it.”
Integrating 3D Printing and Design
PhD student Huaishu Peng led Guimbretière into a more recent line of research—how 3D printing is integrated with design.
Their inspiration for a series of projects came from The Reflective Practitioner (Basic Books, 1983), a study of how architects work. “The way architects discuss things is they sketch,” Guimbretière says. “They assess the sketch and modify their design based on feedback. I’ve discussed this with people at places like Pixar, and it’s the same for animation. When they are in the design phase, they don’t try to polish what they have.”
No parallel existed for the actual process of 3D printing. The design had to be completed and perfected, and then the printer would make the product. Guimbretière wanted to break that barrier between designer and printer.
His team’s first attempt, led by Peng, was a handheld 3D printer, a wax extruder, that can help a user make an object with layered coils of wax. An actuator in the handheld printer helps to guide the user’s hand with each layer, giving the process precision but more tangibility. Users can also change the shape as they make the object, in addition to adding free-hand elements. “It’s extremely expressive and easy,” Guimbretière says.
The only catch is that the process is slow. In the next project, Guimbretière wanted something faster, more like a sketch. Collaborating with Berlin’s Hasso Plattner Institute, the solution he and his team produced is a fast, wire printing method, where only the outlines or frame of an object are printed. This can be done on existing printer models, and a design that would have taken two hours to complete can be prototyped in 15 minutes.
Building on this method is a project called On-the-Fly Print, where Peng and Guimbretière incorporated a rotating platform and special programming that allows the printer to work incrementally. As soon as one of the parts—the wing of an airplane, for example—is created in the CAD system, the printer can begin printing it. “So you can, within minutes, have a rough cut of what you’ve asked for. Then you can remove it from the printer, verify that it fits and continue,” Guimbretière says.
For the most recent and advanced project in this vein, Guimbretière’s team used a specially programming robotic arm that carries out the printing. An augmented reality headset allows the user to see and manipulate, with a controller, an image of his or her design on a rotating platform. Once the design is set, the arm moves over the platform to begin, while the user continues to work on another element of the design.
“So the printing space and the design space overlap,” Guimbretière says. “Within a minute, you get a physical representation of your design. You can put other things on it and get a sense of how it works and use it to inform your next steps. If you don’t like the outcome, you can build another one in five minutes.”
The robotic arm is programmed to be in conversation with the user. If the user touches the platform, the printer moves away. If the user leaves the area, the printer will finish the job. “There’s very little visible interface,” Guimbretière says. “We had to work very hard to make this happen—you have to do a lot of code, and the computer has to do a lot of work in order for the user to have a seamless experience, but it worked very well.”
Guimbretière is working with collaborators and Cornell’s Center for Technology Licensing to patent the WirePrint and On-the-Fly Print methods and the robotic arm setup.
Robotic Hands-on Teaching
Looking ahead, Guimbretière is thinking about how this collaborative interaction between people and machines might apply to remote teaching scenarios. Could a professor in New York City teach students in Ithaca remotely, with a robotic arm as a physical extension, pointing things out or putting things on the board? Another idea is having a completely virtual space, where, through augmented reality headsets, the teacher and students share a space. “I see what they see, and they see me, and the system simultaneously can merge us together,” Guimbretière says.
He is currently weighing the pros and cons of these and other setups. “There’s a lot of different visions, but the idea is to help, say, a high school in a place that’s remote, where they don’t have the expert to teach something hands-on like building electronic circuits. We imagine that you send them a robotic system, and we can help them physically without having to be there. If the interaction is well done, we can have a very fruitful way to collaborate.”
This fits in Guimbretière’s overall aims—to use technology to solve problems. “At 10 years old, I knew I wanted to build things to help others,” he says. “I’m very lucky now because I can do that and demonstrate things people thought could not be done.”