FERG

ACCV 2016 Panels

Modeling Stylized Character Expressions via Deep Learning

We propose DeepExpr, a novel expression transfer approach from humans to multiple stylized characters. We first train two Convolutional Neural Networks to recognize the expression of humans and stylized characters independently. Then we utilize a transfer learning technique to learn the mapping from humans to characters to create a shared embedding feature space. This embedding also allows human expression-based image retrieval and character expression-based image retrieval. We use our perceptual model to retrieve character expressions corresponding to humans. We evaluate our method on a set of retrieval tasks on our collected stylized character dataset of expressions. We also show that the ranking order predicted by the proposed features is highly correlated with the ranking order provided by a facial expression expert and Mechanical Turk experiments.

SIGGRAPH Asia 2017 Education Panels

Before and After AR/VR: Empowering Paradigm Shifts in Education

The development and potential of Virtual Reality (VR) and Augmented Reality (AR) technologies have already begun to transform classrooms and teaching in ways unimaginable just ten years ago. The increasing integration of these tools and experiences into educational environments has ushered in the possibility of profound changes in the way we think, learn and communicate. Applications for and in education are at the forefront of these changes. AR/VR can enhance the way teachers teach and students learn on all levels from primary school to post graduate education and in all content areas. This new way of experiencing and understanding the world can bring about great opportunities to improve teaching environments and support teachers in their mission to improve the skills and experiences of their students. AR and VR devices now available have made these experiences more affordable, interfaces for uses in education have improved enormously, and people from countries far and wide are able to contribute and connect in ways never available before. When teachers are able to design content that is delivered using AR or VR environments, and students can explore knowledge in a completely different context, opportunities emerge that allow for unique and exciting learning experiments. This panel brings together international experts in industry and education who are making significant contributions to education using these technologies. The panelists will present their newest and ongoing education initiatives, creative and innovative projects, and their plans and predictions for the future. They will discuss the broader ramifications of the dissemination of these new tools.

Girls in STEM: investigating the reasons that too few female students enter technical fields

The purpose of this panel is to provide a forum for educators to present their concerns about the uneven representation of female students in the technical fields of Science, Technology, Engineering and Mathematics (STEM). The goal of the panel is to further the discussion that began at the Symposium on Education in SIGGRAPH Asia in 2016. The number of female students, worldwide that choose to enroll in technical fields is considerably smaller than the number of male students. In this panel we present potential explanations for this phenomenon and propose ways to improve the situation in the future. STEM graduates become future professionals in all areas of Computer Graphics and related technical fields including Computer Vision, Machine Learning and others. Increasing the number of female students pursuing degrees with a focus on STEM areas will also increase the numbers of female professionals pursuing jobs in Computer Graphics.

WiCV IEEE 2017 Panels

Learning Stylized Character Expressions from Humans

We present DeepExpr, a novel expression transfer system from humans to multiple stylized characters via deeplearning. We developed : 1) a data-driven perceptual modelof facial expressions, 2) a novel stylized character data setwith cardinal expression annotations : FERG (Facial Expression Research Group) - DB, and 3) a mechanism to accurately retrieve plausible character expressions from human expression queries. We evaluated our method on a setof retrieval tasks on our collected stylized character datasetof expressions. We have also shown that the ranking orderpredicted by the proposed features is highly correlated withthe ranking order provided by a facial expression expert andMechanical Turk (MT) experiments.

WACV 2018 IEEE Panels

Learning to Generate 3D Stylized Character Expressions from Humans

We present ExprGen,a system to automatically generate 3D stylized character expressions from humans in a perceptually valid and geometrically consistent manner. Our multi-stage deep learning system utilizes the latent variables of human and character expression recognition convolutional neural networks to control a 3D animated character rig. This end-to-end system takes images of human faces and generates the character rig parameters that best match the human’s facial expression. ExprGen generalizes to multiple characters, and allows expression transfer between characters in a semi-supervised manner. Qualitative and quantitative evaluation of our method based on Mechanical Turk tests show the high perceptual accuracy of our expression transfer results.