12th CODH Seminar (Online) - AI for Culture: From Japanese Art to Anime

Theme

We welcome two distinguished guests, Yingtao Tian from Google Brain Tokyo, and Yanghua Jin from Preferred Networks, to discuss relationship between AI and culture, from Japanese art to anime, including topics such as KaoKore dataset, generative models and machine creativity.

The hashtag is '#codh12'.

Basic Information

Date	August 5 (Wed), 2020, 18:00-20:00 JST
Venue	Online
Fee	Free
Language	English No translation to Japanese

Program

Please note that all time is in JST (Japan Standard Time) = UTC+9. You can look up your local time using Google or other services.

18:00-18:05	Opening	Tarin Clanuwat (ROIS-DS Center for Open Data in the Humanities / National Institute of Informatics)
18:05-18:15	"Collection of Facial Expressions" and "KaoKore Dataset": Data-driven art history research and possibility of collaboration with machine learning	Chikahiko Suzuki (ROIS-DS Center for Open Data in the Humanities / National Institute of Informatics)
18:15-18:30	Face Detection on Pre-modern Japanese Artworks for Semi-Automatic Annotation	Alexis Mermet (Ecole Polytechnique Fédérale de Lausanne (EPFL) / National Institute of Informatics)
18:30-19:00	KaoKore: A Pre-modern Japanese Art Facial Expression Dataset	Yingtao Tian (Google Brain Tokyo)
19:00-19:30	Crypko - a new workflow for anime character creation	Yanghua Jin (Preferred Networks)
19:30-20:00	Question and Discussion	Moderator: Tarin Clanuwat (ROIS-DS Center for Open Data in the Humanities / National Institute of Informatics)

Registration

Thank you for your participation. The total view was 230, and the maximum simultaneous viewers was 84.

Abstract and Bio

"Collection of Facial Expressions" and "KaoKore Dataset": Data-driven art history research and possibility of collaboration with machine learning

In my talk, I will explain “Collection of Facial Expressions (KaoKore)” and "KaoKore Dataset". KaoKore is a CODH project which collects images of faces from Japanese art works. This project aims to promote data-driven research in humanities, especially art history research field. I will introduce process of creating KaoKore and example of art history research using KaoKore. In the second half of my talk, I will explain “Kaoreko dataset” briefly. This dataset is rework of KaoKore in a machine learning friendly format.　I would like to talk about expectations of collaboration between machine learning and art history research based on KaoKore dataset.

Chikahiko Suzuki (ROIS-DS Center for Open Data in the Humanities / National Institute of Informatics)

A Project Assistant Professor at the ROIS-DS Center for Open Data in the Humanities and National Institute of Informatics. After studying Art History, Cultural Resources Studies and Digital Humanities, his main research interest is in applying informatics and open data to humanities research fields. Currently, he is focusing on IIIF (international image interoperability framework).

Presentation Slides: DOI:10.20676/00000380

Link: Collection of Facial Expressions (KaoKore)

Face Detection on Pre-modern Japanese Artworks for Semi-Automatic Annotation

During this talk, we will discuss our automated method for face detection on Pre-modern Japanese artworks. This project's goal is to allow art historians to use automated detectors; accelerating the process of creating new facial expressions collections as KaoKore. First we will quickly present our method for detection and how we use the KaoKore collection to train detectors. Then, in the second half of our talk, we will discuss our experiments on the KaoKore collection and present our detection results on a non-annotated collection. Finally, we will quickly present what are the next steps for our research.

Alexis Mermet (Ecole Polytechnique Fédérale de Lausanne (EPFL) / National Institute of Informatics)

A Master student at Ecole Polytechnique Fédérale de Lausanne and an intern at NII for the last 5 months. Studied first Communication Systems and followed with a Master in Data science.

Presentation Slides: DOI:10.20676/00000381

Link: KaoKore Dataset

KaoKore: A Pre-modern Japanese Art Facial Expression Dataset

From classifying handwritten digits to generating strings of text, the datasets which have received long-time focus from the machine learning community vary greatly in their subject matter. This has motivated a renewed interest in creating datasets which are socially and culturally relevant, so that algorithmic research may have a more direct and immediate impact. One such area is in history and the humanities, where better machine learning models could help to accelerate research. Along this line, newly created benchmarks and models have been proposed for historical Japanese cursive writing. At the same time, using machine learning for historical Japanese illustrations and artwork has remained largely uncharted. In this talk, Yingtao would be presenting the proposed new dataset KaoKore, which consists of faces from Pre-modern Japanese Illustrations, as well as demonstrating its value as both a classification dataset as well as a creative and artistic dataset, which we explore using generative models. It is hoped that the presented work bridges the research of humanity and machine learning.

Yingtao Tian (Google Brain Tokyo)

Yingtao Tian is a Research Software Engineer in Google Brain Tokyo. Prior to that, he obtained his PhD at Stony Brook University in May 2019, advised by Prof. Steven Skiena. His research interests lie in generative models and representation learning, as well as their applications in natural language processing, knowledge base modeling, social network modeling, image generation and bioinformatics, and much more. His current focus concerns the intersection between generative models and agents interacting with external words, as well as bridging machine learning with humanity research.

Presentation Slides: DOI:10.20676/00000383

Link: KaoKore Dataset

Crypko - a new workflow for anime character creation

Deep learning has shown great potentials of machine creativity. As a subarea of entertainment art, anime and manga have been a unique part of the Japanese economy and social culture. However, it takes tremendous efforts to master the skill of drawing, after which we are first capable of designing characters. In Preferred Networks, we are motivated to bridge the gap between amateurs and professional creators by adopting cutting-edge deep learning technologies. In this talk, I will present our recent efforts on exploring Japanese anime character creation with the help of machine creativity. More specifically, how we build practical systems to support amateurs to create their ideal anime characters and to automatically animate characters from a single illustration.

Yanghua Jin (Preferred Networks)

Yanghua Jin is a research engineer at Preferred Networks. Before that, he obtained an undergraduate degree in Computer Science and Engineering from Fudan University, Shanghai, China. His research interests include deep generative models and its application to Japanese animes and games. He founded the Crypko project to build real-world applications of anime deep generative models before joining Preferred Networks.

Presentation Slides: DOI:10.20676/00000382

Link: Crypko

Twitter

Twitterを#codh12で検索

Past CODH Seminars