NAVER CLOVA

Motivation and Goal

Recently, several successful AI systems such as Amazon Alexa, Google Assistant, and NAVER X LINE CLOVA are developed based on AI-assistant platforms. These AI platforms contain several common technologies including speech recognition/synthesis, natural language understanding, image recognition, and dialog recommendation.
Building a successful MLAIP is a challenging mission because it requires a novel combination of heterogeneous machine learning models in a unified framework with efficient data processing. The goals of this workshop are to investigate and advance important topics in Machine Learning for AI Platforms (MLAIPs) further. In addition, we expect to provide the collaboration opportunities to researchers on ML theory and diverse application domains as well as industrial engineers.

Important dates

Location

Program Schedule

Keynote Speech

Invited Speakers

• · Nako Sung (CLAIR, NAVER Corp.)

  · Title: NSML: A Machine Learning Platform That Enables You to Focus on Your Models

  · Abstract

  Machine learning libraries such as TensorFlow and PyTorch simplify model implementation. However, researchers are still required to perform a non-trivial amount of manual tasks such as GPU allocation, training status tracking, and comparison of models with different hyperparameter settings. We propose a system to handle these tasks and help researchers focus on models. We present the requirements of the system based on a collection of discussions from an online study group comprising 25k members. These include automatic GPU allocation, learning status visualization, handling model parameter snapshots as well as hyperparameter modification during learning, and comparison of performance metrics between models via a leaderboard. We describe the system architecture that fulfills these requirements and present a proof-of-concept implementation, NAVER Smart Machine Learning (NSML). We test the system and confirm substantial efficiency improvements for model development.

  · Biography

  Nako Sung is a leader of CLOVA AI Research, NAVER Corp. He received the BS degree from the Department of Computer Science and Engineering, Seoul National University. He was experienced in the game AI and development fields including Neowiz and NC soft for 18 years field. He is interested in deep reinforcement learning, deep generative models, and their real-world applications.

• · Kyomin Jung (Seoul National University)

  · Title: Efficient and Accurate Rumor Detection in Online Social Networks

  · Abstract

  The problem of identifying rumors is of practical importance especially in online social networks, since information can diffuse more rapidly and widely than the offline counterpart. In this talk, I will present our work that identifies characteristics of rumors in online social network by examining the following three aspects of diffusion: temporal, structural, and linguistic. For the temporal characteristics, we propose a new periodic time series model that considers daily and external shock cycles, where the model demonstrates that rumor likely have fluctuations over time. We also identify key structural and linguistic differences in the spread of rumors and non-rumors. Our selected features classify rumors with high precision and recall in the range of 87% to 92% on a Twitter data set. I will also present analysis of performance levels over varying time windows—from the first three days to nearly two months. Based on these findings, we suggest a new rumor classification algorithm that achieves competitive accuracy over both short and long time windows.

  · Biography

  Kyomin Jung is an associate professor in the Electrical and Computer Engineering department at Seoul National University (SNU), and has a joint appointment in the Mathematics department SNU. He worked at KAIST Computer Science department from 2009 to 2013. He received Ph.D. at MIT department of Mathematics in 2009, and B.Sc. at Seoul National University department of Mathematics in 2003. During his Ph.D., he worked at Microsoft Research Cambridge (2008), IBM T.J. Watson Research (2007), and Bell Labs (2006) as research internships. He was a recipient of Samsung Lee Kun Hee Scholarship for graduate study, and he was a gold medalist at the IMO (International Mathematical Olympiad) 1995. He is a recipient of the excellent new faculty funding from NRF Korea in 2012, and he is a member of SNU Big Data Institute. His research area includes machine learning and natural language processing with applications to social network analysis, recommendation systems, and machine translation. His work on machine learning and social network analysis have been published in major journals and conferences including IEEE PAMI, NIPS, ICML, AAAI, STOC, SODA, SIGMETRICS, ICDE and ICDM.

• · Huamin Qu (HKUST)

  · Title: Visual Analytics for Explainable AI

  · Abstract

  Classification is a fundamental problem in machine learning. In practice, interpretability is a desirable property of classification models (classifiers) in critical areas, such as security, medicine and finance. For instance, a quantitative trader may prefer a more interpretable model with less expected return due to its predictability and low risk. Unfortunately, the best-performing classifiers in many applications (e.g., deep neural networks) are complex machines whose predictions are difficult to explain. Thus, there is a growing interest in using visualization to understand, diagnose and explain intelligent systems in both academia and in industry. Many challenges need to be addressed in the formalization of explainability, and the design principles and evaluation of explainable intelligent systems. In this talk, I will first briefly introduce the concept and background of explainable classifiers. After that I will present thee works done at HKUST which use visualization to help explain deep neural networks: 1) RNNVis, a visual analytics tool for understanding and comparing recurrent neural networks (RNNs) for text-based applications. 2) CNNComparator, a visual analytics method to compare two different snapshots of a trained CNN model taken after different numbers of epochs. 3) DeepTracker, a visual analytics solution to reveal the rich dynamics of CNN training processes and help machine learning experts better understand, debug, and optimize CNNs.

  · Biography

  Huamin Qu is a professor in the Department of Computer Science and Engineering (CSE) at the Hong Kong University of Science and Technology. He also serves as the coordinator of the newly founded Human-Computer Interaction (HCI) group at the CSE department. His main research interests are in visualization and human-computer interaction, with focuses on urban informatics, social network analysis, e-learning, and explainable artificial intelligence. He has co-authored more than 100 refereed papers including more than 40 papers in the IEEE Transactions on Visualization and Computer Graphics (TVCG). His research has been recognized by many awards including 7 best paper/honorable mention awards, 2009 IBM Faculty Award, 2014 Higher Education Scientific and Technological Progress Award (Second Class) from the Ministry of Education of China, 2015 HKICT Best Innovation (Innovative Technology) Silver Award from the Hong Kong Institution of Engineers, 2015 APICTA Merit Award in E-Learning from the Asia Pacific ICT Alliance, and 2016 Distinguished Collaborator Award from Huawei Nah's Ark Lab. He obtained a BS in Mathematics from Xi'an Jiaotong University, China, an MS and a PhD (2004) in Computer Science from the Stony Brook University.

• · Il Chul Moon (KAIST)

  · Title: Interaction between Machine Learning and Multi-Agent System

  · Abstract

  Under the umbrella of artificial intelligence, the breakthroughs in the machine learning (ML) has been spread to the sibling disciplines, such as multi-agent system (MAS). This presentation introduces the recent theoretic interplay between ML and MAS. First, ML could be an automated controller to strengthen the operation and the validity of MAS, and we show this fusion through a case study of automatic calibration of MAS. Second, MAS could be a playground of individual ML instances to interact and compete between them, which we survey the researches, such as Gang of GANs.

  · Biography

  Il-Chul Moon received the Ph.D. degree from the School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA, in 2008. He is currently an Assistant Professor with the Department of Industrial and Systems Engineering, Korea Advanced Institute of Science and Technology, Daejeon, Korea. His research interests include the overlapping area of computer science, management, sociology, and operations research, and also military command and control analysis, counterterrorism analysis, intelligence analysis, and disaster management.

• · Lucy Eunjung Park (Papago, NAVER Corp.)

  · Title: My model has higher BLEU, can I ship it?

  · Abstract

  In the real world, there is a discrepancy between AI research and actual AI services. It is not enough get higher scores with a machine learning model according to some performance measure such as BLEU -- there is much more to do before and after a model is delivered to actual users. In this talk, we explore what such discrepancies are, and share several ways to overcome them.

  · Biography

  Lucy is a research scientist at NAVER. During her Ph.D. studies at Seoul National University, she investigated better representations for text. Her current research area is machine translation -- focused on user log analysis and multilingual NMT.

Accepted Posters

Organizers

• · Byoung-Tak Zhang (Seoul National University)
• · Sungroh Yoon (Seoul National University)
• · Dit-Yan Yeung (Hong Kong University of Science and Technology)
• · Sung Kim (Hong Kong University of Science and Technology)
• · Jaesik Choi (Ulsan National Institute of Science and Technology)
• · Jung-Woo Ha (CLOVA, NAVER Corp.)