Our Research Areas
Our Publications

#Reinforcement Learning #Publications

Efficiently Learning the Value Distribution for Actor-Critic Methods

ICML 2021

Reinforcement Learning (RL) has become one of the major categories in the field of machine learning in the recent years via the breakthroughs such as approximation of complex non-linear function through deep neural networks. Among these, a distributional perspective on the value function estimation has contributed on taking a big jump in the performance of RL algorithms. However, proper discussions of distributional RL (DRL) are still limited to specific algorithms or network architectures such as Q-learning or deterministic policy gradient. In the situation at hand, we have worked to address some of the critical aspects in RL that was left out in the distributional perspective. The details and the findings of this journey can be found in our recent work 'GMAC: A Distributional'

Daniel Wontae Nam, Younghoon Kim, Chan Y. Park

#Reinforcement Learning #ML Applications

Structural optimization of a one-dimensional freeform metagrating deflector via deep reinforcement learning

ACS Photonics 2022

The increasing demand on a versatile high-performance metasurface requires a freeform design method that can handle a huge design space, which is many orders of magnitude larger than that of conventional fixed-shape optical structures. In this work, we formulate the designing process of one-dimensional freeform Si metasurface beam deflectors as a reinforcement learning problem to find their optimal structures consistently without requiring any prior metasurface data. During training, a deep Q-network-based agent stochastically explores the device design space around the learned trajectory optimized for deflection efficiency. The devices discovered by the agents show overall improvements in maximum efficiency compared to the ones that state-of-the-art baseline methods find at various wavelengths and deflection angles.

D. Seo, D. W. Nam, J. Park, C. Y. Park, and M. S. Jang

#Reinforcement Learning #ML Applications

Sample-efficient inverse design of freeform nanophotonic devices with physics-informed reinforcement learning

Nanophotonics 2024

Finding an optimal device structure in the vast combinatorial design space of freeform nanophotonic design has been an enormous challenge. In this study, we propose physics-informed reinforcement learning (PIRL) that combines the adjoint-based method with reinforcement learning to improve the sample efficiency by an order of magnitude compared to conventional reinforcement learning and overcome the issue of local minima. To illustrate these advantages of PIRL over other conventional optimization algorithms, we design a family of one-dimensional metasurface beam deflectors using PIRL, exceeding most reported records. We also explore the transfer learning capability of PIRL that further improves sample efficiency and demonstrate how the minimum feature size of the design can be enforced in PIRL through reward engineering. With its high sample efficiency, robustness, and ability to seamlessly incorporate practical device design constraints, our method offers a promising approach to highly combinatorial freeform device optimization in various physical domains.

Chaejin Park , Sanmun Kim , Anthony W. Jung , Juho Park , Dongjin Seo , Yongha Kim , Chanhyung Park , Chan Y. Park and Min Seok Jang

#Computer Vision

In-Season Wall-to-Wall Crop-Type Mapping Using Ensemble of Image Segmentation Models

IEEE Transactions on Geoscience and Remote Sensing (Volume 61)


This work applies computer vision to multispectral satellite images to create pre-harvest agricultural maps at large scales. We demonstrate the effectiveness of our work not only by outperforming existing approaches in terms of evaluation metrics but also by generating wall-to-wall corn and soybean maps across the entire US corn-belt, spanning over 2 million square kilometers. We’re able to generate highly accurate maps approx. six months before the corresponding year’s data is released by the USDA. Such preharvest maps can be very useful for forecasting yields and mitigating any supply shortages.

Sheir A. Zaheer, Youngryel Ryu, Junghee Lee, Zilong Zhong, Kyungdo Lee

#Computer Vision

RCV2023 Challenges: Benchmarking Model Training and Inference for Resource-Constrained Deep Learning

ICCV 2023 workshop


This paper delves into the results of two resource-constrained deep learning challenges, part of the workshop on Resource-Efficient Deep Learning for Computer Vision (RCV) at ICCV 2023, focusing on memory and time limitations. The challenges garnered significant global participation and showcased a range of intriguing solutions. The paper outlines the problem statements for both tracks, summarizes baseline and top-performing approaches, and provides a detailed analysis of the methods used. While the presented solutions constitute promising initial progress, they represent the beginning of efforts needed to address this complex issue. We conclude by emphasizing the importance of sustained research efforts to fully address the challenges of resource-constrained deep learning.

Rishabh Tiwari, Arnav Chavan, Deepak Gupta, Gowreesh Mago, Animesh Gupta, Akash Gupta, Suraj Sharan, Yukun Yang, Shanwei Zhao, Shihao Wang, Youngjun Kwak, Seonghun Jeong, Yunseung Lee, Changick Kim, Subin Kim, Ganzorig Gankhuyag, Ho Jung, Junwhan Ryu, HaeMoon Kim, Byeong H. Kim, Tu Vo, Sheir Zaheer, Alexander Holston, Chan Park, Dheemant Dixit, Nahush Lele, Kushagra Bhushan, Debjani Bhowmick, Devanshu Arya, Sadaf Gulshad, Amirhossein Habibian, Amir Ghodrati, Babak Bejnordi, Jai Gupta, Zhuang Liu, Jiahui Yu, Dilip Prasad, Zhiqiang Shen

#ML Applications

Free-form optimization of nanophotonic devices: from classical methods to deep learning

Nanophotonics 2022


Nanophotonic devices have enabled microscopic control of light with an unprecedented spatial resolution by employing subwavelength optical elements that can strongly interact with incident waves. However, to date, most nanophotonic devices have been designed based on fixed-shape optical elements, and a large portion of their design potential has remained unexplored. It is only recently that free-form design schemes have been spotlighted in nanophotonics, offering routes to make a break from conventional design constraints and utilize the full design potential. In this review, we systematically overview the nascent yet rapidly growing field of

Juho Park, Sanmun Kim, Daniel Wontae Nam, Haejun Chung, Chan Y. Park and Min Seok Jang.

#ML Applications

Inverse design of organic light-emitting diode structure based on deep neural networks

Nanophotonics 2021


The optical properties of thin-film light emitting diodes (LEDs) are strongly dependent on their structures due to light interference inside the devices. However, the complexity of the design space grows exponentially with the number of design parameters, making it challenging to optimize the optical properties of multilayer LEDs with rigorous electromagnetic simulations. In this work, we demonstrate an artificial neural network that can predict the light extraction efficiency of an organic LED structure in 30 ms, which is ∼103 times faster than the rigorous simulation in a single-treaded execution with root-mean-squared error

Sanmun Kim, Jeong Min Shin, Jaeho Lee, Chanhyung Park, Songju Lee, Juho Park, Dongjin Seo, Sehong Park, Chan Y. Park and Min Seok Jang

#Computer Vision

Investigating Pixel Robustness using Input Gradients


This post aims to cover main concepts from the paper ‘Where to be Adversarial Perturbations Added? Investigating and Manipulating Pixel Robustness using Input Gradients’ by Hwang et al _. The paper connects the gradients of input features to the robustness of a classification model, and shows that the robustness can be manipulated indirectly through changing the gradient flows within the model. Adversarial attack can be defined as a process of generating adversarial examples to a given classifier, which are samples that are misclassified by the model but are only slightly different from correctly classified samples drawn from the data distribution . Projected Gradient Descent (PGD) is a popular attack method that iteratively generates adversarial examples as the following

Jisung Hwang, Younghoon Kim, Sanghyuk Chun, Jaejun Yoo, Ji-Hoon Kim & Dongyoon Han

#Reinforcement Learning

Distilling Curiosity for Exploration


This post is an introduction to the paper 'Curiosity Bottleneck: Exploration by Distilling Task-Specific Novelty' by Kim et al. The paper deals with informative exploration method when task-irrelevant noise are present within the visual observation. By distilling the informative from the uninformative, the agent is able to successfully ignore the distractive visual entities when making decision about choice of action or calculating the intrinsic reward for exploration. Exploration vs. exploitation is a well known paradox in reinforcement learning. A careful tradeoff between the two is required for the optimal performance of the learning algorithms.

Youngjin Kim, Wontae Nam, Hyunwoo Kim, Ji-Hoon Kim, Gunhee Kim