Our Projects

GraphDB and Graph Neural Network

2021.05.20 | Research

1. Introduction This post aims to introduce the integration of graph database (GraphDB) and deep model based Machine Learning (ML) on a graph. While this has not caught a wide audience yet, we believe this can be one of the successful applications of Machine Learning. Like improvement in ML opened the promising field of optimizing Relational Database System (RDBS) using ML, we expect the improvement of Machine Learning on a graph can shed a new light on the optimization of GraphDB. ๐Ÿ“Œ Integration of database and machine learning in RDBS Here we show the example how ML can be helpful in enhancing

Jiyoung PARK

TVM: An Automated End-to-End Optimizing Compiler for Deep Learning

2021.02.21 | Research, KAIST EE Co-op, Internship

0. Disclaimer Apache TVM์€ ํ˜„์žฌ๋„ ๊พธ์ค€ํžˆ ๊ฐœ๋ฐœ์ค‘์ธ ํ”„๋กœ์ ํŠธ์ด๊ธฐ์— ํ™ˆํŽ˜์ด์ง€, ์ฝ”๋“œ ๋“ฑ์ด ๋น ๋ฅด๊ฒŒ ๋ฐ”๋€Œ๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋ณธ ํฌ์ŠคํŠธ๋Š” 2020/08 ~ 2021/01์— ์ž‘์„ฑ๋œ ๊ธ€์ž…๋‹ˆ๋‹ค. ์ตœ์‹ ์˜ ๋‚ด์šฉ๊ณผ ๋งํฌ/๋‚ด์šฉ ๋“ฑ์ด ๋‹ฌ๋ผ์งˆ ์ˆ˜ ์žˆ์Œ์„ ๋ฏธ๋ฆฌ ๋ฐํž™๋‹ˆ๋‹ค. ๋ณธ ํฌ์ŠคํŠธ๋Š” TVM์„ ์ฒ˜์Œ ์ ‘ํ•˜๋Š” ์‚ฌ๋žŒ๋“ค์—๊ฒŒ ์ฃผ์•ˆ์  ๋“ฑ์„ ์ œ์‹œํ•˜๋Š” ์—ญํ• ์„ ํ•˜๋Š” ๊ฒƒ์ด ๋ชฉํ‘œ์ž…๋‹ˆ๋‹ค. TVM์„ ์ฒ˜์Œ ๋ณด์‹œ๋Š” ๋ถ„๋“ค๋„ ์ด ๊ธ€์„ ํ†ตํ•ด ์–ด๋Š ์ •๋„ ํฐ ๊ทธ๋ฆผ์„ ๊ทธ๋ฆด ์ˆ˜ ์žˆ์—ˆ์œผ๋ฉด ์ข‹๊ฒ ์Šต๋‹ˆ๋‹ค. ์ž๋ฃŒ์˜ ์ผ๋ถ€๋Š” ๋…ผ๋ฌธ์—์„œ, ์ผ๋ถ€๋Š” ํ™ˆํŽ˜์ด์ง€์—์„œ, ์ผ๋ถ€๋Š” ์ปจํผ๋Ÿฐ์Šค ์ž๋ฃŒ์—์„œ ๋ฐœ์ทŒํ•˜์˜€๊ณ , ์ด๋“ค์„ ๋ฐ”ํƒ•์œผ๋กœ ํ•œ ๋ฒˆ์—ญ+๊ฐ€์ด๋“œ๋ผ์ธ ์ •๋„๋ผ๊ณ  ๋ณด๋ฉด ๋  ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. TVM echosystem์— ์ž‘์€ ๋„์›€์ด ๋  ์ˆ˜ ์žˆ๋‹ค๋ฉด ํ–‰๋ณตํ•  ๊ฒƒ ๊ฐ™์Šต๋‹ˆ๋‹ค. ์ตœ๊ทผ ๋”ฅ๋Ÿฌ๋‹์— ๊ด€๋ จ๋œ ์ƒˆ๋กœ์šด ๊ธฐ์ˆ ๋“ค์ด ๋ฌด์„œ์šด ์†๋„๋กœ ๋“ฑ์žฅํ•˜๊ณ , ๋˜ ๋ฐœ์ „ํ•˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค.

Woojin LEE

ML2 multiagent RL mini-game environments

2020.04.28 | Research, Internship

1. Introduction This post is an introduction to a simple multi-agent reinforcement learning environment, ML2-MARL-ENV, that can be used to train multi-agent RL algorithms. It aims to explain the use cases of the environment to help future RL researchers in training a multi-agent RL. Another purpose of this blog is to share my personal experiences that I have come across in the development stage, which will hopefully help others better understand the nature of the project. The API of the environment follows that of the convention of OpenAI gym environment. 2. Recent research and motivations 2.1 Backgrounds Deep ...

Taemin HA


2020.04.17 | Research, KAIST EE Co-op, Internship

1. About LLVM IR (Intermediate Representation) LLVM ์€ SSA-based ์ปดํŒŒ์ผ๋Ÿฌ์ž…๋‹ˆ๋‹ค. SSA(Static Single Assignment)๋Š” ๋ณ€์ˆ˜๊ฐ€ ํ•œ๋ฒˆ๋งŒ ํ• ๋‹น๋˜๋„๋ก ํ•˜์—ฌ ๋ณ€์ˆ˜์˜ ๋ณต์žก์„ฑ์„ ์ค„์ด๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. LLVM์˜ ํฐ ์žฅ์ ์€ ์›ํ•˜๋Š” ๊ธฐ๋Šฅ์„ ์ƒˆ๋กœ ์ถ”๊ฐ€ํ•˜๊ธฐ ์‰ฌ์šด ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์˜คํ”ˆ์†Œ์Šค์ด๋ฉด์„œ API๊ฐ€ ์ž˜ ๋ฌธ์„œํ™” ๋˜์–ด์žˆ๊ณ , ๊ธฐ๋Šฅ์ด ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋กœ ๋ถ„๋ฆฌ๋˜์–ด ๋™์ž‘ํ•˜๊ธฐ ๋•Œ๋ฌธ์ž…๋‹ˆ๋‹ค. LLVM์˜ ์ฝ”์–ด ๋ผ์ด๋ธŒ๋Ÿฌ๋ฆฌ๋Š” Optimizer์ด๊ณ  Optimizer์˜ ์†Œ์Šค์™€ ํƒ€๊ฒŸ์— ๋…๋ฆฝ์ ์ธ ์ค‘๊ฐ„ ํ‘œํ˜„์ธ intermediate representation(IR)์„ ๋ฐ›์•„ ์ตœ์ ํ™”ํ•˜์—ฌ ์ƒˆ๋กœ์šด IR์„ ๋„์ถœํ•ฉ๋‹ˆ๋‹ค. ํ˜„์‹ค์˜ CPU๋Š” ๋ ˆ์ง€์Šคํ„ฐ๊ฐ€ ํ•œ์ •๋˜์–ด ์žˆ์ง€๋งŒ, IR์€ ํƒ€๊ฒŸ์˜ ๋ ˆ์ง€์Šคํ„ฐ ์ˆ˜์™€ ๊ด€๊ณ„๊ฐ€ ์—†์–ด์•ผ ํ•˜๊ธฐ ๋•Œ๋ฌธ์— ๊ฐ€์ƒ ๋ ˆ์ง€์Šคํ„ฐ์˜ ์ˆ˜๋ฅผ ๊ณ„์† ๋Š˜๋ฆฌ๋ฉด์„œ ์ž‘์„ฑ๋ฉ๋‹ˆ๋‹ค. LLVM์—์„œ์˜ SSA๋Š” ๋ ˆ์ง€์Šคํ„ฐ์— ...

Sooyeon LEE

Reinforcement learning library (RL2)

2020.10.28 | Opensource

Reinforcement Learning Library for deep RL algorithms A opensource library built to provide a RL algorithm development framework. RL2 is built on the philosophy to unify different RL algorithm under a generalized core, to recycle codes across algorithms and make modifications to algorithms easy. Structure Simplified layout of components that consists the RL2 structure. Worker : Governs the env-agent interaction loop Env : Environment object Agent : Algorithm specific agent that governs information from/to the environment Model : Takes care of everything related to inferring and updating the neural network ...

Team ML2

Value Function Geometry

2020.09.28 | Research, Internship

1. Introduction Recently, research on the geometric properties of value functions and attempts to apply these properties to various fields of Reinforcement Learning (RL) have been actively conducted. This post is an introduction to Value Function Geometry including an introduction of the paper 'The Value Function Polytope in Reinforcement Learning' by Dadashi et al. This post aims to explain the basic concepts of value function linear approximation and representation learning. Also, the paper 1 establishes the geometric shape of value function space in finite state-action Markov decision processes: a general polytope. By reimplementing the paper myself,

Bongsoo YI

MAS tutorials

2021.06.15 | Opensource

Preface ์ด ํŠœํ† ๋ฆฌ์–ผ์€ Multi-agent ๋ฌธ์ œ๋ฅผ Deep Reinforcement Learning (DRL) ์˜ ๊ด€์ ์œผ๋กœ ๋ฐ”๋ผ๋ณด๋ฉฐ Snake Leaderboard๋ฅผ ์‚ฌ์šฉํ•จ์— ์žˆ์–ด multi-agent๋ฌธ์ œ์˜ ์ ‘๊ทผ๋ฒ•์„ ๋”์šฑ ํ™•์žฅ์‹œํ‚ค๊ณ  ์‹ถ์€ ๋ถ„๋“ค์„ ์œ„ํ•œ ์ •๋ณด๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ํŠนํžˆ๋‚˜ ๊ฐ•ํ™”ํ•™์Šต์˜ ์ธก๋ฉด์—์„œ์˜ multi-agent ๋ฌธ์ œ๋ฅผ ๋‹ค๋ฃจ๊ธฐ ์œ„ํ•ด์„œ๋Š” ์•„์ฃผ ๋งŽ์€ ์ด๋ก ์  ๋ฐฐ๊ฒฝ์ง€์‹์„ ์š”๊ตฌํ•˜์ง€๋งŒ ์ด ํŠœํ† ๋ฆฌ์–ผ์—์„œ๋Š” ์ตœ๋Œ€ํ•œ ํž˜์„ ๋นผ๊ณ  ์กฐ๊ธˆ ๋” ๊ฐœ๋…์ ์œผ๋กœ ์ ‘๊ทผํ•˜๋ คํ•ฉ๋‹ˆ๋‹ค. ๋งŒ์•ฝ ํŠœํ† ๋ฆฌ์–ผ ์ „์ฒด๋ฅผ ํ•˜๋‚˜์˜ ๊ฐœ๋…์œผ๋กœ ์š”์•ฝํ•œ๋‹ค๋ฉด free-for-all ํ•œ ์ƒํ™ฉ์—์„œ์˜ Multi-agent Deep Reinforcement Learning (MDRL)์œผ๋กœ ์š”์•ฝํ•  ์ˆ˜ ์žˆ์„ํ…๋ฐ ๋ณธ๋ก ์œผ๋กœ ๋“ค์–ด๊ฐ€๊ธฐ์— ์•ž์„œ ์ด MDRL์„ ํ•œ ๋‹จ์–ด์”ฉ ๋–ผ์–ด์„œ ์‚ดํŽด๋ณด๋Š” ๊ฒƒ์œผ๋กœ ์‹œ์ž‘ํ•˜๊ฒ ์Šต๋‹ˆ๋‹ค. ์šฐ์„  MDRL์€ ํฌ๊ฒŒ 3๊ฐ€์ง€์˜ ๊ฐœ๋…์œผ๋กœ ๋‚˜๋ˆŒ ...

Team ML2