https://twitter.com/andrewwhite01/status/1670794000398184451



https://arxiv.org/abs/2304.05376

ChemCrow: Augmenting large-language models with chemistry toolsOver the last decades, excellent computational chemistry tools have beendeveloped. Their full potential has not yet been reached as most arechallenging to learn and exist in isolation. Recently, large-language models(LLMs) have shown strong performance in tasks across domains, but struggle withchemistry-related problems. Moreover, these models lack access to externalknowledge sources, limiting their usefulness in scientific applications. Inthis study, we introduce ChemCrow, an LLM chemistry agent designed toaccomplish tasks across organic synthesis, drug discovery, and materialsdesign. By integrating 18 expert-designed tools, ChemCrow augments the LLMperformance in chemistry, and new capabilities emerge. Our agent autonomouslyplanned and executed the syntheses of an insect repellent, threeorganocatalysts, and guided the discovery of a novel chromophore. Ourevaluation, including both LLM and expert assessments, demonstrates ChemCrow'seffectiveness in automating a diverse set of chemical tasks. Surprisingly, wefind that GPT-4 as an evaluator cannot distinguish between clearly wrong GPT-4completions and Chemcrow's performance. There is a significant risk of misuseof tools like ChemCrow, and we discuss their potential harms. Employedresponsibly, our work not only aids expert chemists and lowers barriers fornon-experts, but also fosters scientific advancement by bridging the gapbetween experimental and computational chemistry. Publicly available code canbe found at https://github.com/ur-whitelab/chemcrow-publicarxiv.org


์ง€๋‚œ ์ˆ˜์‹ญ ๋…„ ๋™์•ˆ ๋›ฐ์–ด๋‚œ ๊ณ„์‚ฐ ํ™”ํ•™ ๋„๊ตฌ๊ฐ€ ๊ฐœ๋ฐœ๋˜์—ˆ์Šต๋‹ˆ๋‹ค. ํ•˜์ง€๋งŒ ๋Œ€๋ถ€๋ถ„ ํ•™์Šตํ•˜๊ธฐ ์–ด๋ ต๊ณ  ๋…๋ฆฝ์ ์œผ๋กœ ์กด์žฌํ•˜๊ธฐ ๋•Œ๋ฌธ์— ์•„์ง ๊ทธ ์ž ์žฌ๋ ฅ์„ ์™„์ „ํžˆ ๋ฐœํœ˜ํ•˜์ง€ ๋ชปํ–ˆ์Šต๋‹ˆ๋‹ค. ์ตœ๊ทผ ๋Œ€๊ทœ๋ชจ ์–ธ์–ด ๋ชจ๋ธ(LLM)์€ ์—ฌ๋Ÿฌ ์˜์—ญ์˜ ์ž‘์—…์—์„œ ๊ฐ•๋ ฅํ•œ ์„ฑ๋Šฅ์„ ๋ณด์˜€์ง€๋งŒ ํ™”ํ•™ ๊ด€๋ จ ๋ฌธ์ œ์—์„œ๋Š” ์–ด๋ ค์›€์„ ๊ฒช๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ ์ด๋Ÿฌํ•œ ๋ชจ๋ธ์€ ์™ธ๋ถ€ ์ง€์‹ ์†Œ์Šค์— ๋Œ€ํ•œ ์ ‘๊ทผ์„ฑ์ด ๋ถ€์กฑํ•˜์—ฌ ๊ณผํ•™ ์‘์šฉ ๋ถ„์•ผ์—์„œ ์œ ์šฉ์„ฑ์ด ์ œํ•œ๋ฉ๋‹ˆ๋‹ค. ์ด ์—ฐ๊ตฌ์—์„œ๋Š” ์œ ๊ธฐ ํ•ฉ์„ฑ, ์‹ ์•ฝ ๊ฐœ๋ฐœ ๋ฐ ์žฌ๋ฃŒ ์„ค๊ณ„ ์ „๋ฐ˜์— ๊ฑธ์นœ ์ž‘์—…์„ ์ˆ˜ํ–‰ํ•˜๋„๋ก ์„ค๊ณ„๋œ LLM ํ™”ํ•™ ์—์ด์ „ํŠธ์ธ ์ผํฌ๋กœ์šฐ(ChemCrow)๋ฅผ ์†Œ๊ฐœํ•ฉ๋‹ˆ๋‹ค. ์ „๋ฌธ๊ฐ€๊ฐ€ ์„ค๊ณ„ํ•œ 18๊ฐœ์˜ ๋„๊ตฌ๋ฅผ ํ†ตํ•ฉํ•˜์—ฌ ํ™”ํ•™ ๋ถ„์•ผ์˜ LLM ์„ฑ๋Šฅ์„ ๊ฐ•ํ™”ํ•œ ์ผํฌ๋ผ์šฐ๋Š” ์ƒˆ๋กœ์šด ๊ธฐ๋Šฅ์„ ์„ ๋ณด์ž…๋‹ˆ๋‹ค. ์ €ํฌ ์—์ด์ „ํŠธ๋Š” ๋ฐฉ์ถฉ์ œ์™€ ์„ธ ๊ฐ€์ง€ ์œ ๊ธฐ ์ด‰๋งค์˜ ํ•ฉ์„ฑ์„ ์ž์œจ์ ์œผ๋กœ ๊ณ„ํšํ•˜๊ณ  ์‹คํ–‰ํ–ˆ์œผ๋ฉฐ, ์ƒˆ๋กœ์šด ๋ฐœ์ƒ‰๋‹จ ๋ฐœ๊ฒฌ์„ ์•ˆ๋‚ดํ–ˆ์Šต๋‹ˆ๋‹ค. LLM๊ณผ ์ „๋ฌธ๊ฐ€ ํ‰๊ฐ€๋ฅผ ๋ชจ๋‘ ํฌํ•จํ•œ ๋‹น์‚ฌ์˜ ํ‰๊ฐ€๋Š” ๋‹ค์–‘ํ•œ ํ™”ํ•™ ์ž‘์—…์„ ์ž๋™ํ™”ํ•˜๋Š” ๋ฐ ์žˆ์–ด ์ผํฌ๋ผ์šฐ์˜ ํšจ์œจ์„ฑ์„ ์ž…์ฆํ•ฉ๋‹ˆ๋‹ค. ๋†€๋ž๊ฒŒ๋„ ํ‰๊ฐ€์ž๋กœ์„œ์˜ GPT-4๋Š” ๋ช…๋ฐฑํžˆ ์ž˜๋ชป๋œ GPT-4 ์™„๋ฃŒ์™€ ์ผํฌ๋กœ์šฐ์˜ ์„ฑ๋Šฅ์„ ๊ตฌ๋ถ„ํ•  ์ˆ˜ ์—†๋‹ค๋Š” ์‚ฌ์‹ค์„ ๋ฐœ๊ฒฌํ–ˆ์Šต๋‹ˆ๋‹ค. ์ผํฌ๋กœ์šฐ์™€ ๊ฐ™์€ ๋„๊ตฌ๋Š” ์˜ค์šฉ๋  ์œ„ํ—˜์ด ํฌ๋ฉฐ, ๊ทธ ์ž ์žฌ์  ํ•ด์•…์— ๋Œ€ํ•ด ๋…ผ์˜ํ•ฉ๋‹ˆ๋‹ค. ์ฑ…์ž„๊ฐ ์žˆ๊ฒŒ ์‚ฌ์šฉ๋˜๋Š” ์ผํฌ๋กœ์šฐ์˜ ์ž‘์—…์€ ์ „๋ฌธ ํ™”ํ•™์ž๋ฅผ ๋•๊ณ  ๋น„์ „๋ฌธ๊ฐ€์˜ ์žฅ๋ฒฝ์„ ๋‚ฎ์ถœ ๋ฟ๋งŒ ์•„๋‹ˆ๋ผ ์‹คํ—˜ ํ™”ํ•™๊ณผ ๊ณ„์‚ฐ ํ™”ํ•™ ์‚ฌ์ด์˜ ๊ฐ„๊ทน์„ ์ขํ˜€ ๊ณผํ•™์  ๋ฐœ์ „์„ ์ด‰์ง„ํ•˜๋Š” ๋ฐ ๊ธฐ์—ฌํ•ฉ๋‹ˆ๋‹ค. ๊ณต๊ฐœ์ ์œผ๋กœ ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ์ฝ”๋“œ๋Š” ๋‹ค์Œ์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

์•ค๋“œ๋ฅ˜ ํ™”์ดํŠธ๋Š” ๋ฏธ๊ตญ ๋กœ์ฒด์Šคํ„ฐ๋Œ€ ํ™”ํ•™๊ณตํ•™๊ณผ ๋ถ€๊ต์ˆ˜์ž„