https://www.microsoft.com/en-us/research/blog/efficiently-and-effectively-scaling-up-language-model-pretraining-for-best-language-representation-model-on-glue-and-superglue/?OCID=msr_blog_TNLRV5_tw

Microsoft AI at Scale ์˜ ์ผํ™˜์œผ๋กœ Turing NLP ๋ชจ๋ธ ์ œํ’ˆ๊ตฐ์€ ์ฐจ์„ธ๋Œ€ AI ๊ฒฝํ—˜์„ ์ง€์›ํ•˜๊ธฐ ์œ„ํ•ด Microsoft ์ „์ฒด์—์„œ ๋Œ€๊ทœ๋ชจ๋กœ ์‚ฌ์šฉ๋˜๊ณ  ์žˆ์Šต๋‹ˆ๋‹ค. ์˜ค๋Š˜ ์šฐ๋ฆฌ๋Š” ์ตœ์‹  Microsoft Turing ๋ชจ๋ธ(T-NLRv5)์ด SuperGLUE ๋ฐ GLUE ๋ฆฌ๋”๋ณด๋“œ์˜ ์ตœ์ƒ์œ„์— ์žˆ์œผ๋ฉฐ ์ธ๊ฐ„์˜ ์„ฑ๋Šฅ ๋ฐ ๊ธฐํƒ€ ๋ชจ๋ธ์„ ํ›จ์”ฌ ๋Šฅ๊ฐ€ ํ•œ๋‹ค๋Š” ์‚ฌ์‹ค์„ ๋ฐœํ‘œํ•˜๊ฒŒ ๋œ ๊ฒƒ์„ ๊ธฐ์˜๊ฒŒ ์ƒ๊ฐํ•ฉ๋‹ˆ๋‹ค . ํŠนํžˆ, T-NLRv5๋Š” ์ธ๊ฐ„ ํŒจ๋ฆฌํ‹ฐ๊ฐ€ ์•„์ง ์ถฉ์กฑํ•˜์ง€ ๋ชปํ•œ ๋งˆ์ง€๋ง‰ ๋‘ ๊ฐœ์˜ GLUE ์ž‘์—…์ธ GLUE ๋ฒค์น˜๋งˆํฌ์—์„œ MNLI ๋ฐ RTE์—์„œ ๋จผ์ € ํœด๋จผ ํŒจ๋ฆฌํ‹ฐ๋ฅผ ๋‹ฌ์„ฑํ–ˆ์Šต๋‹ˆ๋‹ค. ๋˜ํ•œ, T-NLRv5๋Š” ์ตœ๊ทผ์˜ ์‚ฌ์ „ ํ›ˆ๋ จ ๋ชจ๋ธ๋ณด๋‹ค ๋” ํšจ์œจ์ ์ด๋ฉฐ, 50% ๋” ์ ์€ ๋งค๊ฐœ๋ณ€์ˆ˜์™€ ์‚ฌ์ „ ํ›ˆ๋ จ ์ปดํ“จํŒ… ๋น„์šฉ์œผ๋กœ ๋น„์Šทํ•œ ํšจ๊ณผ๋ฅผ ๋‹ฌ์„ฑํ•ฉ๋‹ˆ๋‹ค.