https://www.semianalysis.com/p/google-we-have-no-moat-and-neitherGoogle "We Have No Moat, And Neither Does OpenAI"Leaked Internal Google Document Claims Open Source AI Will Outcompete Google and OpenAIwww.semianalysis.com
๋ฌธ๋ซ์ close ai
๊ฐ์ ๋ก open ์์ผ๋ฒ๋ฆฌ๋ ๋ ผ๋ฌธ ๋์ดย ย ใ ใ
https://arxiv.org/abs/2403.06634Stealing Part of a Production Language ModelWe introduce the first model-stealing attack that extracts precise, nontrivial information from black-box production language models like OpenAI's ChatGPT or Google's PaLM-2. Specifically, our attack recovers the embedding projection layer (up to symmetries) of a transformer model, given typical API access. For under $20 USD, our attack extracts the entire projection matrix of OpenAI's Ada and Babbage language models. We thereby confirm, for the first time, that these black-box models have a hidden dimension of 1024 and 2048, respectively. We also recover the exact hidden dimension size of the gpt-3.5-turbo model, and estimate it would cost under $2,000 in queries to recover the entire projection matrix. We conclude with potential defenses and mitigations, and discuss the implications of possible future work that could extend our attack.arxiv.org
์ ์ด๊ฑฐ ๋ด๊ฐ ์ฌ๋ฆด๋ ค๊ณ ํ๋๋ฐ ์ด์ ์ ์ ๋ฌผ ๋ชปํ๋ค ์ด์ฉ๋ฉด ํ ๋ ํธ์ ๋ ๋์๋์ ์๋ค๋๋ฐ ใทใทใท
์ด๋ฏธ ๋ ๋ํ์์ ์จ๊ฐ ํ์ทจ์๋๋ฅผ ๋ค ํ๊ณ ์์ํ ๋ฐ, ๋ ผ๋ฌธ๋์จ๊ฒ ๋ญ
ํ