Evolutionary Optimization of Model Merging RecipesWe present a novel application of evolutionary algorithms to automate the creation of powerful foundation models. While model merging has emerged as a promising approach for LLM development due to its cost-effectiveness, it currently relies on human intuition and domain knowledge, limiting its potential. Here, we propose an evolutionary approach that overcomes this limitation by automatically discovering effective combinations of diverse open-source models, harnessing their collective intelligence without requiring extensive additional training data or compute. Our approach operates in both parameter space and data flow space, allowing for optimization beyond just the weights of the individual models. This approach even facilitates cross-domain merging, generating models like a Japanese LLM with Math reasoning capabilities. Surprisingly, our Japanese Math LLM achieved state-of-the-art performance on a variety of established Japanese LLM benchmarks, even surpassing models with significantly more parameters, despite not being explicitly trained for such tasks. Furthermore, a culturally-aware Japanese VLM generated through our approach demonstrates its effectiveness in describing Japanese culture-specific content, outperforming previous Japanese VLMs. This work not only contributes new state-of-the-art models back to the open-source community, but also introduces a new paradigm for automated model composition, paving the way for exploring alternative, efficient approaches to foundation model development.arxiv.org


여러 모델들을 병합하는 효과적인 방법 개발함

그걸로






일본어 기반 LLM "EvoLLM-JP"와

이미지인식 얹은 "EvoVLM-JP" 개발












일본어 프롬프트 인식되는 이미지생성모델 "EvoSDXL-JP" 개발












우키요에 생성 모델 "Evo-Ukiyoe" 와 채색모델 "Evo-Nishikie" 개발












https://huggingface.co/SakanaAI



딥러닝 공부까진 못해도 블로그읽고 풀오픈소스로뿌린모델은 좀 써보자

이새끼들은 걍 특갤이 세상의 전부임