3D-printed Deep Learning neural network uses light instead of electronsIt's a novel idea, using light diffracted through numerous plates instead of electrons. And to some, it might seem a little like replacing a computer with an abacus, but researchers at UCLA have high hopes for their quirky, shiny, speed-of-light artificial neural network.newatlas.com


Just a moment...Just a moment...www.science.org



이거를 한 5년 전쯤 고딩때 봤는데


회절등 광학적 효과를 써서 머신러닝 모델을 물리적으로 구현한거

걍 빛을 띡 하고 쏘면 바로 결과가 튀어나오는거라 추론에 있어서는 연산시간, 에너지 효율 모두 압도적임

다만 파라미터를 못바꾸기 때문에 학습은 불가능함.


엄청난 아이디어라고 했는데 생각보다 이 분야가(optical machine learning?) 주목을 못받는것 같다고 생각 했었음


https://arxiv.org/abs/2302.10360?fbclid=IwAR3HYUF6PT4vqUjer5ltelzGAiVC4neWZBLgLA7omnYOxSDcNpYlS3MSwFE


Optical TransformersThe rapidly increasing size of deep-learning models has caused renewed and growing interest in alternatives to digital computers to dramatically reduce the energy cost of running state-of-the-art neural networks. Optical matrix-vector multipliers are best suited to performing computations with very large operands, which suggests that large Transformer models could be a good target for optical computing. To test this idea, we performed small-scale optical experiments with a prototype accelerator to demonstrate that Transformer operations can run on optical hardware despite noise and errors. Using simulations, validated by our experiments, we then explored the energy efficiency of optical implementations of Transformers and identified scaling laws for model performance with respect to optical energy usage. We found that the optical energy per multiply-accumulate (MAC) scales as $\frac{1}{d}$ where $d$ is the Transformer width, an asymptotic advantage over digital systems. We conclude that with well-engineered, large-scale optical hardware, it may be possible to achieve a $100 \times$ energy-efficiency advantage for running some of the largest current Transformer models, and that if both the models and the optical hardware are scaled to the quadrillion-parameter regime, optical computers could have a $>8,000\times$ energy-efficiency advantage over state-of-the-art digital-electronic processors that achieve 300 fJ/MAC. We analyzed how these results motivate and inform the construction of future optical accelerators along with optics-amenable deep-learning approaches. With assumptions about future improvements to electronics and Transformer quantization techniques (5$\times$ cheaper memory access, double the digital--analog conversion efficiency, and 4-bit precision), we estimated that optical computers' advantage against current 300-fJ/MAC digital processors could grow to $>100,000\times$.arxiv.org


근데 이런 최신 연구도 있네..