I think we’re at a point were the hardware right now does not fit with the algorithm being used. Since they take so much power due to our computers being digital. Having a transistor only capable of holding 1 state (0V or 5V usually) is inefficient. The heat add up as you multiply especially with LLMs. There seems to be a potential for analog where a transistor acts more on a range 0 - 5v. Which in theory could store more information or directly represent what LLM runs on (floating point). For more context 1 float tends to be 32bits.
1 bit is 1 transistor so 1 float = 32 transistor. While an analog transistor could be 1 float = 1 analog transistor.
Do you have a link to any research on a push to analog transistors and their properties? I have been reading up on transistors (and vacuum tubes) but haven’t seen any discussion on this.
Also much lower voltages are typical in modern transistors, from 1-1.5v.
I think we’re at a point were the hardware right now does not fit with the algorithm being used. Since they take so much power due to our computers being digital. Having a transistor only capable of holding 1 state (0V or 5V usually) is inefficient. The heat add up as you multiply especially with LLMs. There seems to be a potential for analog where a transistor acts more on a range 0 - 5v. Which in theory could store more information or directly represent what LLM runs on (floating point). For more context 1 float tends to be 32bits. 1 bit is 1 transistor so 1 float = 32 transistor. While an analog transistor could be 1 float = 1 analog transistor.
Do you have a link to any research on a push to analog transistors and their properties? I have been reading up on transistors (and vacuum tubes) but haven’t seen any discussion on this.
Also much lower voltages are typical in modern transistors, from 1-1.5v.
My mistake, I made it look like that’s a fact I’ll edit that as my opinion. Though here is one I can find.
https://www.nature.com/articles/s43588-024-00753-x
Article also https://spectrum.ieee.org/analog-ai-2669898661