Happy 2024! I wrote a retrospective on early AI research to finish off 2023: srhm.ca/revisiting_t... — writing it brought me to archive.org/details/bits... which is a treasure trove of early documents on AI research at MIT. Worth checking!
Tim Dettmers of LLM.int8() fame: "MoE layers [like Mixtral's] are very different. You can also quantize them to 1 bit without any problem." Hoping this means Q2_K will be more competitive! It'd be a blessing for my 32GB system. twitter.com/Tim_Dettmers...
