minus-squarej4k3@lemmy.worldtoLocalLLaMA@sh.itjust.works•Anyone found "optimal" settings for llama.cpp partial offload?linkfedilinkEnglisharrow-up1·8 days agoI just use Oobabooga. I wrote a script to loop and display the memory remaining on the GPU to optimize the split for each model, but that was it. linkfedilink
minus-squarej4k3@lemmy.worldtoTechnology@lemmy.world•Russia Issues Ominous Warning About Undersea Internet CableslinkfedilinkEnglisharrow-up0·4 months agoMusk made a deal with Putin to extort people into using Starlink. linkfedilink
I just use Oobabooga. I wrote a script to loop and display the memory remaining on the GPU to optimize the split for each model, but that was it.