The 5090 is basically useless for AI dev/testing because it only has 32GB. Mind as well get an array of 3090s.

The AI Max is slower and finicky, but it will run things you’d normally need an A100 the price of a car to run.

But that aside, there are tons of workstations apps gated by nothing but VRAM capacity that this will blow open.

There are simple tests to out LLMs, mostly things that will trip up the tokenizers or sampling algorithms (with character counting being the most famous example). I know people hate captchas, but it’s a small price to pay.

Also, while no one really wants to hear this, locally hosted “automod” LLMs could help seek out spam too. Or maybe even a Kobold Hoard type “swarm.”