Locally? Arcee 14B and the 14B Deepseek distill are currently the best models that fill fit.
I’d recommend hosting them with TabbyAPI instead of ollama, as they will be much faster and more VRAM efficient. But this is more fuss.
Honestly, I would just try free APIs like Gemini, Groq, and such through open web ui, or use really cheap APIs like openrouter. Newer 14B models are okay, but they’re definitely lacking that “encyclopedic intelligence” larger models have.
Uh, title is a bit clickbaity, editorialized. Amazon isn’t changing books yet, they are planning to make it possible for publishers to do so, I think, and also recoking ownership. And the video is not great either.
Depends for sure. I usually try the 32B first, but give really “hard” queries to some API model.
I mean, not with certainty. If the risk of your input leaking is that great, you can just host your own VM with the 32B to be more certain.
There are plenty of free LLM APIs you can use with something like Open Web UI, on any machine. I still use them myself.
You can’t practically self-host Deepseek R1.
Look, I use the 32B distil on my 3090 every day, but it is not the same thing as full R1. And people need to stop conflating the two.
And (theoretically) API usage through one of many R1 providers is private.
Not full R1, which is developed differently than any of the distillations.
It’s honestly not that big a deal, as it’s not like knowing anything about how it was trained (beyond the config) would help you modify it. It’s still highly modifiable. It’s not like anyone can afford to replicate it.
It would be nice to publish the hyperparameters for research purposes, but… shrug.
I think a subset of the exact training data/hyperparameters would help with quantization-aware-training, maybe, but that’s all I got.
Well you can merge it with the original model, to any degree, to get any sliding scale of “bias” you want.
Practically, though, I guess that’s not super practical, as very few have the hardware or cash to deploy a custom full R1 themselves.
In the 32B range? I think we have plenty of uncensored thinking models there, maybe try fusion 32B.
I’m not an expert though, as models trained from base Qwen have been sufficient for that, for me.
That’s the great thing about open models. Censorship? Once identified, all it takes is one person and a bit of cash to get rid of it, though it seems Perplexity did a particularly good job (unlike some “abliterated” models that are pretty dumbed down).
Daily plug for Cromite, which is explicity built for anti-fingerprinting (through not just blocking, but spoofing and stripping systems out) and de-Googling:
Precisely.
And it’s a “hard” sci fi universe rooted in theoretically possible physics. No FTL or causality violations.
Would recommend Orion’s Arm for “theoretical” but fiction takes on structures like this:
https://www.orionsarm.com/eg-article/5067d430e6021
That article is not comprehensive either, their universe is quite expansive.
Two of my favorites may be “W-brains,” computing structures with very carefully arranged wormhole pairs serving as data buses to overcome the latency of communicating at such scale, and “neural stars,” another take which is a computational structure inside a neutron star sized volume/mass (again, to overcome latency issues).
There are much smaller megastructures too, depending on where in the timeline you are looking.
I mean, you are right, and way more people should be using openSUSE :P
I will say Arch-derived distros are a good experience if you want to learn how the terminal and other systems work. They’re engineered to be configurable; the documentation is great. But if you just want to use your computer without opening too many hoods, it’s fundamentally not an optimal system.
Another thing is that many people just want their new laptop to work, and for it to game on linux. Sometimes it does not just work. If you start pulling in fixes and packages that are not supported on your distro, you can screw up any distro very quickly (and this includes the AUR, unofficial Fedora repos and such). If the community packages these, stages them, tests them against all official packages, and they work out-of-the-box… that’s one less hazard.
Yeah, it uses only ram swap by default. If you aren’t going over a ton, it shouldn’t matter.
I just have weird workloads that spike memory usage a ton for short times, so I have to go out to my nvme.
Exactly, and you can get mesa git and some other “fix” packages natively with community integration if you find you need them, without veering off track all by yourself.
I skipped nobara for that very reason.
And I was more familiar with Arch anyway.
Actually I had this one!
Something about their swap config makes it very fragile unless you use RAM swap as enabled by default, and I kept having this when I disabled it for reasons. It was much better once I re enabled it, though occasionally I still have severe issues going way, way, over my RAM pool.
I don’t mention that much because swapping to like 64GB on a 32GB system seems like an uncommon use case.
Mostly this ^.
There’s just not really demand for C++ in the kernel; that’s not the case with Rust.