especially in the Apple space

Offtopic, we are discussing linux laptops.

With Framework its 5 screws

Aw gee, then Framework wins! With Focus IR16 its nine phillips screws… 😩

Either you’re holding onto the case for many many years continuously upgrading, which I question why an old case is so valuable to not just replace it too when you replace the entire mainboard, or you’re not, which makes me question the entire “upgradeable” concept.

Either way it doesn’t really make sense. You can easily upgrade the parts that don’t require a whole new mainboard, already, on literally any other laptop (hard drives, RAM, network card, battery).

It’s neat that you can customize the ports on it and swap them out, which is the only real difference from any other laptop, but to me it seems like a gimmick and doesn’t justify framework premium price when there are plenty of laptops out there with the ports that I need already

Also upgrading CPUs that are so close in generation and only a marginal performance difference like you have done is atypical and does not reflect the purchasing behavior of 99% of users. There’s no real perceivable performance difference between those two CPUs for what most people actually use a laptop to do: web browse.

reusing 90% of the parts

Oops you need a whole new mainboard anyway to upgrade the CPU… oops you need new DDR5 RAM for the new CPU… oops these framework parts cost a premium at about the same cost as a new laptop anyay. Congrats, you now have an upgraded laptop in an old case that’s already gone through wear and tear… at least you kept the SSD that could have been popped into a new laptop as a secondary drive?

If the laptop is old enough to merit CPU upgrade, then its likely already experienced plenty of wear and tear. Also I never said anything about the screen wearing out, I specifically said the case. I gave credit that at least the screen would carry over

in a few years

You’d think the point of repair and upgrade would be for the laptop to exist and be used for more than just a “few years” (otherwise what is the point?) so consider the realistic and more likely case of upgrading it more than a few years from now. Its worth it to keep an old used case, especially when you are paying a premium for framework?

It’s great that it can be repaired easily though and there are internal parts available for purchase, but you can also find internal parts to many laptops available if you look for them, the only real difference is ease of repair

I considered buying framework for my laptop but once I thought about it realistically, it stopped making much sense.

So… all the normal stuff that is normally upgradeable on a normal laptop is upgradeable for framework too? Good point…

The only “replacing the entire guts” you would do is if you replaced the mainboard for a CPU upgrade

That’s exactly my point, yes. Again, the “upgradeability” of a framework laptop is unrealistic at best and a scam at worst. It’s exactly as upgradeable as most laptops unless you’re replacing the whole mainboard which is not very realistic. By that point there is likely enough wear and tear that it makes no sense to keep the case, keyboard, and screen… and with framework premium prices you aren’t saving money on the SSD or RAM (which, no, you likely can’t reuse on a CPU upgrade, most likely you’ll be going DDR4->DDR5). I do give them credit for repairability, which is great, but “upgradeability” specifically is basically a marketing scam and will not make any sense for 99% of users.

I expected the downvotes on my comment because my opinion goes against the framework fanboys, but I hope my suggestion of KFocus IR16 is not discounted because of my opinions on framework. It is truly a great choice for linux compatibility

https://kfocus.org/spec/spec-ir16.html this is an absolute gem, built just for linux. It comes with KUbuntu preinstalled but can be wiped and replaced with any flavor of linux, and all of the hardware and laptop functionality is fully supported by linux.

Framework is way overhyped and even more overpriced. Its “upgradeability” is totally unrealistic at best, scam at worst. Sure you can pop in a new USB port or display output, but that’s about it before you’re replacing the entire guts to upgrade it and keeping just the worn out case and screen…? Gee sounds great… Repairability is a real point for framework though. Can you still not update the BIOS on linux? Its linux support is historically not great but may have improved

It’s the pro driver for workstation use. If you are gaming then you don’t need it. The gaming driver is only open source

I don’t want any proprietary drivers

So then you don’t want any NVIDIA.

The AMD open source Linux driver performs better than their Windows driver. And there is no proprietary AMD Linux driver, the official AMD driver for Linux is open source.

You can overwrite the model by using the same name instead of creating one with a new name if it bothers you. Either way there is no duplication of the llm model file

What I am talking about is when layers are split across GPUs. I guess this is loading the full model into each GPU to parallelize layers and do batching

Can you try setting the num_ctx and num_predict using a Modelfile with ollama? https://github.com/ollama/ollama/blob/main/docs/modelfile.md#parameter

Are you using a tiny model (1.5B-7B parameters)? ollama pulls 4bit quant by default. It looks like vllm does not used quantized models by default so this is likely the difference. Tiny models are impacted more by quantization

I have no problems with changing num_ctx or num_predict

Models are computed sequentially (the output of each layer is the input into the next layer in the sequence) so more GPUs do not offer any kind of performance benefit

Ummm… did you try /set parameter num_ctx # and /set parameter num_predict #? Are you using a model that actually supports the context length that you desire…?

Hawley’s statement called DeepSeek “a data-harvesting, low-cost AI model that sparked international concern and sent American technology stocks plummeting.”

data-harvesting

???

It runs offline… using open-source software that provably does not collect or transmit any data…

It is low-cost and out-competes American technology, though, true

We don’t, I already have a steam deck. Touchpads.