Experimenters have had overnight tests confirming they have OPEN SOURCE DeepSeek R1 running at 200 tokens per second on a NON-INTERNET connected Raspberry Pi.
How. I though u needed huge amounts of vram on exorbitantly prices GPUs to run LLM with decent capacity? Are the just running a really small model or is it hyper parametrised? Or is the “thinking” process just that effective u can make up for a weak LLM?
Even though it is the smallest of the distilled models that model still outperforms GPT 4o and Claude Sonnet 3.5.
The 7B parameter models crush the older models on performance benchmarks. The 14 billion parameter model is very competitive with OpenAI o1 mini in many metrics.
How. I though u needed huge amounts of vram on exorbitantly prices GPUs to run LLM with decent capacity? Are the just running a really small model or is it hyper parametrised? Or is the “thinking” process just that effective u can make up for a weak LLM?
Yea sounds like it’s their smallest model