Wolfam alpha actually has an LLM api so your local models can call its factual database for information when doing calculations through tool calling. I thought you might find that cool. Its a shame there is no open alternative to WA they know their dataset is one of a kind and worth its weight in gold. Maybe ond day a hero will leak it 🤪

Models running on gguf should all work with your gpu assuming its set up correctly and properly loaded into the vram. It shouldnt matter if its qwen or mistral or gemma or llama or llava or stable diffusion. Maybe the engine you are using isnt properly configured to use your arc card so its all just running on your regular ram which limits things? Idk.

Intel arc gpu might work with kobold and vulcan without any extra technical setup. Its not as deep in the rabbit hole as you may think, a lot of work was put in to making one click executables with nice guis that the average person can work with…

Models

Find a bartowlski made quantized gguf of the model you want to use. Q4_km is recommended average quant to try first. Try to make sure it all can fit within your card size wise for speed. Shouldnt be a big problem for you with 20gb vram to play with. Hugging face gives the size in gb next to each quant.

Start small with like high quant of qwen 3 8b. Then a gemma 12b, then work your way up to a medium quant of deephermes 24b.

Thinking models are better at math and logical problem solving. But you need to know how to communicate and work with llms to get good results no matter what. Ask it to break down a problem you already solved and test it for comprehension.

kobold engine

Download kobold.cpp, execute it like a regular program and adjust settings in graphical interface that pops up. Or make a startup script with flags.

For input processing library, see if Vulcan processing works with Intel arc. Make sure flash attention is enabled too. Offload all layers of the model I make note of exactly how many layers each model has during startup and specify it but it should figure it out smartly even if not.

As an offgrid person with an actual electrical engineering degree who built my system ground up, visiting the diy solar fourms is a trip

I think offgriders belong in the same crazy camp like healing crystals chicks, and antivaxxers.

Its funny, I feel the same way about suburbanites and generally neurotypicals who speedrun a college debt right out of highschool for a career path that became over saturated with competition a decade before they applied. Then legally binding themselves to the first fuck buddy to provide emotional support/external validation, poping out two kids, further endebting themselves with unending rent/mortgage payments and using the financial + parental responsibility as an excuse to work a 9-5 for the rest of their lives. I can’t imagine having a life slaved to work with so little to look forward to besides vacations twice a year, watching TV, mowing the grass, bitching about HOA, and buying another car/empty status symbol. All before the age of 25.

It takes a special kind of crazy or stupid to blindly follow socital status quo of wanting the slop of comfort, convinence, and status. So easily convinced into racking themselves with lifelong debt equating to indentured slavery while giving into your hormonal monkey instincts for creating social bonding family structures in this political/economic climate.

You can use discord in your web browser with some privacy addons like fingerprint and user gent spoofing to help restrict how much gets leaked to discord. If you install it as an app that runs in background you better believe they’re collecting more data and metrics.

Any device someone ask my help with figuring out. Its rarely the appliance that pisses me off and more the blatant learned helplessness and fundimental inability for fellow adults to rub two braincells together on figuring out a new thing or to troubleshoot a simple problem. A lifetime of being the techie fixer bitch slave constantly delegated the responsibility of figuring out everyones crap for them has left me jaded to the average persons mental capacity and basic logical application abilities.

For all the verbal fellatio Office Space receives I was expecting it to be a god-like ultimate peak of human culture type deal but in reality it was a mid movie humor and plot wise. Its not bad but its very catery to a specific audience I wasn’t part of. I can see it being one of the first and few relatable films for white collar cubicle boglins at the turn of the century which feels like pretty much the sole reason of why I have to see it occasionally referenced 25 years later.

I would receommend you read over the work of the person who finetuned a mistral model on many us army field guides to understand what fine tuning on a lot of books to bake in knowledge looks like.

If you are a newbie just learning how this technology works I would suggest trying to get RAG working with a small model and one or two books converted to a big text file just to see how it works. Because its cheap/free t9 just do some tool calling and fill up a models context.

Once you have a little more experience and if you are financially well off to the point 1-2 thousand dollars to train a model is who-cares whatever play money to you then go for finetuning.

It is indeed possible! The nerd speak for what you want to do is ‘finetune training with a dataset’ the dataset being your books. Its a non-trivial task that takes setup and money to pay a training provider to use their compute. There are no gaurentees it will come out the way you want on first bake either.

A soft version of this thats the big talk right now is RAG which is essentially a way for your llm to call and reference an external dataset to recall information into its active context. Its a useful tool worth looking into much easier and cheaper than model training but while your model can recall information with RAG it won’t really be able to build an internal understanding of that information within its abstraction space. Like being able to recall a piece of information vs internally understanding the concepts its trying to convey. RAG is for wrote memorization, training is for deeper abstraction space mapping

There are some pretty close physical analogs that are fun to think about. You cant move a black hole by exerting physical force on it in the normal way so practically infinite gravity wells are like a immovable “object”, though if you’re sufficently nerdy enough you can cook some fun ways to harness its gravitational rotation into a kind of engine, or throw another black hole at it to create a big explosion and some gravitational waves which are like a kind of unstoppable force moving at the speed of light.

If you were running amd GPU theres some versions of llama.cpp engine you can compile with rocm compat. If your ever tempted to run a huge model with partial offloaded CPU/ram inferencing you can set the program to run with highest program niceness priority which believe it or not pushes up the token speed slightly

deleted by creator

You’re welcome. Also, whats your gpu and are you using cublas (nvidia) or vulcan(universal amd+nvidia) or something else for gpu postprocessing?

Enable flash attention if you havent already

True! Most browsers don’t have native gemini protocol support. However a web proxy like the ones I shared allow you to get gemini support no matter the web browser. Gemtext is a simplified version of markdown which means its not too hard to convert from gemtext to html/webpage. So, by scraping information from bloated websites, formatting it into the simple gemtext format markdown, then mirroring it back as a simple web/html page, it works together nicely to re-render bloated sites on simple devices using gemini as a formatting medium technology. You don’t really need to understand gemini protocol to use newswaffle + portal.mozz.us proxy in your regular web browser

Hemp fiber is stronger than steel tensile strength wise, a bunch of it in thick layers is like a proto-kevlar

Its called that because you have a new appreciation for your life after coming back from that alive… “Thank God I didn’t drop to my death!”

“Hey, how’s it going”

stares blankly at you like a deer looking at headlights for 15.50 seconds with an uncomfortable silence.

“Good.”

walks away

Ill have you know I run a dual side business of selling ball-bouncing-in-polygon software as NFTs as well as counting the r-'s in various spellings of strawberry for the private defense sector…

during the time I was born TVs were small square boxes powered by glass tubes and turny knobs. I want to say 480p but tbh if you were using a junky 10 inch display at the turn of the century on satallite it was closer to like 240p. The jump from square 480p to widescreen 720/1080 was an actual graphical revolution for most people in a very big way, especially for watching movies that were shot in wide. In terms of games 1080p is both where 16:9 took off and the point where realistic looking graphics meet acceptable resolution for like skin pours and godrays shit like that. GTA5, TLOU and RDR are the examples that come to mind from the AAA 1080p era and their original states still probably hold up today.

When the 4k stuff finally came around and it was advertised as the next revolution I was excited man. However compared to going from 480 to 1080 it wasn’t a huge change tbh. It seems once you’re already rendering skin detail and individual blades of grass, or simulating atmospheric condition godrays, there isn’t much more that can be drastically improved just by throwing a billion more polygons at a mesh and upscaling textures. The compute power and storage space required to get these minimal detail gains also starts escalating hard. Its such bullshit that modern AAA games are like 80gb minimum with half of that probably being 4k textures.

I will say that im like the opposite of a graphics snob and slightly proud of it so my opinions on 4k and stuff are biased. Im happy with 1080p as a compromise between graphical quality and compute/disk space required. Ive never played a 1080p at maximum graphics and wanted for more. Im not a competitive esports player, im not a rich tech bro who can but the newest upgraded gpu and 500tb of storage. I don’t need my games to look hyperrealistic. I play games for the fun gameplay and the novel experiences they provide. Some of the best games I’ve ever played look like shit and can be played on a potato. Most of the games I found boring were AAA beautiful open worlds that were as wide and pretty as an ocean but gameplay wise it was as deep as a dried up puddle. I hopped off the graphics train a very long time ago, so take my cloud yelling with a grain of salt.

“I use Arch bt-”

“ITS SHiTE!”

“…excuse me?”

" YOUR BLOODY ROLLING RELEASE DISTRO IS FUCKING RAW. HOW MANY TIMES HAVE YOU RECOOKED IT AFTER A DEPENDENCY PACKAGE BROKE?"

“B-bhut chef… Its a rolling release bleeding distro that expects users to compile with the help of a wik-”

“I ASKED HOW MANY TIMES YOU HAD TO RECOMPILE IT THIS YEAR YOU FUCKING DONKEY”

“5 times sir.”

“FIVE FUCKING TIMES??? JESUS CHRIST DID I ASK FOR CONSTANT MAINTENANCE WITH A SIDE OF COMPUTER PROGRAMS IN BETWEEN? IF I WANTED A RAW OPERATING SYSTEM I WOULD HAVE BECOME A FLAGSMAN INSTEAD OF A CHEF AND ASKED FOR A DISH OF “GENTOO”. COOK ME A REAL OPERATING SYSTEM.”