I did nothing and I’m all out of ideas!

  • 4 Posts
  • 263 Comments
Joined 2 years ago
cake
Cake day: June 11th, 2023

help-circle

  • I’ve never used oobabooga but if you use llama.cpp directly you can specify the number of layers that you want to run on the GPU with the -ngl flag, followed by the number.

    So, as an example, a command (on linux) from the directory you have the binary, to run its server would look something like: ./llama-server -m "/path/to/model.gguf" -ngl 10

    Another important flag that could interest you is -c for the context size.

    This will put 10 layers of the model on the GPU, the rest will be on RAM for the CPU.

    I would be surprised if you can’t just connect to the llama.cpp server or just set text-generation-webui to do the same with some setting.

    At worst you can consider using ollama, which is a llama.cpp wrapper.

    But probably you would want to invest the time to understand how to use llama.cpp directly and put a UI in front of it, Sillytavern is a good one for many usecases, OpenWebUI can be another but - in my experience - it tends to have more half baked features and the development jumps around a lot.

    As a more general answer, no, the safetensor format doesn’t directly support quantization, as far as I know






  • MechanizetoProton @lemmy.worldIs Proton Down?
    link
    fedilink
    English
    arrow-up
    9
    ·
    edit-2
    1 month ago

    Their systems currently report that everything’s fine, which - to be fair - could be a misreporting and change at any moment
    Anecdotally all their landing sites load fine for me

    Tackling it from the other side: could it be a problem with your DNS?
    Did you try with another network? Like wifi or mobile data

    EDIT: Formatting

    EDIT: Tried again and now the email service seems to not be loading

    EDIT 2: It is being reported on other sites too, but currently there’s nothing official I could find, not even on their Mastodon or Twitter various accounts

    EDIT 3: On reddit the volunteer mod alex_herrero wrote an hour ago that “The team is aware and [the] status page will be updated shortly”.




  • That’s the bad thing about social media. If no one was doing it before, someone is now!

    Jokes aside it’s possible, but with the current LLMs I don’t think there’s really a need for something like that.

    Malicious actors usually try to spend the least amount of effort possibile for generalized attacks, because you end up having to often restart when found out.

    So they probably just feed an LLM with some examples to get the tone right and prompt it in a way that suits their uses.

    You can generate thousands of posts while Lemmy hasn’t even started to reply to one.

    If you instead want to know if anyone is taking all the comments on lemmy to feed to some model training… Yeah, of course they are. Federation makes it incredibly easy to do.



  • Hangs, reboots or does it turn off? Depending on that it could be a plethora of things.

    Do you get any error prompts? Any red alerts when restarting?

    Did you check that the monitor works steadily on another system/OS?

    Did you try another DE, even on a live usb?

    Is the HDD/SSD healthy?

    Are all the fans working? Is it the thermal protection?

    Is the PSU healthy and or the power connection damaged?

    Does your system have a centralized logging like journalctl or can you reach the single log files to check and add more information?

    It could literally be anything, even aliens.




  • I assumed it was a shitpost, instead it is a real tweet. What a time to be alive.

    Jokes aside the only real reason I can fathom for the collectibles company to call their mother is because they had used it as the contact number in the registry. I would be surprised if this was some kind of intimidation tactic instead of just miscommunication - in the sense they probably just wanted to legally intimidate the itch’s owner not their immediate family. They are not 2K /s.


  • AFAIK it is still a tuning of llama 3[.1], the new Base models will come with the release of 4 and the “Training Data” section of both the model cards is basically a copy paste.

    Honestly I didn’t even consider the fact they would not be giving Base models anymore before reading this post and, even now, I don’t think this is the case. I went to search the announcements posts to see if there was something that could make me think about it being a possibility, but nothing came out.

    It is true that they released Base models with 3.2, but there they had added a new projection layer on top of that, so the starting point was actually different. And 3.1 did supersede 3…

    So I went and checked the 3.3 hardware section and compare it with the 3 one, the 3.1 one and the 3.2 one.

    3 3.1 3.2 3.3
    7.7M GPU hours 39.3M GPU hours 2.02M GPU hours 39.3M GPU hours

    So yeah, I’m pretty sure the base of 3.3 is just 3.1 and they just renamed the model in the card and added the functional differences. The instruct and base versions of the models have the same numbers in the HW section, I’ll link them at the end just because.

    All these words to say: I’ve no real proof, but I will be quite surprised if they will not release the Base version of 4.

    Mark Zuckerberg on threads

    Link to post on threads
    zuck a day ago
    Last big AI update of the year:
    •⁠ ⁠Meta AI now has nearly 600M monthly actives
    •⁠ ⁠Releasing Llama 3.3 70B text model that performs similarly to our 405B
    •⁠ ⁠Building 2GW+ data center to train future Llama models
    Next stop: Llama 4. Let’s go! 🚀

    Meta for Developers

    Link to post on facebook
    Today we’re releasing Llama 3.3 70B which delivers similar performance to Llama 3.1 405B allowing developers to achieve greater quality and performance on text-based applications at a lower price point.
    Download from Meta: –

    Small note: I did delete my previous post because I had messed up the links, so I had to recheck them, whoops




  • It’s probably a problem with the UEFI, the windows info got overwritten, and you can probably fix this with efibootmgr

    It happened to me too, but unfortunately it was some years ago and I’m not at home to find the related notes that I took. I remember there was a windows utility to rewrite the boot loader. But probably in your case the boot partition is still okay, just the UEFI entry got overwritten and you just have to add it back manually.

    Check the troubleshooting section of the wiki page to have a tip on the windows booting location


  • The whole information is in this paragraph:

    The central bank’s governing council gave unanimous support to October’s decision to cut rates by 0.25 percentage points to 3.25 per cent, arguing that “the disinflationary trend was getting stronger” and that it was important to avoid “harming the real economy by more than was necessary”.

    The rest is basically padding and speculation, which can be summarized with:

    “On december there could be a cut up to 50 points, or none, no one knows because we have no idea what is going to happen and we are going in blind. The growth was higher than expected (.4 instead of .2) but the inflation was too (2, instead of 1.7).”

    What a time to be alive. I’m not worried at all about the future.