• 0 Posts
  • 14 Comments
Joined 4 months ago
cake
Cake day: June 30th, 2025

help-circle
  • NotANumber@lemmy.dbzer0.comtoScience Memes@mander.xyzSquiggly Boie
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    3 days ago

    Yes genetic algorithms are something different. Though they are used sometimes in training or architecting NNs, but not at the scale of modern LLMs.

    Fyi you can have all or nothing outputs from a perceptron or other network. It all depends on the activation function. Most LLMs don’t use that kind of activation function, but it is possible. Have you heard of bitnet? They use only one of three states for the neuron output in an LLM. It’s interesting stuff.





  • NotANumber@lemmy.dbzer0.comtoScience Memes@mander.xyzSquiggly Boie
    link
    fedilink
    English
    arrow-up
    15
    ·
    edit-2
    3 days ago

    To be more specific this is an MLP (Multi-Layer Perceptron). Neural Network is a catch all term that includes other things such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Diffusion models and of course Transformers.

    What you are arguing online is some variant of a Generative Pre-trained Transformer, which do have MLP or MoE layers but that’s only one part of what they are. They also have multi-headed attention mechanisms and embedding + unembedding vectors.

    I know all this and wouldn’t call myself a machine learning expert. I just use the things. Though I did once train a simple MLP like the one in the picture. I think it’s quite bad calling yourself a machine learning expert and not knowing all of this stuff and more.







  • That’s another one I don’t understand. In my country at least when I grew up (born 2001) most kids didn’t get chickenpox vaccines. I didn’t have one and actually caught the virus. I think I even had a scar from it. I know someone about 4 years younger than me who also is scarred from it. Not sure if they started giving it out now. I certainly hope so.