Site Logo
Alastair Drong

    I doubled my AI server's RAM and it got slower (Homelab - Ep. 0x006)

    Table of Contents

    Description

    I double the memory in my local LLM server so it can run an 80 billion parameter model on a machine with an 8GB graphics card. Every stick of RAM tested fine, and the server still refused to POST.

    #Homelab #LocalLLM #llamacpp #SelfHosted #AI

    Testing Results: https://github.com/dev-dull/local_llm_tests/tree/main/results Job hunt tooling: https://github.com/dev-dull/job-search-automations/tree/main/job-board