KoboldAI AMD ROCm running and benchmarking - Does 7950X3d extra CPU cores help with CPU inference?

5 months ago
16

In my testing, even with OpenBLAS for GGML 13B Hypermantis. it didn't seem to hit the Ryzen 9 7950X3D (set to 'Prefer cache' in BIOS) that much at 43% usage only (in contrast to 100% CPU usage for my Ryzen 5 7600X), ROCm with New Dawn fades did hit it at 55-62%. but you will probably still be limited by your GPU since dual-channel RAM is just too fully saturated by the CPU. Trident Z5 Neo 6000 MhZ overclocked RAM.

Mirrored on Odysee
Bluesky social(Twitter FOSS):https://bsky.app/profile/newjersey14.bsky.social
Revolt server(Discord equal):https://rvlt.gg/8h3PzaWt
Dlive:https://dlive.io?ref=asleepinglaffey

Crypto donation addresses if for some reason you want to back me up without ads (It's better with that 9 USD or even 60 USD compared to Youtube constantly pushing Youtube premium on you for 9 USD monthly while intentionally causing a war with Ublock Origin and Freetube,or trying to view my videos constantly to support me with trashy ad monetization rates, trust me!)

Bitcoin:bc1qzl8p4pn8ujvpyreedv0a69k0ks86du3lhkdrynnjp6ypsse9wjtspnm3du
ETH/Dai Stablecoin:0x922480ca4a9bcbcbd49668d3c871f57d9e747928
TRX/USDT(Probably does not work with Dogecoin or USDC):TLdGipgMC31cTcyQXfQbEHRbPRhmTssCYi
Monero/XMR(Liquidity issues as much as i like it):43rnZ8a5nFi8SGf6kePRSPKyvvN6cSxyJeKivmWca
Cun5wA1feZ4Gr88EdoHgcGfcNMj4q4hpEKBwYfjt

Loading 1 comment...