MODEL × QUANTIZATIONMistral Small 24B
Mistral Small 24B
Q5_K_M
More quality with a larger footprint. The estimate includes weight overhead and a basic runtime reserve.
Weights17.8 GB
Estimated total22.0 GB
Bits assumption5.5
HeadroomRecommended
FIT MAP
Candidate hardware
OFFLOAD
RTX 3080 10GB
fast older GPU constrained by 10 GB VRAM
10 GB VRAMInspect →OFFLOADRTX 3060 12GB
budget used GPU with useful 12 GB capacity
12 GB VRAMInspect →OFFLOADRTX 4060 Ti 16GB
efficient 16 GB consumer option
16 GB VRAMInspect →OFFLOADRTX 4070 Ti Super 16GB
faster 16 GB card for mid-size models
16 GB VRAMInspect →OFFLOADRTX 4080 Super 16GB
high throughput but still limited to 16 GB
16 GB VRAMInspect →OFFLOADRTX 5060 Ti 16GB
current mid-range 16 GB option
16 GB VRAMInspect →OFFLOADRTX 5070 Ti 16GB
current performance card with 16 GB ceiling
16 GB VRAMInspect →OFFLOADRTX 5080 16GB
fast compute with a 16 GB memory ceiling
16 GB VRAMInspect →FITSRTX 3090 24GB
used-market 24 GB local AI favorite
24 GB VRAMInspect →FITSRTX 4090 24GB
strong 24 GB inference and image generation
24 GB VRAMInspect →COMFORTABLERTX 5090 32GB
fast 32 GB consumer flagship
32 GB VRAMInspect →COMFORTABLERTX A6000 48GB
workstation capacity for larger quantized models
48 GB VRAMInspect →COMFORTABLEL40S 48GB
data-center inference and image generation
48 GB VRAMInspect →COMFORTABLERTX PRO 6000 Blackwell 96GB
large-memory professional workstation GPU
96 GB VRAMInspect →