What could change the answer
- Long context and KV cache
- Batch size or concurrent users
- Runtime overhead and driver support
- CPU/RAM offload
- Image encoders and auxiliary models
MODEL × HARDWARECan RTX PRO 6000 Blackwell 96GB
Can RTX PRO 6000 Blackwell 96GB
run Mistral 7B v0.3?
Yes, with headroom.
Q4_K_M MEMORY VERDICTCOMFORTABLE
6.6 GB estimated requirement vs 96.0 GB usable-memory assumption.
Model weights4.4 GB
Runtime reserveIncluded
Hardware stackCUDA
Speed claimNot estimated