Personal AI Development Environment Built with RTX 5090 + WSL2 — A Practical Setup Fully Utilizing 32GB GPU

Why RTX 5090 + WSL2? The 32GB VRAM of the RTX 5090 is a practical choice for local inference of large LLM models. Compared to the RTX 4090 (24GB), VRAM capacity is improved by 33%, increasing the room for model size expansion. With vLLM's batch processing, parallel inference can fully utilize the 32GB VRAM. CUDA 12.8 is the latest toolkit, offering full compatibility with PyTorch and triton. In a WSL2 environment, the Windows host's GPU driver directly provides the GPU, allowing users to benefit from Linux toolchains (vLLM, TensorRT, llama.cpp, etc.). Overall System Configuration vLLM Server (Resident Process) systemctl --user enable vllm.service systemctl --user start vllm.service Model: Infers models like Nemotron 9B in FP8. Controls usage with gpu-memory-utilization . TensorRT Shogi AI Optimizes FP8 quantized models with TensorRT to achieve high-speed inference. Streamlit App Provides UI for displaying LLM inference results, search forms, and more. GPU Sharing in Practice The vLLM s