CHTC GPU Lab: LLM Inference Example

A minimal example project for running Large Language Model (LLM) inference on CHTC GPU Lab at UW-Madison.

Why This Project?

CHTC’s documentation is great, but setting up LLM inference involves several gotchas:

• GPU Lab nodes cannot directly access /staging via paths in submit files
• Version conflicts between container PyTorch and pip-installed …