KTransformers

KTransformers is a high-performance inference optimization framework for large language models, supporting hybrid CPU–GPU inference to enable deployment and operation of large-scale models on consumer-grade or resource-constrained hardware. Core features include heterogeneous computing scheduling, quantized inference support, and compatibility with multiple mainstream model architectures, aiming to lower hardware barriers for local large model deployment and improve inference efficiency.