苹果神经引擎

In recent years, with the AI compute improvements of mobile chips such as Qualcomm Snapdragon and Apple Neural Engine, and the maturation of inference frameworks such as Llama.cpp and MLC-LLM, models under 7B parameters can already achieve usable generation speeds on flagship phones.

An NPU (neural processing unit) is a processor designed specifically for neural network inference and training tasks, and its architecture typically includes large multiply-accumulate arrays (MAC arrays), on-chip caches, and dedicated data scheduling units.

主流NPU被广泛应用于手机、自动驾驶和边缘服务器中，如华为昇腾、苹果神经引擎和高通的Hexagon处理器。