Qualcomm CEO Says Smartphones Will Run 100B AI Models by 2028

Qualcomm CEO Cristiano Amon believes smartphones will be able to continuously run AI models with 100 billion parameters by 2028, despite major challenges involving memory capacity, cost, and heat.

Speaking with Fireside Alpha, Amon said AI companies are already developing smartphones capable of running 100B large language models. He did not name the companies involved.

Memory remains one of the biggest obstacles to running such large AI models on smartphones. With the current DRAM supply situation, smartphones are unlikely to move beyond the 16GB RAM range in the near future. Amon, however, expects manufacturers to equip future devices with significantly more memory, even as DRAM remains expensive and supply stays tight.

Running 100B AI Models Would Require Much More RAM

Current hardware makes running a conventional 100B model locally extremely difficult.

A 100-billion-parameter model quantized to 4-bit would require about 50GB of memory just to store its parameters. Smartphones would also need additional memory for the operating system, applications, and the model’s context cache.

Even reducing the model to 2-bit quantization would require around 25GB of memory for the model alone. That figure remains higher than the memory capacity of most current smartphones.

The problem comes as smartphone manufacturers have already started moving away from higher RAM configurations because of rising DRAM costs and supply constraints. Just a year ago, some Android smartphones offered as much as 24GB of LPDDR5X RAM.

Memory capacity is also not the only challenge. Continuously running such a large model would generate substantial heat. Sustained AI workloads could cause smartphones to reach their thermal limits and throttle performance.

Quantization and New Chip Packaging Could Help

Apple and Qualcomm have started addressing some hardware limitations through changes in chipset packaging with platforms such as the A20 Pro and Snapdragon 8 Elite Extreme Gen 6.

However, these changes do not remove the underlying memory bottleneck. More aggressive approaches would still be necessary to run 100B models on smartphones.

One option is aggressive model quantization. A 1-bit version could significantly reduce memory requirements and potentially allow a large model to fit within smartphone RAM.

The downside is model quality. Performance can degrade significantly when quantization goes below 4-bit, particularly for reasoning workloads.

Sparse MoE Models Could Offer Another Solution

Another possible approach is using sparse Mixture-of-Experts, or MoE, models with expert offloading.

MoE models activate only a small portion of their experts for each token. Active experts could remain in RAM while the rest of the model stays on the smartphone’s flash storage.

This could reduce the amount of DRAM required at any given time.

Apple has previously researched running large language models using flash storage. However, smartphone UFS and NVMe storage remain slower than DRAM, which can reduce inference speeds when model data has to move between storage and memory.

Smaller Distilled Models May Be More Practical

Another option would be to use a smaller distilled model with around 20 billion to 30 billion parameters.

Such a model could potentially approach the quality of a 100B model while requiring significantly fewer hardware resources. This could be a more practical strategy for smartphone manufacturers.

For now, Amon has not provided a detailed roadmap explaining how smartphones will overcome the memory, storage, power, and thermal limitations required to continuously run 100B models by 2028.

Still, technologies such as sparse MoE architectures, flash-based model storage, aggressive quantization, and smaller distilled models could provide possible paths toward running increasingly capable AI models directly on smartphones.

The post Qualcomm CEO Says Smartphones Will Run 100B AI Models by 2028 appeared first on ProPakistani.

Exit mobile version