StarFire 是 OPPO 自研的端云协同机器学习平台，该平台把云端能力、算力与终端设备相结合，是安第斯智能云的六大核心能力之一。为了促进 AI 模型的开发进程，提升智能终端的AI能力，StarFire 与 Google Cloud 以及高通合作，首次将 Google Cloud Vertex AI Neural Architecture Search (Vertex AI NAS) 落地到智能手机上。本文将着重探讨在智能终端上开发AI模型的挑战，Vertex AI NAS 对 AI 模型开发的优势，详细说明 OPPO 如何使用 Vertex AI NAS 提升智能终端上 AI 模型的能效。
OPPO adopted this strategy because we believe in the potential of AI capabilities on mobile devices. On one hand, running AI models on end devices can better protect user privacy by keeping user data on mobile hardware, instead of sending them to the cloud. On the other hand, the computing capabilities of mobile chips are rapidly increasing to support more complex AI models. By linking cloud platforms with mobile chips for AI model training, we can leverage cloud computing resources to develop high-performance machine learning models that adapt to different mobile hardware.
In 2022, OPPO started implementing the AI engineering strategy on StarFire, which is our self-developed machine learning platform that merges cloud with end devices and serves, forming one of the six capabilities of AndesBrain. Through StarFire, we’re able to take advantage of various advanced cloud technologies to meet our development needs. To facilitate the AI model development process and enhance AI capabilities on mobile devices, we’ve collaborated with Google Cloud and Qualcomm Technologies to embed the Google Cloud Vertex AI Neural Architecture Search (Vertex AI NAS) on a smartphone for the first time. Let’s explore what we learned.
One major bottleneck of developing AI models on mobile devices is the limited computing capabilities of mobile chips compared to computer chips. Before using Vertex AI NAS, OPPO’s engineers mainly used two methods to develop AI models that can be supported by mobile devices. One is simplifying the neural networks trained on cloud platforms through network pruning or model compressing to make them suitable for mobile chips. The other is adopting lighter neural network architectures built on technologies like depthwise separable convolutions.
These two methods come with three challenges:
The neural architecture search technology was first developed by the Google Brain team in 2017 to create AI trained to optimize the performance of neural networks according to developers’ needs. By automatically discovering and designing the best architecture for a neural network for a specific task, the neural architecture search technology enables developers to more easily achieve better AI model performance.
Vertex AI NAS is currently the only fully-managed neural architecture search service available on a public cloud platform. As OPPO’s machine learning platform StarFire is cloud-based, we can easily connect Vertex AI NAS with our platform to develop AI models. On top of that, we chose to adopt Vertex AI NAS for on-device AI model development because of the following three advantages:
1. Automated neural network design:As mentioned, developing AI models on mobile devices can be labor intensive and time consuming. Because the neural network design is automated through Vertex AI NAS, we can greatly reduce development time and easily adapt an AI model to different mobile chips.
2. Custom reward parameters: Vertex AI NAS supports custom reward parameters, which is rare among the NAS tools. This means that we can freely add the search constraints that we need our AI models to be optimized for. By leveraging this feature, we have added power as a search constraint and successfully lowered the energy consumption of our AI model on mobile devices by 27%.
3. No need to compress AI models for mobile devices: Based on the real-time rewards sent back from the connected mobile chips, Vertex AI NAS can directly design a neural network architecture suitable for mobile devices. The end result can be run on end devices without being further processed, which saves time and effort for AI model adaptation.
Lowering power consumption is key to providing excellent user experience for AI models on mobile devices, particularly the computing intensive models related to multimedia and image processing. If an AI model consumes too much power, mobile devices can overheat and quickly drain their battery life. That is why the primary aim of using Vertex AI NAS for OPPO is to boost energy efficiency of AI processing on mobile devices.
To achieve this goal, we first added power as a custom search constraint to Vertex AI NAS, which only supports latency and memory rewards by default. This way, Vertex AI NAS can search neural networks based on the rewards of power, latency, and memory, letting us reduce power consumption of our AI models while reaching our desired levels of latency and memory consumption.
Then, we connected the StarFire platform with Vertex AI NAS through Cloud Storage. At the same time, StarFire is linked with a smartphone equipped with Qualcomm’s Snapdragon 8 Gen 2 chipset through the SDK provided by Qualcomm. Under this structure, Vertex AI NAS can constantly send the latest neural network architecture via Cloud Storage to StarFire, which then exports the model to the chipset for testing. The test results are sent back to Vertex AI NAS again through StarFire and Cloud Storage, allowing it to conduct the next round of architecture search based on the rewards.
This process was repeated until we achieved our target. In the end, we realized a 27% reduction in power of our AI model and a 40% reduction in computing latency, while maintaining the same level of accuracy rate before the optimization.
Broadening the application range
OPPO 安第斯智能云（AndesBrain）是服务个人、家庭与开发者的泛终端智能云，致力于“让终端更智能”。作为 OPPO 三大核心技术之一，安第斯智能云提供端云协同的数据存储与智能计算服务，是万物互融的“数智大脑”。
原文始发于微信公众号（安第斯智能云）：StarFire的智能终端AI能力进化之路
- 左青龙
- 微信扫一扫
-
- 右白虎
- 微信扫一扫
-
评论