On November 6, the 26th China Patent Awards Ceremony was held in Wuxi. The patent "A User Request Processing Method and Device" submitted by Alibaba Cloud won the China Patent Gold Award. This was the only cloud vendor to win the gold award this year. This patented technology can dynamically sense changes in computing power demand in real time, especially in response to the violent demand tides in the Internet and AI era, and achieves on-demand allocation, on-demand scaling, and safe supply of ultra-large-scale intelligent computing power—it can sustain it during peak times, recover it during troughs, and securely isolate multiple tenants.

The China Patent Award is jointly selected by the State Intellectual Property Office and the World Intellectual Property Organization. It is the highest honor in the field of patents in my country. There are no more than 30 invention patent gold awards in each national selection. It is a national-level authoritative certification of corporate innovation capabilities and patent industrialization value. At present, Alibaba Cloud has deployed more than 10,000 patents around the world, covering more than ten countries and regions, focusing on core technology fields such as cloud computing, artificial intelligence, large models and chips.

Solve three major problems in ultra-large-scale intelligent computing power scheduling in the AI ​​era

Ultra-large-scale intelligent computing power clusters in the AI ​​​​era are faced with violent demand tides: sudden traffic, training peaks, and inference peaks are intertwined and superimposed, constantly testing the limits of computing power elasticity. Between peaks and valleys, the demand for computing power can vary hundreds of times - according to peak configuration, a large number of resources will be idle under normal conditions; according to daily configuration, when the peak comes, there will be risks of insufficient capacity and unstable service. Therefore, the elastic unification of heterogeneous computing power, instantaneous response to dynamic loads, and safe isolation in a multi-tenant environment have become the three core challenges of ultra-large-scale computing power in the AI ​​era.

The core invention of Alibaba Cloud's patent is a new way of processing user requests. Traditional solutions rely on pre-allocating resources and passively waiting for requests, resulting in a long-term mismatch between resource configuration and real load. The method proposed in this patent can dynamically sense demand changes in real time and intelligently match requests with underlying heterogeneous computing power, so that each request can receive a timely and adaptive computing power response.

Specifically, the system senses fluctuations in business load in real time between user requests and massive computing power, and dynamically allocates computing power to where it is most needed. Automatically expands capacity during business peak times and automatically shrinks during business troughs, allowing computing power to be supplied on demand like "water and electricity" and used at any time, using every resource on the cutting edge. The elastic preheating strategy and the improvement of scheduling efficiency also make application startup in seconds and millisecond-level large model inference response a reality. At the same time, the data of each application is processed and stored in independent operating environments, which are isolated from each other and do not interfere with each other. While improving resource utilization efficiency, it also builds an end-to-end protective barrier for data security.

Elastic computing power serves various industries in the AI ​​era more efficiently

Relying on the accumulation of technologies such as elastic computing power scheduling, Alibaba Cloud has continued to incubate a series of products covering general computing, model training and agent operation. Lingjun Zhenwu super node instances, Agent Sandbox, container service AI inference suite AI Serving Stack, distributed cache Fluid, etc. have been launched one after another, and have been widely used in thousands of industries in the AI ​​era.

For example, MiniMax can launch 5,000 sandboxes in 10 seconds based on Alibaba Cloud Agent Sandbox, reducing the total cost of ownership by 25%; Dark Side of the Moon Kimi achieves elastic expansion of tens of thousands of sandboxes per minute, shortening startup time by 50%, and supports stable business response with elastic supply; Step Star combines Agent Sandbox with Alibaba Cloud ACK to run millions of sandboxes stably, compressing the model evaluation cycle from weeks to days. 51job AI Recruitment Assistant accepts 10,000-level pulse requests through Agent Sandbox and hierarchical hibernation, and supports session data recovery; Kingsoft Office is based on Agent Sandbox and Alibaba Cloud's cloud-native infrastructure, achieving millisecond-level sandbox acquisition through the preheating pool, combining tidal scheduling and multi-cluster disaster recovery, taking into account experience, cost and service continuity.

Yi Li, patent inventor and head of container and sandbox R&D at Alibaba Cloud, said, "Cloud computing has become a key productivity as important as electricity and water conservancy. From the mobile Internet to the AI ​​era, Alibaba Cloud has always focused on real-time perception of load changes, unified scheduling of heterogeneous computing power, and safe isolation in multi-tenant environments. Allowing computing power to flow and accurately match on demand, lower the threshold for large model training inference and agent operation, and promote the large-scale implementation of intelligent applications."

edit

Resend Email Modify Email