New tech boosts real-time data compression for AI

Information and communication technologies, such as IoT and cloud computing, are cornerstones of modern society. The efficiency of these services depends heavily on the speed of processing and transmitting digital data, including image and sensor data. For instance, systems that identify individuals from surveillance camera images must streamline the entire process, from image transmission and artificial intelligence (AI) recognition to delivering results to users.

Since data transfer often consumes more time than AI computation, there is a critical need for a data compression method that minimizes both data volume and communication time while preserving the integrity of the original data. The new technology can accelerate data transmission for AI computations and reduce the volume of data transmitted over wide-area networks, such as 5G/6G wireless communications and the Internet, resulting in significant global power savings.

Conventional data compression technologies compress and store data in finite chunks. For this purpose, they require bulky memory and processors that increase the size of the compressor and limit real-time compression to finite data streams.

Researchers at the University of Tsukuba have addressed these challenges by developing a technology that automatically detects frequently occurring data patterns and compresses them to a minimum of one bit after a single pass through the compressor. This technology ensures complete real-time data restoration. The research is published in the journal IEEE Access.

Unlike previous technologies that can compress only one unit of data to one bit, the new method simultaneously compresses multiple data units to one bit, thereby enhancing the compression efficiency by 10%–30% compared to conventional methods.

In addition, this innovative technology enables the development of high-speed, compact data compression modules in hardware without the need for additional processors, memory, or other devices. If integrated into semiconductor chips and AI systems, this technology could revolutionize data transmission by boosting speed, reducing data volume, and conserving power in communication pathways, propelling us toward the vision of Society 5.0.

Source