TY - JOUR
T1 - Improving Marlin's compression ratio with partially overlapping codewords
AU - Martinez, Manuel
AU - Sandfort, Kai
AU - Dube, Danny
AU - Serra-Sagrista, Joan
N1 - Publisher Copyright:
© 2018 IEEE.
PY - 2018/7/19
Y1 - 2018/7/19
N2 - Marlin [1] is a Variable-to-Fixed (VF) codec optimized for decoding speed. To achieve its speed, Marlin does not encode the current state of the input source, penalyzing compression ratio. In this paper we address this penalty by partially encoding the current state of the input in the lower bits of the codeword. Those bits select which chapter in the dictionary must be used to decode the next codeword. Each chapter is specialized for a subset of states, improving compression ratio. At the same time, we use one victim chapter to encode all rare symbols, increasing the efficiency of the rest of them. The decoding algorithm remains the same, only now codewords have overlapping bits. Mapping techniques allow us to combine common chapters and thus keep an efficient use of the L1 cache. We evaluate our approach with both synthetic and real data sets, and show significant improvements in low entropy sources, where compression efficiency can improve from 93.9% to 98.6%.
AB - Marlin [1] is a Variable-to-Fixed (VF) codec optimized for decoding speed. To achieve its speed, Marlin does not encode the current state of the input source, penalyzing compression ratio. In this paper we address this penalty by partially encoding the current state of the input in the lower bits of the codeword. Those bits select which chapter in the dictionary must be used to decode the next codeword. Each chapter is specialized for a subset of states, improving compression ratio. At the same time, we use one victim chapter to encode all rare symbols, increasing the efficiency of the rest of them. The decoding algorithm remains the same, only now codewords have overlapping bits. Mapping techniques allow us to combine common chapters and thus keep an efficient use of the L1 cache. We evaluate our approach with both synthetic and real data sets, and show significant improvements in low entropy sources, where compression efficiency can improve from 93.9% to 98.6%.
KW - Entropic encoding
KW - High Throughput
UR - http://www.scopus.com/inward/record.url?scp=85050963275&partnerID=8YFLogxK
U2 - 10.1109/DCC.2018.00041
DO - 10.1109/DCC.2018.00041
M3 - Article
AN - SCOPUS:85050963275
SN - 1068-0314
SP - 325
EP - 334
JO - Data Compression Conference Proceedings
JF - Data Compression Conference Proceedings
ER -