I am trying to decompress LZO-compressed data in Python using the lzo library. While the same compressed data is successfully decompressed in C++ using the lzo1z_decompress function, the equivalent Python implementation with lzo.decompress throws the following error:
lzoERR: Header error - invalid compressed data
Decompression failed: Header error - invalid compressed data
What I have tried Here is the C++ code snippet that works perfectly for decompression:
void PacketParser(char* rawBuffer)
{
short maxPktSize = 506, headerOffset = 8;
int offsetRaw = 0;
int offsetData = 0;
int dataLen = 0;
int cNetId = (int)((rawBuffer+offsetRaw)[0]);
offsetRaw += 2;
short iNoPackets=0;
memcpy(&iNoPackets, rawBuffer+offsetRaw, 2);
offsetRaw += 2;
iNoPackets = ntohs(iNoPackets);
for(int t = 0; t < iNoPackets; t++)
{
short iCompLen = 0;
memcpy(&iCompLen, rawBuffer+offsetRaw, 2);
offsetRaw += 2;
iCompLen = ntohs(iCompLen);
if(iCompLen>0)
{
int lzoOutSize = 4096;
int lzoERR, lzoERRCount=0;
///Decompress using LZO
lzo_uint i_len = (lzo_uint)iCompLen;
lzo_uint o_len = (lzo_uint)-1;
lzo_byte i_buff[1024];
lzo_memset(i_buff,0,1024);
memcpy(i_buff, rawBuffer+offsetRaw, iCompLen);
if (i_buff == NULL) return;
offsetRaw += iCompLen;
lzo_byte o_buff[4096];
lzo_memset(o_buff,0, 4096);
lzoERR = lzo1z_decompress(i_buff,i_len,o_buff,&o_len,NULL);
if(lzoERR!=0)
{
cout<<"lzoERR:"<<lzoERR<<endl;
return;
}
else
{
memcpy(dataBuffer+offsetData,o_buff,o_len);
}
dataLen = o_len;
}
else
memcpy(dataBuffer+offsetData, rawBuffer+offsetRaw, maxPktSize);
//parsing data here
}
}
The Python equivalent implementation is as follows:
def PacketParser(rawBuffer):
print(f"Raw Buffer Compressed data (hex): {rawBuffer.hex()}")
offsetRaw = 0
cNetId = int(rawBuffer[offsetRaw])
offsetRaw += 2
print(f"cNetId|{cNetId}")
iNoPackets = struct.unpack('!H', rawBuffer[offsetRaw:offsetRaw + 2])[0]
offsetRaw += 2
print(f"iNoPackets|{iNoPackets}")
iCompLen = struct.unpack('!H', rawBuffer[offsetRaw:offsetRaw + 2])[0]
offsetRaw += 2
print(f"iCompLenFirst|{iCompLen}")
i_buff = rawBuffer[offsetRaw:offsetRaw + iCompLen]
offsetRaw += iCompLen
print(f"Compressed data (hex): {i_buff.hex()}")
try:
decompressed_data = lzo.decompress(i_buff)
print(f"Decompressed Data: {decompressed_data}")
except lzo.error as e:
print(f"Decompression failed: {e}")
Observations
The Python code prints the compressed data as a hexadecimal string, which matches the expected input. Despite this, lzo.decompress consistently throws the header error.
Question
What am I doing wrong in the Python code? Is there any specific way to handle lzo1z_decompress behavior in Python? How can I achieve error-free decompression of this data using Python?