Page 1 of 1

Wrapping DEFLATED data

Posted: Sat Jun 15, 2013 12:57 am
by daryl
Hi Folks,

I have recently posted a question to the beginners forum titled "Inflating ZIP Deflated data string ", which I won't repeat here, but I have another angle I am trying on this problem, and was hoping maybe someone with more experience might be able to help. The issue is this: I have to read some deflated compressed data from a network socket and then inflate it back to the original. I discovered that LiveCode has the revZip functions to deal with zipped data, but they appear to only work on file archives. So created a file archive, added my data using the revZipAddUncompressedItemWithData function as I didn't want the data compressed yet again, closed the archive, opened it and used the revZipExtractItemToVariable function to get the data back from the archive and into a variable. That all worked, except the data was still deflated.

So having read the compress definition in the dictionary and about how to wrap deflated data with a header and a trailer, I thought I would try that, and I had partial success. That is, I can write my wrapped data to file and running the unix "file" command it tells me it's "file.gz: gzip compressed data, from Unix" but if I try to gunzip it I get "gzip: gzip.gz: invalid compressed data--format violated". I have done a hex dump to make sure the file has the 10 byte header and 8 byte trailer with my data in the middle as described in the dictionary and it looks like what I expect.

It is my understanding that both zip and gzip use the DEFLATE algorithm to compress data, and that the difference between the two is in the header and footer, so I am not sure what I'm doing wrong, as my data is deflated data, and I can read the same data and deflate it by using a perl script and their Compress::Zlib module.

Does anyone know if there are any plans on extending the revZip functions to include deflating a deflated binary string from a variable?

Any help, thoughts, pointers to documentation, etc. would be greatly appreciated.

Thanks,

Daryl

Re: Wrapping DEFLATED data

Posted: Sat Jun 15, 2013 4:05 am
by mwieder
Daryl- I cross-linked this to the Engine Contributors forum.

http://forums.runrev.com/viewtopic.php?f=66&t=15576

Re: Wrapping DEFLATED data

Posted: Sat Jun 15, 2013 4:16 am
by monte
You tried just using the decompress function I assume?

Re: Wrapping DEFLATED data

Posted: Sat Jun 15, 2013 4:27 am
by mwieder
Daryl-

Both zip and gzip *can* use the DEFLATE algorithm, but there are about a dozen different compression algorithms that *could* be used.
Do you have control over the sending mechanism as well? i.e., can you force it to use a different compression algorithm?
I'm reasonably sure that the revZip functions use only one method of compression, and I don't know which that is.

Re: Wrapping DEFLATED data

Posted: Sun Jun 16, 2013 3:42 pm
by daryl
Thanks Guys for your replies,

> ... Do you have control over the sending mechanism as well? i.e., can you force it to use a different compression algorithm?

Yes, but it's not my code so I will consult tomorrow with the developer to get more detailed information. It is my understanding that the code uses zip to compress the data, but I will get more information and details on Monday.

> You tried just using the decompress function I assume?

Yes, I've have tried using the decompress function, without success. It's what led me to examine the compress function in the dictionary.

> Daryl- I cross-linked this to the Engine Contributors forum.

Thanks, when I get more information/details I will also post it there.

Again, thanks for all your replies, I really appreciate it.

Regards,

Daryl

Re: Wrapping DEFLATED data

Posted: Mon Jun 17, 2013 6:02 pm
by Janschenkel
Well, there is a way to trick the 'decompress' function into inflating the data, but you'll need to know the length of the uncompressed data.
It involves tampering with the header and trailer of the compressed data; but the point is moot if you don't know the uncompressed length.

Jan Schenkel.

Re: Wrapping DEFLATED data

Posted: Mon Jun 17, 2013 8:57 pm
by daryl
Folks,

For anyone wanting to follow this issue, and to not post the same thing twice, I have posted my followup to the Engine Contributors forum at:

http://forums.runrev.com/viewtopic.php? ... 769#p78769

as I am hoping to see if it might be possible to add a LiveCode interface to the zlib in memory inflate and deflate functions, as described in RFC 1950, available at: http://www.ietf.org/rfc/rfc1950.txt.

Thank you everybody for your help.

Regards,

Daryl