zlib crashes when given windowBits: 8

[jcoglan]

Version: 4.8.2, 6.10.2, 7.6+, 8.0+
Platform: Darwin liskov.home 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64
Subsystem: zlib

On the above versions of Node, this program crashes:

var zlib = require('zlib');
zlib.createDeflateRaw({windowBits: 8});

I believe this coincides with bumping zlib from 1.2.8 to 1.2.11, but I'm not 100% sure. I'm not sure if 8 is no longer a legal value, but zlib.Z_MIN_WINDOWBITS equals 8 on affected platforms.

[NickNaso]

From zlib documentation seems that current implementation of deflate algorithm do not support windowsBits value of 8.
I report what i found from here https://www.zlib.net/manual.html

For the current implementation of deflate(), a windowBits value of 8 (a window size of 256 bytes) is not supported.

This behaviour is documented only on version 8 documentation:
https://nodejs.org/dist/latest-v8.x/docs/api/zlib.html#zlib_zlib_createdeflateraw_options

[addaleax]

This is a duplicate of #13082, I think.

Version: 4.8.2, 6.10.2, 7.6+, 8.0+

On 8+ it should throw an error, though? Can you make sure?

[jcoglan]

From zlib documentation seems that current implementation of deflate algorithm do not support windowsBits value of 8.

This is going to be a problem; permessage-deflate and the Autobahn test suite require that if a server is asked to only use a window size of N to send messages to the client, it commits to using either N or a lower value, including N=8.

This is a duplicate of #13082, I think.

It might be; I saw that issue and was unsure. Autobahn is the reason I found the error -- I maintain the permessage-deflate, websocket-driver and faye-websocket packages.

On 8+ it should throw an error, though? Can you make sure?

It does, and also on the latest 4.x and 6.x. On 7.x the error cannot be caught, and on 5.x the problem does not exist. My issue isn't so much about whether it throws a catchable exception so much as that we cannot set windowBits to 8.

Is this a new permanent limitation in zlib? Will other software need to take this limitation into account, and will Z_MIN_WINDOWBITS be updated?

[refack]

@jcoglan see #13098 (comment)
which leads to http://zlib.net/manual.html#Advanced (For the current implementation of deflate(), a windowBits value of 8 (a window size of 256 bytes) is not supported.)

[jcoglan]

@refack I've read that thread but there's a couple of things I still don't know:

• Why is 8 no longer a valid windowBits value?
• What are my options as someone maintaining a library that is supposed to support windowBits=8?

[addaleax]

@jcoglan You might be interested in looking at madler/zlib#171 :)

[jcoglan]

@addaleax Thank you, that was really helpful and has got me to make progress with my issue.

I now know from this source and the Node changelog that:

• in Node v4.8.2, v6.10.2, and v8.0.0, zlib is bumped from 1.2.8 to 1.2.11 due to some CVEs
• some time between these versions, zlib stopped accepting windowBits=8 for raw streams

Does anybody know if this windowBits change is related to the CVEs, of if it's an unrelated change swept up in these releases? I'm having trouble understanding the details of the security reports.

[MylesBorins]

If the change is unrelated to the CVE should we try and patch support for LTS?

[addaleax]

I think we should patch that in LTS, yes (I assume just using 9 when 8 was passed would be the easiest and most sensible thing). @MylesBorins Can you do the PR(s)?

[jcoglan]

Does anyone know why zlib stopped accepting 8, if it's not related to a security thing? Only reason I hesitated to do this replacement in my library is that I figured there must be some reason zlib removed this behaviour.

[addaleax]

I think the linked zlib thread contains the relevant thoughts on that. If you want to be certain, I’d recommend asking them more directly, I don’t think any collaborators here know for sure.

[MylesBorins]

I'll try and dig into this a bit tomorrow unless someone is really amped to do it 😎

[MylesBorins]

SOOOOO this is fun

It would appear that zlib is already uses 9bits. The problems people are running into are edge cases that cause things to asplode.

This patch over on the optipng sourceforge claims to be a fix fix. I'm unsure of how this code would be licensed so I'm not provided a patch at this time. The copyright might belong to @ctruta who I believe works at IBM with @mhdawson.

The patch can be applied it to zlib master with minor conflicts. It was then possible to both build zlib and run the test suite. This patch could be cherry-picked to any of our release lines. I tested both master and v6.x and every thing seems alright. We did not appear to be testing this edge case, so we should try and get a test together that proves things are working for this specific window size.

The patch was originally meant to be applied 1.1.3 with the hope it would land in 1.1.4. A partial fix landed in 1.1.4 with a promise for a full fix in 1.1.5. The change was reverted in the next release 1.2.0. In 1.2.0.2 windowBit = 8 would be rounded up to 9 bits, this is the current state of master.

@Mandler is there history about this patch that we should know?

edit: it is dawning on me right now that even with all my spelunking that I simply needed to test...

var zlib = require('zlib');
zlib.createDeflateRaw({windowBits: 8});

and this was broken between 1.2.8 to 1.2.11...

going to do some bisecting. That was some fun history though 😄 ... still curious why the fix never landed