Clone
 

aleksey yeschenko <aleksey@yeschenko.com> in Netty

Prevent ByteToMessageDecoder from overreading when !isAutoRead (#9252)

Motivation:

ByteToMessageDecoder only looks at the last channelRead() in the batch

of channelRead()-s when determining whether or not it should call

ChannelHandlerContext#read() to consume more data when !isAutoRead. This

will lead to read() calls issued unnecessaily and unprompted if the very

last channelRead() didn't result in at least one decoded message, even

if there have been messages decoded from other channelRead()-s in the

current batch.

Modifications:

Track decode outcomes for the entire batch of channelRead() calls and

only issue a read in BTMD if the entire batch of channelRead() calls

yielded no complete messages.

Result:

ByteToMessageDecoder will no longer overread when the very last read

yielded no message, but the batch of reads did.

Prevent ByteToMessageDecoder from overreading when !isAutoRead (#9252)

Motivation:

ByteToMessageDecoder only looks at the last channelRead() in the batch

of channelRead()-s when determining whether or not it should call

ChannelHandlerContext#read() to consume more data when !isAutoRead. This

will lead to read() calls issued unnecessaily and unprompted if the very

last channelRead() didn't result in at least one decoded message, even

if there have been messages decoded from other channelRead()-s in the

current batch.

Modifications:

Track decode outcomes for the entire batch of channelRead() calls and

only issue a read in BTMD if the entire batch of channelRead() calls

yielded no complete messages.

Result:

ByteToMessageDecoder will no longer overread when the very last read

yielded no message, but the batch of reads did.

Fix LZ4 encoder/decoder performance with (default) xxHash32 (#9249)

Motivation:

Lz4FrameEncoder and Lz4FrameDecoder in their default configuration use

an extremely inefficient way to checksum direct byte buffers. In

particular, for every byte checksummed, a single-element byte array is

being allocated and a JNI cal is made, which in some internal testing

makes a 25x difference in total throughput and allocates *a lot* of

garbage.

Modifications:

Lz4XXHash32, an implementation of ByteBufChecksum specifically for use

by Lz4FrameEncoder and Lz4FrameDecoder, is introduced. It utilises

xxHash32 block API which provides a hash() method that accepts a

ByteBuffer as an argument. Lz4FrameEncoder and Lz4FrameDecoder are

modified to use this implementation by default.

Result:

Lz4FrameEncoder and Lz4FrameDecoder perform well again when operating

on direct byte buffers with default checksum configuration; a public

implementation is provided for those who need to override the seed.

Fix LZ4 encoder/decoder performance with (default) xxHash32 (#9249)

Motivation:

Lz4FrameEncoder and Lz4FrameDecoder in their default configuration use

an extremely inefficient way to checksum direct byte buffers. In

particular, for every byte checksummed, a single-element byte array is

being allocated and a JNI cal is made, which in some internal testing

makes a 25x difference in total throughput and allocates *a lot* of

garbage.

Modifications:

Lz4XXHash32, an implementation of ByteBufChecksum specifically for use

by Lz4FrameEncoder and Lz4FrameDecoder, is introduced. It utilises

xxHash32 block API which provides a hash() method that accepts a

ByteBuffer as an argument. Lz4FrameEncoder and Lz4FrameDecoder are

modified to use this implementation by default.

Result:

Lz4FrameEncoder and Lz4FrameDecoder perform well again when operating

on direct byte buffers with default checksum configuration; a public

implementation is provided for those who need to override the seed.

Fix ReflectiveByteBufChecksum with direct buffers (#9244)

Motivation:

ReflectiveByteBufChecksum#update(buf, off, len) ignores provided offset

and length arguments when operating on direct buffers, leading to wrong

byte sequences being checksummed and ultimately incorrect checksum

values (unless checksumming the entire buffer).

Modifications:

Use the provided offset and length arguments to get the correct nio

buffer to checksum; add test coverage exercising the four meaningfully

different offset and length combinations.

Result:

Offset and length are respected and a correct checksum gets calculated;

simple unit test should prevent regressions in the future.

Fix ReflectiveByteBufChecksum with direct buffers (#9244)

Motivation:

ReflectiveByteBufChecksum#update(buf, off, len) ignores provided offset

and length arguments when operating on direct buffers, leading to wrong

byte sequences being checksummed and ultimately incorrect checksum

values (unless checksumming the entire buffer).

Modifications:

Use the provided offset and length arguments to get the correct nio

buffer to checksum; add test coverage exercising the four meaningfully

different offset and length combinations.

Result:

Offset and length are respected and a correct checksum gets calculated;

simple unit test should prevent regressions in the future.