Netty

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Works

WIP

More work on http2 and http

Fix http codec

Aggregator fix

Fix ipfilter handlers

Fix ha codec

unlink when replace handler

Getting started

Remove the intermediate List from ByteToMessageDecoder (and sub-class… (#8626)

Motivation:

ByteToMessageDecoder requires using an intermediate List to put results into. This intermediate list adds overhead (memory/CPU) which grows as the number of objects increases. This overhead can be avoided by directly propagating events through the ChannelPipeline via ctx.fireChannelRead(...). This also makes the semantics more clear and allows us to keep track if we need to call ctx.read() in all cases.

Modifications:

- Remove List from the method signature of ByteToMessageDecoder.decode(...) and decodeLast(...)

- Adjust all sub-classes

- Adjust unit tests

- Fix javadocs.

Result:

Adjust ByteToMessageDecoder as noted in https://github.com/netty/netty/issues/8525.

  1. … 88 more files in changeset.
ByteToMessageDecoder Cumulator improments (#9877)

Motivation:

ByteToMessageDecoder's default MERGE_CUMULATOR will allocate a new buffer and

copy if the refCnt() of the cumulation is > 1. However this is overly

conservative because we maybe able to avoid allocate/copy if the current

cumulation can accommodate the input buffer without a reallocation. Also when the

reallocation and copy does occur the new buffer is sized just large enough to

accommodate the current the current amount of data. If some data remains in the

cumulation after decode this will require a new allocation/copy when more data

arrives.

Modifications:

- Use maxFastWritableBytes to avoid allocation/copy if the current buffer can

accommodate the input data without a reallocation operation.

- Use ByteBufAllocator#calculateNewCapacity(..) to get the size of the buffer

when a reallocation/copy operation is necessary.

Result:

ByteToMessageDecoder MERGE_CUMULATOR won't allocate/copy if the cumulation

buffer can accommodate data without a reallocation, and when a reallocation

occurs we are more likely to leave additional space for future data in an effort

to reduce overall reallocations.

Verify we do not receive multiple content-length headers or a content-length and transfer-encoding: chunked header when using HTTP/1.1 (#9865)

Motivation:

RFC7230 states that we should not accept multiple content-length headers and also should not accept a content-length header in combination with transfer-encoding: chunked

Modifications:

- Check for multiple content-length headers and if found mark message as invalid

- Check if we found a content-length header and also a transfer-encoding: chunked and if so mark the message as invalid

- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9861

Add unit test for leak aware CompositeByteBuf that proves that there is no NPE (#9875)

Motivation:

https://github.com/netty/netty/issues/9873 reported a NPE in previous version of netty. We should add a unit test to verify there is no more NPE

Modifications:

Add a unit test

Result:

Prove that https://github.com/netty/netty/issues/9873 is fixed

#9867 fix confusing method parameter name (#9874)

Motivation:

Parameter name is confusing and not match the actual type.

Modification:

Rename parameter.

Result:

Code cleanup

Detect missing colon when parsing http headers with no value (#9871)

Motivation:

Technical speaking its valid to have http headers with no values so we should support it. That said we need to detect if these are "generated" because of an "invalid" fold.

Modifications:

- Detect if a colon is missing when parsing headers.

- Add unit test

Result:

Fixes https://github.com/netty/netty/issues/9866

Revert "Epoll: Avoid redundant EPOLL_CTL_MOD calls (#9397)"

This reverts commit 250b279bd98e043e8c56dd9b1d6545a85790c10c.

Fix #9770, last frame may contain extra data that doesn't affect decompression (#9832)

Motivation:

Client can split data into different numbers of fragments and sometimes the last frame may contain trash data that doesn't affect decompression process.

Modification:

Added check if last frame is `ContinuationWebSocketFrame` and decompression data is empty

then don't throw an exception.

Result:

Fixes #9770

Ensure scheduled tasks are executed before shutdown (#9858)

Motivation:

In #9603 the executor hung on shutdown because of an abandoned task

on another executor the first was waiting for.

Modifications:

This commit modifies the executor shutdown sequence to include

switching to SHUTDOWN state and then running all remaining tasks.

This ensures that no more tasks are scheduled after SHUTDOWN and

the last pass of running remaining tasks will take it all.

Any tasks scheduled after SHUTDOWN will be rejected.

This change preserves the functionality of graceful shutdown with

quiet period and only adds one more pass of task execution after

the default shutdown process has finished and the executor is

ready for termination.

Result:

After this change tasks that succeed to be added to the executor will

be always executed. Tasks which come late will be rejected instead of

abandoned.

Update ChannelOutboundBuffer.java (#9863)

Motivation:

fix comment of ChannelOutboundBuffer.CHANNEL_OUTBOUND_BUFFER_ENTRY_OVERHEAD

Modifications:

6 (NOT 8) reference fields is right

Result:

Correct comment

Add Server Cookie Parser overload for multiple cookies. (#9856)

Motivation:

Multiple cookie values can be present in a single header.

Modification:

Add `decodeAll` overload which returns all cookies

Result:

Fixes #7210

Note:

This change is not as perscriptive as the ideas brought up in the linked issue. Changing the Set implementation or the equals/compareTo definition is likely a breaking change, so they are practical.

Change DefaultByteBufHolder.equals() to treat instances of different classes as not equal (#9855)

# Motivation:

`DefaultByteBufHolder.equals()` considers another object equal if it's an instance of `ByteBufferHolder` and if the contents of two objects are equal. However, the behavior of `equals` method is not a part of the `ByteBufHolder` contract so `DefaultByteBufHolder`'s version may be causing violation of the symmetric property if other classes have different logic.

There are already a few classes that are affected by this: `DefaultHttp2GoAwayFrame`, `DefaultHttp2UnknownFrame`, and `SctpMessage` are all overriding `equals` method breaking the symmetric property.

Another effect of this behavior is that all instances with empty data are considered equal. That may not be desireable in the situations when instances are created for predefined constants, e.g. `FullBulkStringRedisMessage.NULL_INSTANCE` and `FullBulkStringRedisMessage.EMPTY_INSTANCE` in `codec-redis`.

# Modification:

Make `DefaultByteBufHolder.equals()` implementation only work for the objects of the same class.

# Result:

- The symmetric property of the `equals` method is restored for the classes in question.

- Instances of different classes are not considered equal even if the content of the data they hold are the same.

Replace ObjectUtil.checkNonNull(...) with Objects.requireNonNull(...) (#9864)

Motivation:

We should use Objects.requireNonNull(...) as we require java8

Modifications:

Replace ObjectUtil.checkNonNull(...) with Objects.requireNonNull(...)

Result:

Code cleanup

  1. … 14 more files in changeset.
DefaultHttp2ConnectionEncoder writeHeaders method always send an head frame with a priority (#9852)

Motivation:

The current implementation delegates to writeHeaders(ChannelHandlerContext ctx, int streamId, Http2Headers headers, int streamDependency, short weight, boolean exclusive, int padding, boolean endStream, ChannelPromise promise) that will send an header frame with the priority flag set and the default priority values even if the user didnt want too.

Modifications:

- Change DefaultHttp2ConnectionEncoder to call the correct Http2FrameWriter method depending on if the user wants to use priorities or not

- Adjust tests

Result:

Fixes https://github.com/netty/netty/issues/9842

SnappyFrameDecoderTest ByteBuf leak (#9854)

Motivation:

SnappyFrameDecoderTest has a few tests which fail to close the EmbeddedChannel

and therefore may leak ByteBuf objects.

Modifications:

- Make sure EmbeddedChannel#finishAndReleaseAll() is called in all tests

Result:

No more leaks from SnappyFrameDecoderTest.

Correctly close `EmbeddedChannel` and release buffers in `SnappyFrameDecoderTest` (#9851)

Motivation:

We did not correctly close the `EmbeddedChannel` which would lead to not have `handlerRemoved(...)` called. This can lead to leaks. Beside this we also did not correctly consume produced data which could also show up as a leak.

Modifications:

- Always call `EmbeddedChannel.finish()`

- Ensure we consume all produced data and release it

Result:

No more leaks in test. This showed up in https://github.com/netty/netty/pull/9850#issuecomment-562504863.

Replace synchronized with ConcurrentHashMap in Http2StreamChannelBootstrap (#9848)

Motivation:

97361fa2c89da57e88762aaca9e2b186e8c148f5 replace synchronized with ConcurrentHashMap in *Bootstrap classes but missed to do the same for the Http2 variant.

Modifications:

- Use ConcurrentHashMap

- Simplify code in *Bootstrap classes

Result:

Less contention

Minor simplifications/optimizations to AbstractByteBuf methods (#9845)

Motivation

While working on other changes I noticed some opportunities to

streamline a few things in AbstractByteBuf.

Modifications

- Avoid duplicate ensureAccessible() checks in discard(Some)ReadBytes()

and ensureWritable0(int) methods

- Simplify ensureWritable0(int) logic

- Make some conditional checks more concise

Result

Cleaner, possibly faster code

Include JCTools sources for shaded classes in the sources jar

Motivation:

We should include the shaded sources for JCTools in our sources jar to make it easier to debug.

Modifications:

- Adjust plugin configuration to execute plugins in correct order

- Update source plugin

- Add configuration for shade plugin to generate source jar content

Result:

Fixes https://github.com/netty/netty/issues/6640.

Correctly take architecture into account when define syscalls for recvmmsg and sendmmsg usage (#9844)

Motivation:

https://github.com/netty/netty/pull/9797 changed the code for recvmmsg and sendmmsg to use the syscalls directly to remvove the dependency on newer GLIBC versions. Unfortunally it made the assumption that the syscall numbers are the same for different architectures, which is not the case.

Thanks to @jayv for pointing it out

Modifications:

Add #if, #elif and #else declarations to ensure we pick the correct syscall number (or not support if if the architecture is not supported atm).

Result:

Pick the correct syscall number depending on the architecture.

Update to netty-tcnative 2.0.28.Final (#9846)

Motivation:

netty-tcnative 2.0.28.Final was released

Modifications:

Update to latest version

Result:

Use latest version of netty-tcnative