Netty

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
NioDatagramChannel invalid usage of internalNioBuffer

Motivation:

NioDatagramChannel attempts to unpack a AddressedEnvelope and unconditionally uses internalNioBuffer. However if the ByteBuf is a CompositeByteBuf with more than 1 components, the write will fail and throw an exception.

Modifications:

- NioDatagramChannel should check the nioBufferCount before attempting

to use internalNioBuffer

Result:

No more failure to write UDP packets on NIO when a CompositeByteBuf is

used.

[maven-release-plugin] prepare for next development iteration

  1. … 7 more files in changeset.
[maven-release-plugin] prepare release netty-4.0.56.Final

  1. … 7 more files in changeset.
Adapt to API changes in Conscrypt 1.0.0.RC11

Motivation:

In google/conscrypt#313 the Conscrypt.Engines class was removed in favor

of methods directly on Conscrypt and overloading. The Conscrypt-using

code in Netty used reflection to access the old API, that doesn't exist

anymore. And thus recent versions of Conscrypt fail to enable things

like ALPN with Netty.

Modifications:

Instead of calling Conscrypt.Engines.isConscrypt, call

Conscrypt.isConscrypt.

Result:

Conscrypt detected properly at runtime.

ReadOnlyUnsafeDirectByteBuf.memoryAddress() should not throw

Motivation:

We need the memoryAddress of a direct buffer when using our native transports. For this reason ReadOnlyUnsafeDirectByteBuf.memoryAddress() should not throw.

Modifications:

- Correctly override ReadOnlyUnsafeDirectByteBuf.memoryAddress() and hasMemoryAddress()

- Add test case

Result:

Fixes [#7672].

JdkSslContext supported cipher suites incorrect

Motivation:

JdkSslContext builds the list of supported cipher suites, but assumes that ciphers prefixed with SSL_ and TLS_ will be interchangeable. However this is not the case and only applies to a small subset of ciphers. This results in the JdkSslContext attempting to use unsupported ciphers.

Modifications:

- When building the list of ciphers in JdkSslContext we should first check if the engine supports the TLS_ prefix cipher.

Result:

Fixes https://github.com/netty/netty/issues/7673

Avoid register multiple cleaner task for same thread's FastThreadLocal index

Motivation:

Currently if user call set/remove/set/remove many times, it will create multiple cleaner task for same index. It may cause OOM due to long live thread will have more and more task in LIVE_SET.

Modification:

Add flag to avoid recreating tasks.

Result:

Only create 1 clean task. But use more space of indexedVariables.

Cleanup buffer tests.

Motivation:

There is some cleanup that can be done.

Modifications:

- Use intializer list expression where possible

- Remove unused imports.

Result:

Cleaner code.

SslHandler unwrap out of order promise/event notificaiton

Motivation:

SslHandler#decode methods catch any exceptions and attempt to wrap

before shutting down the engine. The intention is to write any alerts

which the engine may have pending. However the wrap process may also

attempt to write user data, and may also complete the associated

promises. If this is the case, and a promise listener closes the channel

then SslHandler may later propagate a SslHandshakeCompletionEvent user

event through the pipeline. Since the channel has already been closed

the user may no longer be paying attention to user events.

Modifications:

- Sslhandler#decode should first fail the associated handshake promise

and propagate the SslHandshakeCompletionEvent before attempting to wrap

Result:

Fixes https://github.com/netty/netty/issues/7639

Increase timeout and decrement number of operations in AbstractByteBufTest.testToStringMultipleThreads

Motivation:

We saw some timeouts on the CI when the leak detection is enabled.

Modifications:

- Use smaller number of operations in test

- Increase timeout

Result:

CI not times out.

ByteBufUtil.isText method should be safe to be called concurrently

Motivation:

ByteBufUtil.isText(...) may produce unexpected results if called concurrently on the same ByteBuffer.

Modifications:

- Don't use internalNioBuffer where it is not safe.

- Add unit test.

Result:

ByteBufUtil.isText is thread-safe.

Reflective setAccessible(true) will produce scary warnings on the console when using java9+, dont do it

Motivation:

Reflective setAccessible(true) will produce scary warnings on the console when using java9+, while netty still works. That said users may feel uncomfortable with these warnings, we should not try to do it by default when using java9+.

Modifications:

Add io.netty.tryReflectionSetAccessible system property which controls if setAccessible(...) will be used. By default it will bet set to false when using java9+.

Result:

Fixes [#7254].

ObjectCleanerThread must be a deamon thread to ensure the JVM can always terminate.

Motivation:

The ObjectCleanerThread must be a daemon thread as otherwise we may block the JVM from exit. By using a daemon thread we basically give the same garantees as the JVM when it comes to cleanup of resources (as the GC threads are also daemon threads and the CleanerImpl uses a deamon thread as well in Java9+).

Modifications:

Change ObjectCleanThread to be a daemon thread.

Result:

JVM shutdown will always be able to complete. Fixed [#7617].

Add java-doc for implemented methods of io.netty.util.concurrent.Future#cancel(boolean mayInterruptIfRunning)

Motivation:

The methods implement io.netty.util.concurrent.Future#cancel(boolean mayInterruptIfRunning) which actually ignored the param mayInterruptIfRunning.We need to add comments for the `mayInterruptIfRunning` param.

Modifications:

Add comments for the `mayInterruptIfRunning` param.

Result:

People who call the `cancel` method will be more clear about the effect of `mayInterruptIfRunning` param.

Fix HttpPostMultipartRequestDecoder.splitMultipartHeader() String index out of range: -1 with empty header

Motivation:

A Malformed empty header value (e.g. Content-Type: \r\n) will trigger a String index out of range

while trying to parse the multi-part request, using the HttpPostMultipartRequestDecoder.

Modification:

Ensure that the substring() method is called passing the endValue >= valueStart.

In case of an empty header value, the empty header value associated with the header key will be returned.

Result:

Fixes #7620

Set thread context classloader in a doPrivileged block

Motivation:

In a few classes, Netty starts a thread and then sets the context classloader of these threads

to prevent classloader leaks. The Thread#setContextClassLoader method is a privileged method in

that it requires permissions to be executed when there is a security manager in place. Unless

these calls are wrapped in a doPrivileged block, they will fail in an environment with a security

manager and restrictive policy in place.

Modifications:

Wrap the calls to Thread#setContextClassLoader in a AccessController#doPrivileged block.

Result:

After this change, the threads can set the context classloader without any errors in an

environment with a security manager and restrictive policy in place.

Replace reflective access of Throwable#addSuppressed with version guarded access

Motivation:

In environments with a security manager, the reflective access to get the reference to

Throwable#addSuppressed can cause issues that result in Netty failing to load. The main

motivation in this pull request is to remove the use of reflection to prevent issues in

these environments.

Modifications:

ThrowableUtil no longer uses Class#getDeclaredMembers to get the Method that references

Throwable#addSuppressed and instead guards the call to Throwable#addSuppressed with a

Java version check.

Additionally, a annotation was added that suppresses the animal sniffer java16 signature

check on the given method. The benefit of the annotation is that it limits the exclusion

of Throwable to just the ThrowableUtil class and has string text indicating the reason

for suppressing the java16 signature check.

Result:

Netty no longer requires the use of Class#getDeclaredMethod for ThrowableUtil and will

work in environments restricted by a security manager without needing to grant reflection

permissions.

Fixes #7614

DefaultChannelPipeline will not invoke handler if events are fired from handlerAdded

Motiviation:

DefaultChannelPipeline and AbstractChannelHandlerContext maintain state

which indicates if a ChannelHandler should be invoked or not. However

the state is updated to allow the handler to be invoked only after the

handlerAdded method completes. If the handlerAdded method generates

events which may result in other methods being invoked on that handler

they will be missed.

Modifications:

- DefaultChannelPipeline should set the state before calling

handlerAdded

Result:

DefaultChannelPipeline will allow events to be processed during the

handlerAdded process.

[maven-release-plugin] prepare for next development iteration

  1. … 7 more files in changeset.
[maven-release-plugin] prepare release netty-4.0.55.Final

  1. … 7 more files in changeset.
Fix ByteBuf.nioBuffer(...) and nioBuffers(...) docs to reflect reality.

Motivation:

Depending on the implementation of ByteBuf nioBuffer(...) and nioBuffers(...) may either share the content or return a ByteBuffer that contains a copy of the content.

Modifications:

Fix javadocs.

Result:

Correct docs.

ByteBuf.toString(Charset) is not thread-safe

Motivation:

Calling ByteBuf.toString(Charset) on the same buffer from multiple threads at the same time produces unexpected results, such as various exceptions and/or corrupted output. This is because ByteBufUtil.decodeString(...) is taking the source ByteBuffer for CharsetDecoder.decode() from ByteBuf.internalNioBuffer(int, int), which is not thread-safe.

Modification:

Call ByteBuf.nioBuffer() instead of ByteBuf.internalNioBuffer() to get the source buffer to pass to CharsetDecoder.decode().

Result:

Fixes the possible race condition.

ObjectCleaner should continue cleaning despite exceptions

Motivation:

ObjectCleaner inovkes a Runnable which may execute user code (FastThreadLocal#onRemoval) and therefore exceptions maybe thrown. If an exception is thrown the cleanup thread will exit prematurely and we may never finish cleaning up which will result in leaks.

Modifications:

- ObjectCleaner should suppress exceptions and continue cleaning

Result:

ObjectCleaner will reliably clean despite exceptions being thrown.

ObjectCleaner may indefinitely block on ReferenceQueue#poll

Motivation:

ObjectCleaner polls a ReferenceQueue which will block indefinitely. However it is possible there is a race condition between the live set of objects being empty due to the WeakReference being cleaned/cleared and polling the queue. If this situation occurs the cleanup thread may never unblock if no more objects are added to the live set, and may result in an application's failure to gracefully close.

Modifications:

- ReferenceQueue.remove should use a timeout to compensate for the race condition, and avoid dead lock

Result:

No more dead lock in ObjectCleaner when polling the ReferenceQueue.

Provide a Docker image for reproducible builds

Motivation:

It would be good to provide a docker image for people that want to build netty on linux.

Modifications:

Add a docker file

Result:

People can more easily build netty. Fixes [#7585].

    • -0
    • +34
    /docker/Dockerfile-netty-centos6
Correctly take position into account when wrap a ByteBuffer via ReadOnlyUnsafeDirectByteBuf

Motivation:

We did not correctly take the position into account when wrapping a ByteBuffer via ReadOnlyUnsafeDirectByteBuf as we obtained the memory address from the original ByteBuffer and not the slice we take.

Modifications:

- Correctly use the slice to obtain memory address.

- Add test case.

Result:

Fixes [#7565].

Include mvn wrapper to make setup of development env easier

Motivation:

Someone intending to contribute should be able to set up their development environment quickly and easily.

Modifications:

- Added a Maven wrapper such that a local Maven installation isn't necessary.

- Added a .gitattributes such that auto line-endings are enforced.

Result:

- ./mvnw is enough to build.

- Git line-endings are enforced.

- Fixes #7578.

    • binary
    /.mvn/wrapper/maven-wrapper.jar
    • -0
    • +1
    /.mvn/wrapper/maven-wrapper.properties
    • -0
    • +202
    /license/LICENSE.mvn-wrapper.txt
Fixes #7566 by handling concatenated GZIP streams.

Motivation:

According to RFC 1952, concatenation of valid gzip streams is also a valid gzip stream. JdkZlibDecoder only processed the first and discarded the rest.

Modifications:

- Introduced a constructor argument decompressConcatenated that if true, JdkZlibDecoder would continue to process the stream.

Result:

- If 'decompressConcatenated = true', concatenated streams would be processed in

compliance to RFC 1952.

- If 'decompressConcatenated = false' (default), existing behavior would remain.

    • binary
    /codec/src/test/resources/multiple.gz
Remove direct usage of JKS and SunX509

Motivation:

When using netty on android or with for example a IBM JVM it may not be able to build a SslContext as we hardcoded the use of JKS and SunX509 (which both may not be present).

Modifications:

- Use the default algorithm / type which can be override via a System property

- Remove System property check as its redundant with KeyManagerFactory.getDefaultAlgorithm()

Result:

More portable code. Fixes [#7546].

Remove remote initiated renegotiation support

Motivation:

We recently removed support for renegotiation, but there are still some hooks to attempt to allow remote initiated renegotiation to succeed. The remote initated renegotiation can be even more problematic from a security stand point and should also be removed.

Modifications:

- Remove state related to remote iniated renegotiation from OpenSslEngine

Result:

More renegotiation code removed from the OpenSslEngine code path.