Netty

Clone Tools
  • last updated a few seconds ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
Add constants for fixed-header only MQTT messages (#9749)

Motivation:

Currently, the only way to create fixed-header only messages PINGREQ,

PINGRESP and DISCONNECT is to explicitly instantiate a `MqttFixedHeader` like:

```

MqttFixedHeader disconnectFixedHeader = new MqttFixedHeader(MqttMessageType.DISCONNECT,

false, MqttQoS.AT_MOST_ONCE, false, 0);

MqttMessage disconnectMessage = new MqttMessage(disconnectFixedHeader);

```

According to the MQTT spec

(http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718077),

the fixed-header flags for these messages are reserved and must be set to zero, otherwise

the receiver must close the connection. It's easy to mess this up when

you're creating the header explicitly, for e.g by setting the QoS bit to

`AT_LEAST_ONCE`.

As such, provide static constants for PINGREQ, PINGRESP and

DISCONNECT messages that will set the flags correctly for the developer.

Modification:

Add static constants to MqttMessage class to construct PINGREQ, PINGRESP and

DISCONNECT messages that will set the fixed-header flags correctly to 0.

Result:

Easier usage.

Add test to check Connection-Specific headers are removed in HTTP/2 (by HttpConversionUtil.toHttp2Headers) (#9766)

Motivation:

To avoid regression regarding connection-specific headers[1], we should add a test.

[1] https://tools.ietf.org/html/rfc7540#section-8.1.2.2

Modification:

Add test that checks the following headers are removed.

- Connection

- Host

- Keep-Alive

- Proxy-Connection

- Transfer-Encoding

- Upgrade

Result:

There's no functional change.

Update to latest netty-tcnative release (#9763)

Motivation:

There is a new netty-tcnative release. We should use it.

Modifications:

Update to 2.0.27

Result:

Use latest netty-tcnative release

Don't cache key material if sun.security.ssl.X509KeyManagerImpl is used (#9762)

Motivation:

sun.security.ssl.X509KeyManagerImpl will not use "stable" aliases and so aliases may be changed during invocations. This means caching is useless. Because of this we should disable the cache if its used.

Modifications:

- Disable caching if sun.security.ssl.X509KeyManagerImpl is used

- Add tests

Result:

More protection against https://github.com/netty/netty/issues/9747.

At the moment the cache provided by OpenSslCachingKeyMaterialProvider… (#9759)

Motivation:

At the moment te cache is not bound and so lead to huge memory consumpation. We should ensure its bound by default.

Modifications:

Ensure cache is bound

Result:

Fixes https://github.com/netty/netty/issues/9747.

Fix encoding/decoding for UTF-8 stomp commands and headers (#9740)

Motivation:

According STOMP spec (https://stomp.github.io/stomp-specification-1.2.html#Value_Encoding) we have to encode and decode commands and headers to UTF-8

Modification:

Provide ability for StompSubframeDecoder and StompSubframeEncoder work with UTF-8

Introduce ByteBuf#isContiguous() method (#9735)

Motivation

There's currently no way to determine whether an arbitrary ByteBuf

behaves internally like a "singluar" buffer or a composite one, and this

can be important to know when making decisions about how to manipulate

it in an efficient way.

An example of this is the ByteBuf#discardReadBytes() method which

increases the writable bytes for a contiguous buffer (by readerIndex)

but does not for a composite one.

Unfortunately !(buf instanceof CompositeByteBuf) is not reliable, since

for example this will be true in the case of a sliced CompositeByteBuf

or some third-party composite implementation.

isContiguous was chosen over isComposite since we want to assume "not

contiguous" in the unknown/default case - the doc will it clear that

false does not imply composite.

Modifications

- Add ByteBuf#isContiguous() which returns true by default

- Override the "concrete" ByteBuf impls to return true and ensure

wrapped/derived impls delegate it appropriately

- Include some basic unit tests

Result

Better assumptions/decisions possible when manipulating arbitrary

ByteBufs, for example when combining/cumulating them.

Clean up expired docs. (#9756)

Motivation:

Since the `extractFrame` has used `retainedSlice` to avoid memory copy, we should clean this doc that was expired.

Result:

Better doc.

Remove padding when writing CONTINUATION frame (#9752)

Motivation:

Padding was removed from CONTINUATION frame in http2-spec, as showed in [PR](https://github.com/http2/http2-spec/pull/510). We should follow it.

Modifications:

- Remove padding when writing CONTINUATION frame in DefaultHttp2FrameWriter

- Add a unit test for writing large header with padding

Result:

More spec-compliant

Cleanup Recycler to better encapsulate stuff (#9739)

Motivation:

We can move some methods etc to make encapsulation better in Recycler

Modifications:

Move / rename methods to make usage more clear

Result:

Code cleanup

    • -26
    • +35
    /common/src/main/java/io/netty/util/Recycler.java
Use interval instead of mask comparison for Recycler ratio (#9748)

Motivation

The recycling ratio is currently implemented by comparing with a masked

count. The mask operation is not free and also not necessary.

Modification

Change the count(s) to just iterate over the corresponding interval,

which requires only a comparison and no mask.

Also make "first time recycle" behaviour consistent and revert change to

RecyclerTest made in #9727.

Result

Less recycling overhead

    • -12
    • +18
    /common/src/main/java/io/netty/util/Recycler.java
Remove unnecessary line in Http2ClientUpgradeCodec (#9750)

Motivation:

To clean up code.

Modification:

Remove unnecessary line.

Result:

There's no functional change.

Rename SimpleChannelInboundHandler.channelRead0() to messageReceived() (#8819)

Motivation

Per javadoc in 4.1.x SimpleChannelInboundHandler:

"Please keep in mind that channelRead0(ChannelHandlerContext, I) will be

renamed to messageReceived(ChannelHandlerContext, I) in 5.0."

Modifications

Rename aforementioned method and all references/overrides.

Result

Method is renamed.

  1. … 68 more files in changeset.
Enforce ratioMask also for WeakOrderQueue (#9727)

Motivation:

At the moment we only enfore ratioMask for the Stack which means that we only guard against recycle burts when recycled from the same Thread. We should also enforce the ratioMask in the WeakOrderQueue so we also guard against the bursts when recycle from other threads.

Modifications:

- Keep counter in WeakOrderQueue to enforce ratioMask as well

- Adjust unit test

Result:

Better guard against recycle bursts which could pollute the heap unnecessary.

    • -0
    • +13
    /common/src/main/java/io/netty/util/Recycler.java
Test that `NettyBlockHoundIntegration` can be loaded with `ServiceLoader` (#9743)

Motivation:

If something is mis-configured, the "main" test will fail but it is unclear

whether it fails because the integration does not work or it wasn't applied

at all.

Also see:

https://github.com/netty/netty/issues/9738#issuecomment-548416693

Modifications:

This change adds a test that uses the same mechanism as BlockHound does

(`ServiceLoader`) and checks that `NettyBlockHoundIntegration` is present.

Result:

It is now clear whether the integration is not working or it wasn't loaded at all.

Make BlockHound tests run on Java 13 (#9742)

Motivation:

Java 13 requires special flags to be set to make BlockHound work

Modifications:

- Added jdk13 profile to `transport-blockhound-tests`

- Enabled `-XX:+AllowRedefinitionToAddDeleteMethods` on jdk13

Result:

The tests work on Java 13

    • -0
    • +12
    /transport-blockhound-tests/pom.xml
Upgrade various JDK flavors / version in our docker-compose files (#9737)

Motivation:

We should always test with the latest JDK versions on our CI.

Modifications:

Update versions

Result:

Use latest JDK versions on our CI

    • -1
    • +1
    /docker/docker-compose.centos-6.111.yaml
    • -1
    • +1
    /docker/docker-compose.centos-6.113.yaml
    • -1
    • +1
    /docker/docker-compose.centos-6.graalvm1.yaml
    • -1
    • +1
    /docker/docker-compose.centos-6.openj9111.yaml
    • -1
    • +1
    /docker/docker-compose.centos-7.111.yaml
    • -1
    • +1
    /docker/docker-compose.centos-7.113.yaml
Avoid synthetic methods in Recycler (#9736)

Motivation

Currently the visibility of the various Recycler inner classes and their

fields isn't optimal. Some private members are accessed by other classes

resulting in synthetic methods, and other non-private classes/members

are only accessed privately and so can be made private.

Modifications

- Increase/reduce visibility of various fields/methods/classes within

Recycler

- Have WeakOrderQueue extend WeakReference<Thread> to eliminate the

owner field

- Change local DefaultHandle var to DefaultHandle<?> to avoid raw type

compiler warning

Result

Tidier code, fewer implicit methods on hot paths (reducing inlining

depths)

    • -23
    • +26
    /common/src/main/java/io/netty/util/Recycler.java
Remove usage of finalizer in Recycler (#9726)

Motivation:

We currently use a finalizer to ensure we correctly return the reserved back to the Stack but this is not really needed as we can ensure we return it when needed before dropping the WeakOrderQueue

Modifications:

Use explicit method call to ensure we return the reserved space back before dropping the object

Result:

Less finalizer usage and so less work for the GC

    • -16
    • +23
    /common/src/main/java/io/netty/util/Recycler.java
Correctly update size of the Stack before doing any validation in Recycler (#9731)

Motivation:

We null out the element in the array after we decrement the current size of the Stack but not directly write back the updated size to the stored field. This is problematic as we do some validation before we write it back and so may never do so if the validation fails. This then later can lead to have null objects returned where not expected

Modifications:

Update size directly after null out object

Result:

No more unexpected null value possible

Prefer Log4J2 over Log4J1 for default InternalLoggerFactory (#9734)

##Motivation

The InternalLoggerFactory attempts to instantiate different logger

implementations to discover what is available on the class path,

accepting the first implementation that does not throw an exception.

Currently, the default ordering will attempt to instantiate a Log4j1

logger before Log4j2. For environments where both Log4j1 and Log4j2 are

available, this will result in using the older version. It seems that it

would be more intuitive to prefer the newer version, when possible.

##Modifications

Change the default ordering to attempt to use the Log4J2LoggerFactory

before the Log4JLoggerFactory.

##Result

For environments where both Log4j1 and Log4j2 are available on the class

path (but Slf4J is not available), Netty will now use Log4j2 instead of

Log4j1.

Simplify WebSocket handlers constructor arguments hell #9698 (#9699)

### Motivation:

Introduction of `WebSocketDecoderConfig` made our server-side code more elegant and simpler for support.

However there is still some problem with maintenance and new features development for WebSocket codecs (`WebSocketServerProtocolHandler`, `WebSocketServerProtocolHandler`).

Particularly, it makes me ~~crying with blood~~ extremely sad to add new parameter and yet another one constructor into these handlers, when I want to contribute new feature.

### Modification:

I've extracted all parameters for client and server WebSocket handlers into config/builder structures, like it was made for decoders in PR #9116.

### Result:

* Fixes #9698: Simplify WebSocket handlers constructor arguments hell

* Unblock further development in this module (configurable close frame handling on server-side; automatic close-frame sending, when missed; memory leaks on protocol violations; etc...)

Bonuses:

* All defaults are gathered in one place and could be easily found/reused.

* New API greatly simplifies usage, but does NOT allow inheritance or modification.

* New API would simplify long-term maintenance of WebSockets module.

### Example

WebSocketClientProtocolConfig config = WebSocketClientProtocolConfig.newBuilder()

.webSocketUri("wss://localhost:8443/fx-spot")

.subprotocol("trading")

.handshakeTimeoutMillis(15000L)

.build();

ctx.pipeline().addLast(new WebSocketClientProtocolHandler(config));

Fix Http2Headers.method(...) javadocs (#9718)

Motivation:

The javadocs of Http2Headers.method(...) are incorrect, we should fix these.

Modifications:

Correct javadocs

Result:

Fixes https://github.com/netty/netty/issues/8068.

HttpClientCodec need to keep request / response pairs in sync all the… (#9721)

Motivation:

At the moment we miss to poll the method queue when we see an Informational response code. This can lead to out-of-sync of request / response pairs when later try to compare these.

Modifications:

Always poll the queue correctly

Result:

Always compare the correct request / response pairs

Remove todo in StringEncoder (#9719)

Motivation:

StringEncoder is marked @Shareable and CharsetEncoder is not thread-safe.

Modifications:

Remove TODO.

Result:

Less technical debt.

Don't pollute FastThreadLocal for Threads with WeakHashMap if maxDelayedQueues == 0 (#9722)

Motivation:

If maxDelayedQueues == 0 we should never put any WeakHashMap into the FastThreadLocal for a Thread.

Modifications:

Check if maxDelayedQueues == 0 and if so return directly. This will ensure we never call FastThreadLocal.initialValue() in this case

Result:

Less overhead / memory usage when maxDelayedQueues == 0

Introduce MacOSDnsServerAddressStreamProvider which correctly detect all nameserver configuration on MacOS (#9161)

Motivation:

On MacOS it is not really good enough to check /etc/resolv.conf to determine the nameservers to use. We should retrieve the nameservers using the same way as mDNSResponser and chromium does by doing a JNI call.

Modifications:

Add MacOSDnsServerAddressStreamProvider and testcase

Result:

Use correct nameservers by default on MacOS.

    • -0
    • +22
    /license/LICENSE.dnsinfo.txt
    • -0
    • +185
    /resolver-dns-native-macos/pom.xml
    • -0
    • +106
    /resolver-dns-native-macos/src/main/c/dnsinfo.h
    • -0
    • +263
    /resolver-dns-native-macos/src/main/c/netty_resolver_dns_macos.c
    • -3
    • +3
    /transport-native-unix-common/src/main/c/netty_unix_socket.c
    • -0
    • +2
    /transport-native-unix-common/src/main/c/netty_unix_socket.h
Complete todo in SelfSignedCertificate (#9720)

Motivation:

Easier to debug SelfSignedCertificate failures.

Modifications:

Add first throwable as suppressed to thrown exception.

Result:

Less technical debt.

Respect all informational status codes. (#9712)

Motivation:

HTTP 102 (WebDAV) is not correctly treated as an informational response

Modification:

Delegate all `1XX` status codes to superclass, not just `100` and `101`.

Result:

Supports WebDAV response.

Removes a huge maintenance [headache](https://github.com/line/armeria/pull/2210) in Armeria which has forked the class for these features

Update to latest jdk8 release (#9717)

Motivation:

We should use latest jdk8 release to build on CI

Modifications:

Update to latest adoptjdk release

Result:

Use latest jdk8

    • -1
    • +1
    /docker/docker-compose.centos-6.18.yaml
    • -1
    • +1
    /docker/docker-compose.centos-7.18.yaml