Clone
 

roger kapsi <rkapsi@squarespace.com> in Netty

Ability to scoop up events that reach the tail of the ChannelPipeline.

Motivation

There is currently no way to enforce the position of a handler in a ChannelPipeline and assume you wanted to write something like a custom Channel type that acts as a proxy between two other Channels.

ProxyChannel(Channel client, Channel server) {

client calls write(msg) -> server.write(msg)

client calls flush() -> server.flush()

server calls fireChannelRead(msg) -> client.write(msg)

server calls fireChannelReadComplete() -> client.flush()

}

In order to make it work reliably one needs to be able to scoop up the various events at the head and tail of the pipeline. The head side of the pipeline is covered by Unsafe and it's also relatively safe to count on the user to not use the addFirst() method to manipulate the pipeline. The tail side is always at a risk of getting broken because addLast() is the goto method to add handlers.

Modifications

Adding a few extra methods to DefaultChannelPipeline that expose some of the events that reach the pipeline's TailContext.

Result

Fixes #7484

Add listener to returned Future rather than passed in Promise

Motivation

It's cleaner to add listeners to returned Futures rather than provided Promises because the latter can have strange side effects in terms of listeners firing and called methods returning. Adding listeners preemtively may yield also to more OPS than necessary when there's an Exception in the to be called method.

Modifications

Add listener to returned ChannelFuture rather than given ChannelPromise

Result

Cleaner completion and exception handling

Add listener to returned Future rather than passed in Promise

Motivation

It's cleaner to add listeners to returned Futures rather than provided Promises because the latter can have strange side effects in terms of listeners firing and called methods returning. Adding listeners preemtively may yield also to more OPS than necessary when there's an Exception in the to be called method.

Modifications

Add listener to returned ChannelFuture rather than given ChannelPromise

Result

Cleaner completion and exception handling

Handle the possibility of HTTP/0.9 with a better error message

Motivation

RFC 1945 (see section 3.1) says that request lines may not have a version in which case the request is assumed to be HTTP/0.9. We don't necessarily want to support that but the existing Exception should indicate the possibility of the request being HTTP/0.9 and give the user a chance to track it down.

Modifications

Indicate in the Exception's message that the request is possibly HTTP/0.9.

Result

Fixes #6739

Handle the possibility of HTTP/0.9 with a better error message

Motivation

RFC 1945 (see section 3.1) says that request lines may not have a version in which case the request is assumed to be HTTP/0.9. We don't necessarily want to support that but the existing Exception should indicate the possibility of the request being HTTP/0.9 and give the user a chance to track it down.

Modifications

Indicate in the Exception's message that the request is possibly HTTP/0.9.

Result

Fixes #6739

SslHandler#handlerRemoved0() shouldn't care about the SSLEngine being a specific type but only if it's ReferenceCounted

Motivation

SslHandler should release any type of SSLEngine if it implements the ReferenceCounted interface

Modifications

Change condition to check for ReferenceCounted interface

Result

Better use of interfaces

SslHandler#handlerRemoved0() shouldn't care about the SSLEngine being a specific type but only if it's ReferenceCounted

Motivation

SslHandler should release any type of SSLEngine if it implements the ReferenceCounted interface

Modifications

Change condition to check for ReferenceCounted interface

Result

Better use of interfaces

The SNI extension value is ASCII encoded but Netty uses UTF-8.

Motivation

RFC 6066 (https://tools.ietf.org/html/rfc6066#page-6) says that the hostname in the SNI extension is ASCII encoded but Netty decodes it using UTF-8.

Modifications

Use ASCII instead of UTF-8

Result

Fixes #6717

Ability to extend SniHandler and configure it with arbitrary runtime data

Motivation

SniHandler is "hardcoded" to use hostname -> SslContext mappings but there are use-cases where it's desireable and necessary to return more information than a SslContext. The only option so far has been to use a delegation pattern

Modifications

Extract parts of the existing SniHandler into an abstract base class and extend SniHandler from it. Users can do the same by extending the new abstract base class and implement custom behavior that is possibly very different from the common/default SniHandler.

Touches

- https://github.com/netty/netty/commit/f97866dbc6e7edc987de9bf7d31f563ce726833e

- https://github.com/netty/netty/commit/b604a22395f64bca52d155127e2453aa3178af6c

Result

Fixes #6603

Fixing argument names

Motivation

Misleading argument names

Modifications

Stripping xMillis suffix from arguments because there's a TimeUnit

Result

Less confusion

Fixing argument names

Motivation

Misleading argument names

Modifications

Stripping xMillis suffix from arguments because there's a TimeUnit

Result

Less confusion

Detecting actual Channel write idleness vs. slowness

Motivation

The IdleStateHandler tracks write() idleness on message granularity but does not take into consideration that the client may be just slow and has managed to consume a subset of the message's bytes in the configured period of time.

Modifications

Adding an optional configuration parameter to IdleStateHandler which tells it to observe ChannelOutboundBuffer's state.

Result

Fixes https://github.com/netty/netty/issues/6150

Detecting actual Channel write idleness vs. slowness

Motivation

The IdleStateHandler tracks write() idleness on message granularity but does not take into consideration that the client may be just slow and has managed to consume a subset of the message's bytes in the configured period of time.

Modifications

Adding an optional configuration parameter to IdleStateHandler which tells it to observe ChannelOutboundBuffer's state.

Result

Fixes https://github.com/netty/netty/issues/6150

OCSP stapling support for Netty using netty-tcnative.

https://github.com/netty/netty-tcnative/pull/215

Motivation

OCSP stapling (formally known as TLS Certificate Status Request extension) is alternative approach for checking the revocation status of X.509 Certificates. Servers can preemptively fetch the OCSP response from the CA's responder, cache it for some period of time, and pass it along during (a.k.a. staple) the TLS handshake. The client no longer has to reach out on its own to the CA to check the validity of a cetitficate. Some of the key benefits are:

1) Speed. The client doesn't have to crosscheck the certificate.

2) Efficiency. The Internet is no longer DDoS'ing the CA's OCSP responder servers.

3) Safety. Less operational dependence on the CA. Certificate owners can sustain short CA outages.

4) Privacy. The CA can lo longer track the users of a certificate.

https://en.wikipedia.org/wiki/OCSP_stapling

https://letsencrypt.org/2016/10/24/squarespace-ocsp-impl.html

Modifications

https://www.openssl.org/docs/man1.0.2/ssl/SSL_set_tlsext_status_type.html

Result

High-level API to enable OCSP stapling

    • -0
    • +81
    /example/src/main/java/io/netty/example/ocsp/Digester.java
    • -0
    • +182
    /example/src/main/java/io/netty/example/ocsp/OcspUtils.java
  1. … 7 more files in changeset.
OCSP stapling support for Netty using netty-tcnative.

https://github.com/netty/netty-tcnative/pull/215

Motivation

OCSP stapling (formally known as TLS Certificate Status Request extension) is alternative approach for checking the revocation status of X.509 Certificates. Servers can preemptively fetch the OCSP response from the CA's responder, cache it for some period of time, and pass it along during (a.k.a. staple) the TLS handshake. The client no longer has to reach out on its own to the CA to check the validity of a cetitficate. Some of the key benefits are:

1) Speed. The client doesn't have to crosscheck the certificate.

2) Efficiency. The Internet is no longer DDoS'ing the CA's OCSP responder servers.

3) Safety. Less operational dependence on the CA. Certificate owners can sustain short CA outages.

4) Privacy. The CA can lo longer track the users of a certificate.

https://en.wikipedia.org/wiki/OCSP_stapling

https://letsencrypt.org/2016/10/24/squarespace-ocsp-impl.html

Modifications

https://www.openssl.org/docs/man1.0.2/ssl/SSL_set_tlsext_status_type.html

Result

High-level API to enable OCSP stapling

    • -0
    • +81
    /example/src/main/java/io/netty/example/ocsp/Digester.java
    • -0
    • +182
    /example/src/main/java/io/netty/example/ocsp/OcspUtils.java
  1. … 7 more files in changeset.
Allow customization of LocalChannel instances that are being created by LocalServerChannel.

Motivation

It's possible to extend LocalChannel as well as LocalServerChannel but the LocalServerChannel's serve(peer) method is hardcoded to create only instances of LocalChannel.

Modifications

Add a protected factory method that returns by default new LocalChannel(...) but users may override it to customize it.

Result

It's possible to customize the LocalChannel instance on either end of the virtual connection.

Allow customization of LocalChannel instances that are being created by LocalServerChannel.

Motivation

It's possible to extend LocalChannel as well as LocalServerChannel but the LocalServerChannel's serve(peer) method is hardcoded to create only instances of LocalChannel.

Modifications

Add a protected factory method that returns by default new LocalChannel(...) but users may override it to customize it.

Result

It's possible to customize the LocalChannel instance on either end of the virtual connection.

Catch+fire Exceptions thrown from SniHandler's select() method

Motivation

Give the user the ability to back out from SNI negoations.

Modifications

Put a try-catch around the select() call and re-fire any caught Exceptions.

Result

Fixes #5787

Catch+fire Exceptions thrown from SniHandler's select() method

Motivation

Give the user the ability to back out from SNI negoations.

Modifications

Put a try-catch around the select() call and re-fire any caught Exceptions.

Result

Fixes #5787

Expose the ChannelHandlerContext from SniHandler's select() step to the user.

Motivation

I'm looking to harden our SSL impl. a little bit and add some guards agaist certain types of abuse. One can think of invalid hostname strings in the SNI extenstion or invalid SNI handshakes altogether. This will require measuring, velocity tracking and other things.

Modifications

Adding a protected `lookup(ctx, hostname)` method that is called from SniHandler's `select(...)` method which users can override and implement custom behaviour. The default implementation will simply call the AsyncMapper.

Result

It's possible to get a hold onto the ChannelHandlerContext. Users can override that method and do something with it right there or they can delegate it to something else. SniHandler is happy as long as a `Future<SslContext>` is being returned.

Unit Test for SslHandler's handlerRemoved()

Motivation

SslHandler's handlerRemoved() is supposed to release the SSLEngine (which it does) but there is no Test for it to make sure it really happens and doesn't unexpectedly change in the future.

Modifications

Add a Unit Test that makes sure that SslHandler releases the SSLEngine when the Channel gets closed.

Result

Assurance that SslHandler will not leak (ReferenceCounted) SSLEngines.

Unit Test for SslHandler's handlerRemoved()

Motivation

SslHandler's handlerRemoved() is supposed to release the SSLEngine (which it does) but there is no Test for it to make sure it really happens and doesn't unexpectedly change in the future.

Modifications

Add a Unit Test that makes sure that SslHandler releases the SSLEngine when the Channel gets closed.

Result

Assurance that SslHandler will not leak (ReferenceCounted) SSLEngines.

Expose SniHandler's replaceHandler() so that users can implement custom behavior.

Motivation

The SniHandler is currently hiding its replaceHandler() method and everything that comes with it. The user has no easy way of getting a hold onto the SslContext for the purpose of reference counting for example. The SniHandler does have getter methods for the SslContext and hostname but they're not very practical or useful. For one the SniHandler will remove itself from the pipeline and we'd have to track a reference of it externally and as we saw in #5745 it'll possibly leave its internal "selection" object with the "EMPTY_SELECTION" value (i.e. we've just lost track of the SslContext).

Modifications

Expose replaceHandler() and allow the user to override it and get a hold onto the hostname, SslContext and SslHandler that will replace the SniHandler.

Result

It's possible to get a hold onto the SslContext, the hostname and the SslHandler that is about to replace the SniHandler. Users can add additional behavior.

Fix handling of non direct backed PemEncoded.

Motivation:

The private key and certificate that are passed into #serKeyMaterial() could be PemEncoded in which case the #toPEM() methods return the identity of the value.

That in turn will fail in the #toBIO() step because the underlying ByteBuf is not necessarily direct.

Modifications:

- Use toBIO(...) which also works with non direct PemEncoded values

- Add unit test.

Result:

Correct handling of PemEncoded.

Fix handling of non direct backed PemEncoded.

Motivation:

The private key and certificate that are passed into #serKeyMaterial() could be PemEncoded in which case the #toPEM() methods return the identity of the value.

That in turn will fail in the #toBIO() step because the underlying ByteBuf is not necessarily direct.

Modifications:

- Use toBIO(...) which also works with non direct PemEncoded values

- Add unit test.

Result:

Correct handling of PemEncoded.

Adding ability omit the implicit #flush() call in EmbeddedChannel#writeOutbound() and the implicit #fireChannelReadComplete() in EmbeddedChannel#writeInbound().

Motivation

We use EmbeddedChannels to implement a ProxyChannel of some sorts that shovels

messages between a source and a destination Channel. The latter are real network

channels (such as Epoll) and they may or may not be managed in a ChannelPool. We

could fuse both ends directly together but the EmbeddedChannel provides a nice

disposable section of a ChannelPipeline that can be used to instrument the messages

that are passing through the proxy portion.

The ideal flow looks abount like this:

source#channelRead() -> proxy#writeOutbound() -> destination#write()

source#channelReadComplete() -> proxy#flushOutbound() -> destination#flush()

destination#channelRead() -> proxy#writeInbound() -> source#write()

destination#channelReadComplete() -> proxy#flushInbound() -> source#flush()

The problem is that #writeOutbound() and #writeInbound() emit surplus #flush()

and #fireChannelReadComplete() events which in turn yield to surplus #flush()

calls on both ends of the pipeline.

Modifications

Introduce a new set of write methods that reain the same sematics as the #write()

method and #flushOutbound() and #flushInbound().

Result

It's possible to implement the above ideal flow.

Fix for EmbeddedChannel#ensureOpen() and Unit Tests for it

Some PR stuff.

Adding ability omit the implicit #flush() call in EmbeddedChannel#writeOutbound() and the implicit #fireChannelReadComplete() in EmbeddedChannel#writeInbound().

Motivation

We use EmbeddedChannels to implement a ProxyChannel of some sorts that shovels

messages between a source and a destination Channel. The latter are real network

channels (such as Epoll) and they may or may not be managed in a ChannelPool. We

could fuse both ends directly together but the EmbeddedChannel provides a nice

disposable section of a ChannelPipeline that can be used to instrument the messages

that are passing through the proxy portion.

The ideal flow looks abount like this:

source#channelRead() -> proxy#writeOutbound() -> destination#write()

source#channelReadComplete() -> proxy#flushOutbound() -> destination#flush()

destination#channelRead() -> proxy#writeInbound() -> source#write()

destination#channelReadComplete() -> proxy#flushInbound() -> source#flush()

The problem is that #writeOutbound() and #writeInbound() emit surplus #flush()

and #fireChannelReadComplete() events which in turn yield to surplus #flush()

calls on both ends of the pipeline.

Modifications

Introduce a new set of write methods that reain the same sematics as the #write()

method and #flushOutbound() and #flushInbound().

Result

It's possible to implement the above ideal flow.

Fix for EmbeddedChannel#ensureOpen() and Unit Tests for it

Some PR stuff.

Rename `ChannelHandlerContext#fireUserEventTriggered()` argument from `event` to `evt` so it matches the `ChannelInboundHandler#userEventTriggered()` argument's name.

Motivation

When I override ChannelHandler methods I usually (always) refire events myself via

ChannelHandlerContext instead of relieing on calling the super method (say

`super.write(ctx, ...)`). This works great and the IDE actually auto completes/generates

the right code for it except `#fireUserEventTriggered()` and `#userEventTriggered()`

which have a mismatching argument names and I have to manually "intervene".

Modification

Rename `ChannelHandlerContext#fireUserEventTriggered()` argument from `event` to `evt`

to match its handler counterpart.

Result

The IDE's auto generated code will reference the correct variable.

Rename `ChannelHandlerContext#fireUserEventTriggered()` argument from `event` to `evt` so it matches the `ChannelInboundHandler#userEventTriggered()` argument's name.

Motivation

When I override ChannelHandler methods I usually (always) refire events myself via

ChannelHandlerContext instead of relieing on calling the super method (say

`super.write(ctx, ...)`). This works great and the IDE actually auto completes/generates

the right code for it except `#fireUserEventTriggered()` and `#userEventTriggered()`

which have a mismatching argument names and I have to manually "intervene".

Modification

Rename `ChannelHandlerContext#fireUserEventTriggered()` argument from `event` to `evt`

to match its handler counterpart.

Result

The IDE's auto generated code will reference the correct variable.

Fix for a newly intrduced bug in #5377

Motivation

This bug was introduced with #5377 and affects only users who'd like to share/cache/re-use `PemPrivateKey` and `PemX509Certificate` instances.

Modifications

Use `ByteBuf#writeBytes(src, readerIndex, length)` so that the src's readerIndex doesn't change and can consequently be used more than once.

Result

It's possible to share/cache/re-use `PemPrivateKey` and `PemX509Certificate` instances as long as their refCnt remains >= 1.