Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
MODE-2735: improper formatting of full text search query constraints with other than literal RHS

  1. … 2 more files in changeset.
MODE-2734: cannot parse query bind variable names containing arguably esoteric but valid characters

  1. … 1 more file in changeset.
MODE-2564 Fixes the QueryBuilder's OFFSET and LIMIT parameter handling

  1. … 1 more file in changeset.
MODE-2564 Fixes the QueryBuilder's OFFSET and LIMIT parameter handling

  1. … 1 more file in changeset.
MODE-2546 Implements locking at a repository level The repository in ModeShape 5 will be responsible to locking all changed nodes within a transaction and then making sure it unlocks them when the transaction completes. This is contrast to ModeShape 3 and 4 who relied on the ISPN to handle this aspect. This commit also removes all locking from ISPN in order to validate that the new approach work. One consequence of the new approach is that user-transactions crossing over multiple threads will not work anymore (see https://issues.jboss.org/browse/MODE-2495).

  1. … 43 more files in changeset.
MODE-2522 Removes the Joda date-time library dependency and switches to javax.time This also changes the current ModeShape DateTime API removing a lot of the existing methods and encouraging users to use the standard JDK 1.8 API.

  1. … 28 more files in changeset.
MODE-2516 Updates JDK to 1.8 and jboss-parent to the latest version (19) This is the first significant commit of the 5.x series and contains a number of significant changes: - the naming of Maven version properties changed to adopt the standard pattern: 'version.<groupId>.<artifactId>' - build system and dependency updates so that the latest Maven plugin versions function correctly - updating source code to avoid compiler and javadoc warnings

  1. … 154 more files in changeset.
MODE-2166 Adds CAST dynamic operand for JCR-SQL2.

  1. … 12 more files in changeset.
MODE-2481 Added SNS information to the different child references implementations, allowing further optimization in the Context instance being used as well as in the SNS index computing.

  1. … 24 more files in changeset.
MODE-2151 Added support for CHILDCOUNT dynamic operand

Pretty basic support that should prove quite useful in certain situations. This may be relatively

expensive when the repository has nodes with lots of children since it requires loading the parent

node's child references in order to obtain the count. The CHILDCOUNT criteria would therefore work

much better/faster as filtering criteria in a query that already defines criteria that indexes can

use.

  1. … 17 more files in changeset.
Updated code based on the PR review: removed node key buffer from SecureSequence, fixed the shareable nodes type check and disabled/reverted the code regarding different width sequences.

  1. … 11 more files in changeset.
MODE-2209 Implemented a NodeSequence which checks the READ permission for nodes

  1. … 10 more files in changeset.
MODE-2088 Made a few corrections based upon code review feedback.

  1. … 5 more files in changeset.
MODE-2088 Eliminated the ring buffer's ability to submit entries in the same thread, which simplifies the consumers and makes them not need to be concurrent. Also changed the WorkspaceCache to register its own listeners that are better optimized for what they do. Kept the ability for the RepositoryChangeBus to have in-thread listeners (notified in the caller's thread), but these listeners only receive events via this route. Changed the JcrRepository to register various listeners directly on the bus rather than via the RepositoryCache (which was a listener and just delegated the register and unregister methods to the change bus). At this point, all tests pass successfully (multiple build passes).

  1. … 28 more files in changeset.
MODE-2188 Added management of index providers and index definitions

ModeShape now can manage index providers and index definitions via a programmatic API

that allows registering new/updated providers and adding/updating/removing index definitions.

Index definitions are stored as nodes in the system area using a structure such as

"/jcr:system/mode:indexes/{providerName}/{indexDefinition}/{indexColumnDefinition}".

Changes on one process in the cluster will be persisted to the cace store, and events

on all processes in the cluster will signal a refreshing of a snapshot of the index

definitions used by the query system. Additionally, when changes are made to

an index definition, the provider that owns that definition will be notified of the

new/changed or removed index definition.

Index provider information is not stored in the system area, but rather is used to

instantiate the providers in the same way as sequencers and connectors. Thus,

the repository configuration for each process in the cluster needs to define the

index providers; no information is shared across processes. Notice that all process-specific

information about indexes must be defined on the provider. For example, a notional

provider that stores indexes on the file system would likely require as a configuration

setting the directory in which those files are stored, and this could vary in each

process' configuration. BTW, this maintains ModeShape's convention of using the same

repository configuration file on each process in the cluster, albeit with variables

for any configuration parameters that must vary for the different processes.

Every index definition explicitly states the name of the provider that owns it.

If there is no such provider, ModeShape will mark the index definition as 'disabled'

and it will not be used on that process.

At this time it is not yet possible to define index providers or index definitions

in the repository configuration JSON file or the Wildfly subsystem. This will be

done with a future commit.

However, it is possible with this commit to programmatically register a provider

and define multiple indexes. Testing will also be done in a future commit as a

practical and usable provider is available.

The package and most of the interfaces in the index SPI introduced in 4.0.0.Alpha1

(e.g., formerly 'org.modeshape.jcr.spi.query') were renamed to be shorter and

more appropriate for their current & planned scope. For example, the "QueryIndexProvider"

interface is now simply "IndexProvider".

    • -1
    • +3
    ./optimize/AddJoinConditionColumnsToSourcesTest.java
    • -1
    • +3
    ./optimize/ChooseJoinAlgorithmTest.java
    • -1
    • +3
    ./optimize/PushSelectCriteriaTest.java
    • -1
    • +3
    ./optimize/RewriteAsRangeCriteriaTest.java
    • -1
    • +3
    ./optimize/RightOuterToLeftOuterJoinsTest.java
    • -1
    • +2
    ./optimize/RuleBasedOptimizerTest.java
  1. … 64 more files in changeset.
MODE-2018 Made several improvements following a code review

    • -70
    • +70
    ./engine/process/HashJoinSequenceTest.java
    • -6
    • +7
    ./engine/process/SortingSequenceTest.java
  1. … 18 more files in changeset.
MODE-2018 Upgraded MapDB to 0.9.11

Upgraded the MapDB dependency and fixed an apparent memory leak by moving the

BufferManager (used in the new query engine) to be owned by the session, ensuring

that it is garbage collected properly when the session is closed.

Also changed how the BufferManager allocates DB instances to be more lazy, since

MapDB 0.9.11 appears to have more overhead.

  1. … 6 more files in changeset.
MODE-2018 Implemented new query engine.

Refactored the query functionality to now use several new service provider interfaces (SPI),

and implemented a new query engine that can take advantage of administrator-defined indexes.

When no such indexes are defined, the query engine is able to still answer the queries

by "scanning" all nodes in the repository. This is like a regular relational database:

all query functionality works (albeith slowly) even when no indexes are defined, though

to improve performance simply define an appropriate index based upon the query or queries

that are being used.

All of ModeShape's query parsing, planning, and optimization steps are basically unchanged

from the previous query system. There is one addition to the rule-based optimizer: a new

rule looks at query plans and adds the potential indexes that might be of use in each

access query portion of a query plan. Then, the query execution process (see below)

chooses one of the identified indexes based upon the selectivity and cardinality. If no index

is available for that portion of the query plan, then the query engine simply iterates

over all queryable nodes in the repository.

A new kind of component, called a "query index provider", allows the query engine to delegate

various responsibilities around indexes to these providers. For example, a provider must

provide an index planner that can examine the constraints that apply to an access query

and determine if any of the provider's indexes can be used. When they are, ModeShape

adds those indexes to the query plan. If the query engine uses one of those indexes,

then provider must be able to return all of those nodes that satisfy the criteria

as described earlier by its index planner. Finally, as ModeShape content changes, ModeShape

will notify the index providers' of the changes so that they can ensure their indexes

are kept up-to-date with the content.

This means that a provider can implement the functionality using any kind of technology,

and consequently, that ModeShape can begin to leverage multiple kinds of search and index

technology within its query system. The ModeShape community anticipates having providers

that use Lucene, Solr, and ElasticSearch. ModeShape will also likely come with a provider

that maintains file-system based indexes. Additionally, providers can optionally support

indexes on one or more properties. Thus, it will be possible to mix and match

these providers, selecting the best technology for the specific kind of index.

The new query engine does the execution in a very different way than the previous engine,

which used Lucene to determine the tuples (that is, the values in each row) for each access

query and that were then further processed and combined to form the tuples that were returned

in the result set. The new engine instead uses a new concept of a "stream of node keys"

for each access query: what actually implements that stream depends on many factors.

A node sequence is an abstraction of a stream of "rows" containing one or more node keys.

The interfaces are designed to make it possibly to lazily implement a stream in a very

efficient manner. Specifically, a node stream is actually comprised of multiple "batches"

of rows, and batches can be of any size.

Consider when the engine findes no indexes are available for a certain access query. The

engine simply uses a "node sequence" (or NodeSequence) implementation that returns in batches

a row for each node in the repository.

But if an access query involves a criteria on the path of a node, such as

"... WHERE ISSAMENODE('/foo/bar') ...", then ModeShape knows that this query (or portion of

a query) will have only one result, namely the node at "/foo/bar". ModeShape doesn't need

an index to quickly find this node; it merely has to navigate to that path to find the one

node that satisfies this query. ModeShape has several other optimizations, too: it knows

when a query involves all children or descendants of a node at a given path, and can take

this into account when optimizing and executing the query. All of these are handled with

special NodeSequence implementations optimized for each case.

For many access queries (i.e., part of a larger query), the engine will use one of the

indexes identified by one of the providers. When this happens, ModeShape uses other

NodeSequence implementations that utilize the underlying indexes to find the nodes that satisfy

some of the criteria.

The above describes how the engine uses a single NodeSequence instance for each each access

query in a larger query. But how does the engine combine these to determine the ultimate

query results? Basically, the engine constructs a series of functions that process one or more

NodeSequence instances to filter and combine into other NodeSequences.

For example, a custom index might be used to find all nodes that have a 'jcr:lastModified'

timestamp within some range. Presumably this index is used because it has a higher selectivity,

meaning that it will filter out more nodes and return fewer nodes than other indexes.

Other criteria that are also applied to this access query might then be applied by a filter

that processes the actual nodes' property values.

While the result of this commit is a functioning query engine that is shown to work in most

of the query-related unit and integration tests, there still are a few areas that are not complete.

Specifically:

* The new engine does not support full-text search, and currently throws an exception

* No index providers are implemented. Therefore, all queries involve "scanning" the repository.

This can be time consuming, especially for federated repositories. Consequently, all such

tests that query federated content have been disabled/ignored.

    • -0
    • +213
    ./AbstractNodeSequenceTest.java
    • -0
    • +196
    ./BufferManagerTest.java
    • -0
    • +142
    ./NodeSequenceTest.java
    • -0
    • +86
    ./engine/process/DistinctSequenceTest.java
    • -0
    • +367
    ./engine/process/HashJoinSequenceTest.java
    • -0
    • +95
    ./engine/process/RestartableSequenceTest.java
    • -0
    • +148
    ./engine/process/SortingSequenceTest.java
    • -0
    • +298
    ./index/QueryEngineTest.java
    • -1
    • +3
    ./optimize/AddJoinConditionColumnsToSourcesTest.java
    • -1
    • +3
    ./optimize/ChooseJoinAlgorithmTest.java
    • -1
    • +3
    ./optimize/PushSelectCriteriaTest.java
  1. … 217 more files in changeset.
MODE-2081 Changed the remaining files over to the ASL 2.0 license

    • -18
    • +10
    ./model/AbstractQueryObjectTest.java
    • -18
    • +10
    ./optimize/AddAccessNodesTest.java
    • -18
    • +10
    ./optimize/AddJoinConditionColumnsToSourcesTest.java
    • -18
    • +10
    ./optimize/ChooseJoinAlgorithmTest.java
    • -18
    • +10
    ./optimize/PushSelectCriteriaTest.java
    • -18
    • +10
    ./optimize/ReplaceViewsTest.java
    • -18
    • +10
    ./optimize/RewriteAsRangeCriteriaTest.java
    • -18
    • +10
    ./optimize/RightOuterToLeftOuterJoinsTest.java
    • -18
    • +10
    ./optimize/RuleBasedOptimizerTest.java
  1. … 1036 more files in changeset.
MODE-2041 Corrected numerous compiler warninings, JavaDoc errors and warnings, and removed quite a few JavaDoc comments that are inherited via @Override.

  1. … 79 more files in changeset.
MODE-2037: Extended like operation (reverse like implementation)

Added a reverse like (e.g., "RELIKE") constraint that is useful when

the LIKE pattern is stored in a node property and the intent is to

find all nodes that have a pattern that matches a given string:

SELECT *

FROM [service:Locator] AS locator

WHERE relike($phone, locator.[service:phonePattern])

  1. … 15 more files in changeset.
MODE-1680 The xpath query is not well-formed

    • -0
    • +7
    ./xpath/XPathToQueryTranslatorTest.java
  1. … 3 more files in changeset.
MODE-1594 Added test case for JCR-SQL2 queries that were problematic in 2.x

    • -0
    • +15
    ./parse/JcrSql2QueryParserTest.java
MODE-1608 - Fixed the code for Except and Intersect components when the tuples iterators have no more results

Also added test case. These were essentially the same changes for MODE-1595 (for 2.x).

    • -0
    • +191
    ./process/SetOperationComponentsTest.java
  1. … 2 more files in changeset.
MODE-1594 - Fixed ClassCastException for SQL2 queries involving the UNION, INTERSECT, or EXCEPT operators

  1. … 1 more file in changeset.
MODE-1095 Fix support for ORDER BY using a column that isn't in the SELECT

The query framework was changed to ensure all columns used in the ORDER BY clauses

are automatically included in the results. Because the query processing engine

requires the columns to be in the (intermediate/internal) results, including the

columns in the user's result set does not make the query system less efficient.

(In fact, changing the result set to contain/expose only those columns included

in the SELECT clause would actually require additional resources and could be

potentially less efficient.)

This change implements the functionality by adding a new optimizer rule that

rewrites part of the query plan. In particular, the new rule looks for SORT query

plan nodes, processes the Ordering instances to extract the property references,

and adds corresponding columns to all the appropriate PROJECT nodes that are below

the SORT node.

This change altered the query plans for queries with orderings, so the expected

results needed to be changed in several test cases.

    • -8
    • +8
    ./optimize/RuleBasedOptimizerTest.java
  1. … 8 more files in changeset.
MODE-1556 Corrected the handling of != (or <>) query criteria on numeric fields

Numeric values were not being handled consistently in the indexes and in queries. For example,

some values were being treated as integer values in the indexes and queried as longs (which

doesn't work). Additionally, some of the logic of handling and combining "not" queries was

incorrect.

All unit and integration tests pass with these changes.

    • -8
    • +6
    ./xpath/XPathToQueryTranslatorTest.java
  1. … 7 more files in changeset.
MODE-1549 JCR-SQL2 uses '<>' rather than '!=' as not-equal-to operator

ModeShape abstract syntax tree used '!=' as the "not-equal-to" operator, but did

alias '<>' to '!='. This change reverses this so that the '<>' token is considered

the primary operator, and '!=' is aliased.

  1. … 1 more file in changeset.
MODE-1496 JCR-3313 Added fixed JCR TCK test and corrected result set column behavior

Corrected the behavior of query result column names when the query contains a wildcard in the SELECT statement.

    • -2
    • +13
    ./optimize/RuleBasedOptimizerTest.java
  1. … 10 more files in changeset.
MODE-1475 Corrected JQOM query behavior and added TCK tests

Corrected a number of issues with JQOM queries, including how the corresponding string statements

are generated so that they are now valid JCR-SQL2 statements. Also added the JQOM TCK tests

(not sure why they were excluded), which identified quite a few more. These are all now fixed,

excluding several known and logged issues with the TCK tests themselves.

All unit and integration tests pass with these changes.

  1. … 69 more files in changeset.