Clone Tools
  • last updated a few minutes ago
Constraints
Constraints: committers
 
Constraints: files
Constraints: dates
MODE-2615 Fixes the handling of multi-byte UTF-8 characters by the BSON reader

  1. … 1 more file in changeset.
MODE-2562 Fixes the parsing of backslash characters by the JsonReader

  1. … 7 more files in changeset.
MODE-2527 Renames the schematic package to org.modeshape and removes all ISPN related code.

    • -563
    • +0
    ./BsonReadingAndWritingTest.java
  1. … 301 more files in changeset.
MODE-2437 Removed Infinispan test-jar dependencies and reduced as much as possible the regular ISPN API dependencies throughout the code.

  1. … 59 more files in changeset.
MODE-2430 Fixed BsonDataInput stream truncating strings.

  1. … 4 more files in changeset.
MODE-2323 Removed the proxy layer and delta write functionality in Schematic

ModeShape's is no longer using Infinispan in ways where these pieces of functionality are useful.

Removing them should simplify the logic to represent more of what we're actually doing.

    • -148
    • +0
    ./OperationExternalizerTest.java
  1. … 45 more files in changeset.
MODE-2317 Updated the JsonReader to support regular character sequences, even if they begin with \u.

  1. … 3 more files in changeset.
MODE-2317 Updated the JsonReader to support regular character sequences, even if they begin with \u.

Conflicts:

modeshape-schematic/src/test/java/org/infinispan/schematic/internal/document/JsonReaderTest.java

  1. … 3 more files in changeset.
MODE-2309 Corrected handling of non-ASCII characters when reading and writing JSON via binary streams by enforcing UTF-8

  1. … 3 more files in changeset.
MODE-2309 Corrected handling of non-ASCII characters when reading and writing JSON via binary streams by enforcing UTF-8

  1. … 3 more files in changeset.
MODE-2309 Corrected reading and writing JSON files when escaped characters are used.

This fixes the backup and restore issue. Note that ModeShape uses BSON when storing nodes in Infinispan, so this

should not affect persistent storage in a cache store. ModeShape uses JSON for the file connector (extra properties)

and for the REST service; neither of these should be impacted negatively by this change, though like the backup

and restore this change should fix any as-of-yet unseen issues related to escaped characters.

Note that it is not possible to just update the Restore functionality, so with this change new backups will have

to be created. This change does need to be applied to the 3.x branch as well.

    • -0
    • +97
    ./JsonWriterTest.java
  1. … 6 more files in changeset.
MODE-2309 Corrected reading and writing JSON files when escaped characters are used.

This fixes the backup and restore issue. Note that ModeShape uses BSON when storing nodes in Infinispan, so this

should not affect persistent storage in a cache store. ModeShape uses JSON for the file connector (extra properties)

and for the REST service; neither of these should be impacted negatively by this change, though like the backup

and restore this change should fix any as-of-yet unseen issues related to escaped characters.

Note that it is not possible to just update the Restore functionality, so with this change new backups will have

to be created. This change does need to be applied to the 3.x branch as well.

Conflicts:

modeshape-jcr/src/test/java/org/modeshape/jcr/RepositoryBackupAndRestoreTest.java

modeshape-schematic/src/test/java/org/infinispan/schematic/internal/document/JsonReaderTest.java

    • -0
    • +97
    ./JsonWriterTest.java
  1. … 6 more files in changeset.
MODE-2214 Fixed JSON array parsing and corrected invalid files use throughout our tests.

  1. … 5 more files in changeset.
Corrected compiler and JavaDoc warnings

  1. … 25 more files in changeset.
MODE-2082 Added support for single-line comments in JSON files.

  1. … 5 more files in changeset.
MODE-2066 Migrated to Infinispan 6.0.1 and JGroups 3.4.1. Beside the changes involving package migrations and API changes, because the BDB and JDBM cachestore have not been ported to 6.x yet, all references (only unit tests) to those cache stores were removed and replaced with LevelDB. The old FileCacheStore has been discontinued and therefore replaced with SingleFileStore.

  1. … 98 more files in changeset.
MODE-2081 Changed the license for ModeShape code to ASL 2.0.

    • -16
    • +10
    ./BsonReadingAndWritingTest.java
    • -18
    • +10
    ./OperationExternalizerTest.java
  1. … 108 more files in changeset.
MODE-2074 Fixed the usage of CharSet decoders/encoders by the BsonDataInput/Output classes. Also, fixed the algorithm which reads Bson strings, in the case when those strings are larger than the default buffer size.

  1. … 4 more files in changeset.
Schematic improvement to add ability to merge documents

    • -0
    • +50
    ./DocumentEditorTest.java
  1. … 6 more files in changeset.
MODE-1745 Corrections to Schematic's state transfer

Several issues related to state-transfer were identified and corrected within Schematic's DeltaAware implementation. These primarily dealt with using diff-based deltas when using Infinispan 5.2.0 (tested with Beta6 and later), the implementations of the two deltas (and related classes), and how merges are made, and removal of the NullDelta class.

With these changes, the test projects in MODE-1745 and MODE-1746 work better. There still are problems with state transfer during repository initialization. See MODE-1745 for details.

  1. … 10 more files in changeset.
MODE-1733 Changed SchematicEntryDelta behavior

The original problem is that the delta is being merged to a null entry

reference when the entry has been evicted from the cache. The correct

behavior would be to look up the persisted entry and merge the detal

with that. Infinispan is not behaving correctly.

This problem appears to be very similar to ISPN-2095, though we're

not using AtomicHashMap but have our own DeltaAware implementation

(called SchematicEntry).

The fix for ISPN-2095 involved changing AtomicHashMapProxy to use a

specific "DELTA_WRITE" cache flag that was added in Infinispan

5.1.6.FINAL. That means we don't have access to the fix in 5.1.2.FINAL,

so we need to provide an alternative solution for pre-5.1.6.FINAL releases.

The CacheContext now attempts to set that flag only when we're using

Infinispan 5.1.6 or later (e.g., if we can find the "DELTA_WRITE" flag),

but the changes still don't seem to correct the behavior with

Infinispan 5.1.2.FINAL, 5.1.6.FINAL or 5.1.8.Final. Even if it

worked on 5.1.6 or later, we would still have the issue when using

5.1.2.FINAL (which is all AS7.1.1 installations).

We need a solution that works with Infinispan 5.1.2 or later.

One option was to make the SchematicEntryLiteral instances no longer

implement DeltaAware. However, that means that we'd be changing the

class for the values we're storing in Infinispan. It also means that

we could not use DeltaAware in some environments (for example, when

we're using newer versions of Infinispan).

A better approach taken with this commit is to use different Delta

implementations depending upon the configuration. With this commit

there are two Delta implementations: one that works like before by

recording the Operations used to manipulate the documents, and a new

one that simply contains the whole document.

The benefit of this approach is that we can then choose which Delta

implementation we use based upon the Infinispan configuration and version:

- In local (non-clustered) caches, we actually don't need to compute a

Delta. In this case we can use the whole document Delta and a

different DocumentEditor that doesn't actually create any operations.

- In clustered caches that do not use eviction, we can use the

diff-based Delta with an editor that does calculate the change

operations. These Delta instances are correctly merged.

- In clustered caches that do use eviction, we need to use the

whole-document Delta. And since this delta doesn't record operations,

we could use a DocumentEditor that doesn't actually create operations.

All tests (including several new clustering tests) pass with these

changes. Additionally, the clustering test attached by the reporter

to MODE-1733 also passes with these changes, when using either

Infinispan 5.1.2.FINAL or 5.1.8.Final.

  1. … 13 more files in changeset.
MODE-1703 Corrected serialization of Schematic deltas

Array objects were not being serialized/deserialized properly

within the delta's operation instances (e.g., PutOperation).

Normally, all SchematicEntry (e.g., Documents containing values

such as Arrays and other Documents) are serialized using BSON, and

this works correctly in the non-clustered case. However, in the

clustered case, the changes made to one Document are recorded to

create a SchematicEntryDelta that is shipped across the Infinispan

cluster. (This way we can ship only the changes to the

SchematicEntry rather than the whole document.) Thus the problem

only manifest itself in clustered configurations.

The actual problem stemmed from the fact that an

SchematicEntryDelta and its Operation objects are all

externalizable (with custom Externalizers). The Operation objects

can contain individual values, Document or Array objects. These are

also externalizable, but there was a single DocumentExternalizer

for both Document and Array objects and the DocumentExternalizer

wrote/read both to/from BSON.

However, the BSON format (which is simply a binary representation

of JSON documents, with a few extra value types) is only valid for

Documents; a standalone Array cannot be written in BSON format.

The solution was to use the DocumentExternalizer for Documents, but

a new ArrayExternalizer for Arrays. This ArrayExternalizer still

uses BSON, but it uses a subset of the BSON representation to

serialize Array objects. But, it does work well and fixes the

problem.

Quite a few new tests were added to verify that the various

operations with various parameters are indeed externalizable. The

tests don't cover all the permuations, but they do attempt to cover

the different kinds of operations with documents and arrays.

To make the Operation objects easily testable, equals and hashCode

methods were added to all the Operation subclasses. Additionally,

all of the externalizers were manually reviewed for asymmetric read

and write methods (e.g, writing an object to the ObjectOutputStream

but reading an integer from the ObjectInputStream), and several

asymmetries were found in the Externalizers for BSON value types

that ModeShape doesn't currently use, and these asymmetries were

corrected.

All unit and integration tests pass with these changes.

    • -0
    • +140
    ./AbstractExternalizerTest.java
    • -0
    • +57
    ./ArrayExternalizerTest.java
    • -0
    • +60
    ./DocumentExternalizerTest.java
    • -0
    • +153
    ./OperationExternalizerTest.java
  1. … 23 more files in changeset.
MODE-1703 Corrected serialization of Schematic deltas

Array objects were not being serialized/deserialized properly

within the delta's operation instances (e.g., PutOperation).

Normally, all SchematicEntry (e.g., Documents containing values

such as Arrays and other Documents) are serialized using BSON, and

this works correctly in the non-clustered case. However, in the

clustered case, the changes made to one Document are recorded to

create a SchematicEntryDelta that is shipped across the Infinispan

cluster. (This way we can ship only the changes to the

SchematicEntry rather than the whole document.) Thus the problem

only manifest itself in clustered configurations.

The actual problem stemmed from the fact that an

SchematicEntryDelta and its Operation objects are all

externalizable (with custom Externalizers). The Operation objects

can contain individual values, Document or Array objects. These are

also externalizable, but there was a single DocumentExternalizer

for both Document and Array objects and the DocumentExternalizer

wrote/read both to/from BSON.

However, the BSON format (which is simply a binary representation

of JSON documents, with a few extra value types) is only valid for

Documents; a standalone Array cannot be written in BSON format.

The solution was to use the DocumentExternalizer for Documents, but

a new ArrayExternalizer for Arrays. This ArrayExternalizer still

uses BSON, but it uses a subset of the BSON representation to

serialize Array objects. But, it does work well and fixes the

problem.

Quite a few new tests were added to verify that the various

operations with various parameters are indeed externalizable. The

tests don't cover all the permuations, but they do attempt to cover

the different kinds of operations with documents and arrays.

To make the Operation objects easily testable, equals and hashCode

methods were added to all the Operation subclasses. Additionally,

all of the externalizers were manually reviewed for asymmetric read

and write methods (e.g, writing an object to the ObjectOutputStream

but reading an integer from the ObjectInputStream), and several

asymmetries were found in the Externalizers for BSON value types

that ModeShape doesn't currently use, and these asymmetries were

corrected.

All unit and integration tests pass with these changes.

    • -0
    • +140
    ./AbstractExternalizerTest.java
    • -0
    • +57
    ./ArrayExternalizerTest.java
    • -0
    • +60
    ./DocumentExternalizerTest.java
    • -0
    • +153
    ./OperationExternalizerTest.java
  1. … 23 more files in changeset.
MODE-1674 Changed restore test case to use RAM indexes

Even though the test configuration stores content on the file system, it can store the indexes

in RAM because the repository is restarted following the restoration, and the restart will

rebuild the indexes each time.

Also added a query to the test, checking that the content is properly indexed (and indirectly loaded).

  1. … 3 more files in changeset.
MODE-1674 Corrected several issues with backup/restore, the JSON reader/writer, and tests

Changed the backup and restore tests to use file-based persisted repositories rather than

in-memory repositories. When the repositories are restarted during the restore process,

restarting an in-memory repository (and cache) loses all data that was successfully restored.

One issue with the backup and restore process was that, even upon restore, the RepositoryCache's

sourceKey and repositoryKey were generated from the configuration's name and cache name. So

if the restored repository had a different name than the backup or the repository's cache was

named something different, the restored repository would never find the top-level nodes in the

workspaces, and the restored repository would appear to contain no content (even though the cache

actually did contain all the restored entries). This was corrected by using a new

"repository:info" document with metadata about the repository, including the source name and key,

the cache name, the repository name and key, the date the repository was created, and the

version of ModeShape that was used to create the repository. Because this "repository:info"

document is stored in the cache (with a well-known key), the document is included in the backups

and the document can be found upon restore. Note that 3.0.0.Final (and later) will create this

"repository:info" document if not found, so any pre-3.0.0.Final repository would have to be

at least started using 3.0.0.Final (or later) before a backup can be created.

Also discovered and fixed several other issues in the way backups were being read. One of these

had to do with escaped characters in JSON field values (which were not being processed correctly),

and that documents written to JSON were not properly escaping quotes and control characters

that appeared *within* string values. Finally, the JSON reader code (that was added for restore)

that read a file containing multiple JSON documents was not correctly handling the end of the stream.

After all these changes, a full build completes successfully.

  1. … 15 more files in changeset.
MODE-1575 - Fixed deserialization of schematic Binary values

  1. … 2 more files in changeset.
MODE-1515 - Removed chances of possible negative value due to overflow

MODE-1435 Upgraded MongoDB Java driver used to test JSON/BSON I/O

The MongoDB driver is used in some of Schematic's test cases to validate the I/O of the JSON and BSON readers and writers. The version of the MongoDB driver was upgraded to the latest; since this is only used for tests, it has no influence on any non-test artifacts.

  1. … 1 more file in changeset.
MODE-1375 Added support for system property variables in repository configuration JSON documents

The JSON Schema validation logic already looked for cases where a field value object does

not have the correct type expected in the JSON Schema; such situations were reported as

errors (just like the other errors). This was changed so that, in these situations, the

schema validator tries to convert the actual value to the expected type; if that doesn't work,

the error is reported as usual, but if it can be converted, a special "mismatched type" error

is reported instead and includes the actual value, the converted value, the actual type,

and the converted type. (To public users, this is merely just a regular error, though.)

Also added a method to create a copy of a Document using a supplied `ValueTransformer` instance,

which the copy logic uses to transform each of the values within the Document. This copy

logic returns the same Document instance (not a copy) if no values were transformed, but

returns a new Document object if at least one value was transformed.

Added another method to create a copy of a Document using the validation results, where

we look for all of the "mismatched type" errors and use the converted value in place of the

actual value. (This process includes nested documents and arrays.)

Then the RepositoryConfiguration constructors were modified to try to create a copy

of the supplied document using a SystemPropertyTransformer; if a new Document were created

(because variables were replaced), then the new Document is validated against the schema

and any "mismatched type" values are used to create a copy of the document with the

mismatched values converted to the expected values. The RepositoryConfiguration then

uses the new Document with converted values, and the user can continue to validate

the configuration using the existing "validate()" method. This is where any problems

with the system properties will show up to the user. For example, if a variable is

replaced with a system property that cannot be converted to the type expected by the

JSON Schema for that field, a normal validation error will be included in the validation

results.

In general, all this happens transparently to RepositoryConfiguration users.

  1. … 27 more files in changeset.
MODE-1289 New approach for storing/caching JCR content

This is the first commit to start the 3.0 effort, which involves a major change to how

the JCR layer stores and caches information. The new approach is based upon Infinispan and uses

Infinispan's cache loaders for persistence, and JSON-like documents (that are in-memory

structures not needing to parsed/written) are used to store information for each node.

There are several new Maven modules:

- modeshape-jcr-redux

- modeshape-schematic

The 'modeshape-jcr-redux' module will eventually replace the 'modeshape-jcr' module once

the implementation is far-enough along. And the 'modeshape-schematic' module will likely

move into the Infinispan project, so that needs to remain separate.

Although it may seem strange and unkempt to have the new JCR implementation in a new module,

doing so means that we can continue to rebase from 'master' (and the 2.7 work) for at least

some time. When the new module becomes complete enough, we'll move it and replace the

existing 'modeshape-jcr' module. It's also convenient to have both the old and new implementations

around in the same codebase.

The build was changed to focus upon the (few) modules that are oriented around the new

implementation. So the following can be used to build the newer codebase:

mvn clean install

However, the build has a new Maven profile called "legacy" that can be used to build the

old modules. We kept this to make sure that any rebasing can be compiled and verified.

For example, to build everyhing, including the new modules and the 2.x-style modules,

use the following command:

mvn clean install -Plegacy

As the newer 'modeshape-jcr-redux' progresses and other modules (e.g., sequencers, web,

jboss, text extractors) are converted to use the new module, they should be moved

from the 'legacy' profile into the main set of modules in the top-level 'pom.xml'

    • -0
    • +53
    ./BasicArrayTest.java
    • -0
    • +64
    ./BasicDocumentTest.java
    • -0
    • +466
    ./BsonReadingAndWritingTest.java
    • -0
    • +248
    ./CompactJsonWriterTest.java
    • -0
    • +225
    ./JsonPerformanceTest.java
    • -0
    • +307
    ./JsonReaderParserTest.java
    • -0
    • +97
    ./JsonReaderTest.java
    • -0
    • +93
    ./PrettyJsonWriterTest.java
  1. … 440 more files in changeset.