Skip to content

Commit 1ecab10

Browse files
committed
OPENNLP-1928: Document compatibility mode configuration and scope
1 parent 13b2883 commit 1ecab10

2 files changed

Lines changed: 29 additions & 9 deletions

File tree

opennlp-docs/src/docbkx/introduction.xml

Lines changed: 22 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -415,12 +415,28 @@ Arguments description:
415415
</screen>
416416
</para>
417417
<para>
418-
Accepted values are <code>LEGACY</code> and <code>CURRENT</code> (the default),
419-
case-insensitive; any other value throws an
420-
<code>IllegalArgumentException</code>. The setting is independent of
421-
<code>opennlp.whitespace.mode</code>, applies process-wide, so training and
422-
decoding of a given model must use the same value, and is scheduled for
423-
removal in 4.0.
418+
Accepted values are <code>LEGACY</code> and <code>CURRENT</code>,
419+
case-insensitive. An unset or blank property selects <code>CURRENT</code>;
420+
invalid values fail mode initialization. Set the property before OpenNLP
421+
initializes the mode, and use the same setting for training and inference.
422+
Legacy support is scheduled for removal in 4.0; selecting it logs a warning
423+
once per process.
424+
</para>
425+
<para>
426+
This setting affects only classes that explicitly consult
427+
<code>CompatibilityMode</code>. It does not restore every behavior of an
428+
earlier release: parser bracket spacing, corpus validation, and stream
429+
resource fixes are not switched back. It is independent of
430+
<code>opennlp.whitespace.mode</code>, which selects the whitespace definition.
431+
Selecting one does not select the other.
432+
</para>
433+
<para>
434+
Embedders can override the process-wide setting with
435+
<code>CompatibilityMode.setActive</code>; <code>reset()</code> reads the
436+
system property again. These are not per-model settings. Configure them
437+
before constructing or using model components, and do not switch them while
438+
requests are running. Record both modes with the training configuration;
439+
loading a model does not select them automatically.
424440
</para>
425441
</section>
426442
</section>

opennlp-docs/src/docbkx/langdetect.xml

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,13 @@ under the License.
5252
Plane (U+10000 and above) with a single space, which removes most emoji
5353
but also other supplementary characters such as CJK Extension B
5454
ideographs. Since 3.0.0 hyphens and Basic Multilingual Plane characters
55-
are kept; models trained with an earlier release were built with them
56-
blanked and get that output again under
57-
<code>-Dopennlp.compat.mode=LEGACY</code> until they are retrained.
55+
are kept, as are unpaired surrogates. Under
56+
<code>-Dopennlp.compat.mode=LEGACY</code>, each run of ASCII hyphens
57+
and code points U+D83C through U+10FC00 becomes one space instead;
58+
code points above U+10FC00 are kept. This reproduces the earlier
59+
input transformation for models that depend on it. Neither mode
60+
is an emoji-only filter. Keep the mode consistent between training
61+
and inference; see <xref linkend="intro.compatibility.mode"/>.
5862
Deprecated; use EmojiToEmoticonCharSequenceNormalizer to keep
5963
emoji as signal.</entry>
6064
</row>

0 commit comments

Comments
 (0)