Random failing test: TestICUNormalizer2Filter.TestRandomStrings

[paulirwin]

Is there an existing issue for this?

• I have searched the existing issues

Describe the bug

Failed on GitHub actions for PR #1134, windows net6.0 x64:

Failed Lucene.Net.Analysis.Icu.TestICUNormalizer2Filter.TestRandomStrings [1 s]
  Error Message:
   Lucene.Net.Diagnostics.AssertionException : End() called before IncrementToken() returned false!
Data:
  OriginalMessage: Lucene.Net.Diagnostics.AssertionException: End() called before IncrementToken() returned false!
   at Lucene.Net.Analysis.MockTokenizer.End() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\MockTokenizer.cs:line 337
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AssertTokenStreamContents(TokenStream ts, String[] output, Int32[] startOffsets, Int32[] endOffsets, String[] types, Int32[] posIncrements, Int32[] posLengths, Nullable`1 finalOffset, Nullable`1 finalPosInc, Boolean[] keywordAtts, Boolean offsetsAreCorrect, Byte[][] payloads) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 393
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckAnalysisConsistency(Random random, Analyzer a, Boolean useCharFilter, String text, Boolean offsetsAreCorrect, Field field) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 1164
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations, Int32 maxWordLength, Boolean useCharFilter, Boolean simple, Boolean offsetsAreCorrect, RandomIndexWriter iw) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 848
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AnalysisThread.Run() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 655
   at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)

  1) Expected: アルファ20日レムsrsrcd24点psgbリラホーンオームダースbジ, Actual: アルファ20日レムsrsrcd24点psgbリラホーンオームダースdb

term 0, output[i] = アルファ20日レムsrsrcd24点psgbリラホーンオームダースbジ, termAtt = アルファ20日レムsrsrcd24点psgbリラホーンオームダースdb

(Test: Lucene.Net.Analysis.Icu.TestICUNormalizer2Filter.TestRandomStrings)

To reproduce this test result:

Option 1:

 Apply the following assembly-level attributes:

[assembly: Lucene.Net.Util.RandomSeed("0x87a849605d9a63fd:0xca6ed2feac1a159a")]
[assembly: NUnit.Framework.SetCulture("st")]

Option 2:

 Use the following .runsettings file:

<RunSettings>
  <TestRunParameters>
    <Parameter name="tests:seed" value="0x87a849605d9a63fd:0xca6ed2feac1a159a" />
    <Parameter name="tests:culture" value="st" />
  </TestRunParameters>
</RunSettings>

Option 3:

 Create the following lucene.testsettings.json file somewhere between the test assembly and the root of your drive:

{
  "tests": {
     "seed": "0x87a849605d9a63fd:0xca6ed2feac1a159a",
     "culture": "st"
  }
}

Fixture Test Values
=================

 Random Seed:           0x87a849605d9a63fd:0xca6ed2feac1a159a
 Culture:               st
 Time Zone:             (UTC-09:00) Coordinated Universal Time-09
 Default Codec:         Lucene42 (Lucene42RWCodec)
 Default Similarity:    RandomSimilarityProvider(queryNorm=False,coord=crazy): {}

System Properties
=================

 Nightly:               False
 Weekly:                False
 Slow:                  False
 Awaits Fix:            False
 Directory:             random
 Verbose:               False
 Random Multiplier:     1

  Stack Trace:
     at Lucene.Net.Analysis.MockTokenizer.End() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\MockTokenizer.cs:line 337
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AssertTokenStreamContents(TokenStream ts, String[] output, Int32[] startOffsets, Int32[] endOffsets, String[] types, Int32[] posIncrements, Int32[] posLengths, Nullable`1 finalOffset, Nullable`1 finalPosInc, Boolean[] keywordAtts, Boolean offsetsAreCorrect, Byte[][] payloads) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 393
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckAnalysisConsistency(Random random, Analyzer a, Boolean useCharFilter, String text, Boolean offsetsAreCorrect, Field field) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 1164
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations, Int32 maxWordLength, Boolean useCharFilter, Boolean simple, Boolean offsetsAreCorrect, RandomIndexWriter iw) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 848
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AnalysisThread.Run() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 655
   at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)
--- End of stack trace from previous location ---
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations, Int32 maxWordLength, Boolean simple, Boolean offsetsAreCorrect) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 723
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 595
   at Lucene.Net.Analysis.Icu.TestICUNormalizer2Filter.TestRandomStrings() in D:\a\lucenenet\lucenenet\src\Lucene.Net.Tests.Analysis.ICU\Analysis\Icu\TestICUNormalizer2Filter.cs:line 93
1)    at Lucene.Net.Analysis.BaseTokenStreamTestCase.AssertTokenStreamContents(TokenStream ts, String[] output, Int32[] startOffsets, Int32[] endOffsets, String[] types, Int32[] posIncrements, Int32[] posLengths, Nullable`1 finalOffset, Nullable`1 finalPosInc, Boolean[] keywordAtts, Boolean offsetsAreCorrect, Byte[][] payloads) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 126
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckAnalysisConsistency(Random random, Analyzer a, Boolean useCharFilter, String text, Boolean offsetsAreCorrect, Field field) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 1164
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations, Int32 maxWordLength, Boolean useCharFilter, Boolean simple, Boolean offsetsAreCorrect, RandomIndexWriter iw) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 848
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AnalysisThread.Run() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 655
   at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)
   at J2N.Threading.ThreadJob.<.ctor>b__4_0()
   at System.Threading.Thread.StartHelper.Callback(Object state)

2)    at Lucene.Net.Analysis.MockTokenizer.End() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\MockTokenizer.cs:line 337
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AssertTokenStreamContents(TokenStream ts, String[] output, Int32[] startOffsets, Int32[] endOffsets, String[] types, Int32[] posIncrements, Int32[] posLengths, Nullable`1 finalOffset, Nullable`1 finalPosInc, Boolean[] keywordAtts, Boolean offsetsAreCorrect, Byte[][] payloads) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 393
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckAnalysisConsistency(Random random, Analyzer a, Boolean useCharFilter, String text, Boolean offsetsAreCorrect, Field field) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 1164
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations, Int32 maxWordLength, Boolean useCharFilter, Boolean simple, Boolean offsetsAreCorrect, RandomIndexWriter iw) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 848
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.AnalysisThread.Run() in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 655
   at J2N.Threading.ThreadJob.SafeRun(ThreadStart start)
--- End of stack trace from previous location ---
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations, Int32 maxWordLength, Boolean simple, Boolean offsetsAreCorrect) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 723
   at Lucene.Net.Analysis.BaseTokenStreamTestCase.CheckRandomData(Random random, Analyzer a, Int32 iterations) in D:\a\lucenenet\lucenenet\src\Lucene.Net.TestFramework\Analysis\BaseTokenStreamTestCase.cs:line 595
   at Lucene.Net.Analysis.Icu.TestICUNormalizer2Filter.TestRandomStrings() in D:\a\lucenenet\lucenenet\src\Lucene.Net.Tests.Analysis.ICU\Analysis\Icu\TestICUNormalizer2Filter.cs:line 93

  Standard Error Messages:
 TEST FAIL (iteration 118): useCharFilter=True text='\u3301\u33f3\u3355\u33db\u33c5\u3370\u33b0\u3387\u3352\u3342\u330a\u3324\u33c8'

Expected Behavior

The test passes

Steps To Reproduce

No response

Exceptions (if any)

No response

Lucene.NET Version

No response

.NET Version

No response

Operating System

No response

Anything else?

No response