[MLEARN-9] Training neural network sometimes fails due to "unknown identifier" Created: 11/Feb/19 Updated: 27/Mar/19 Resolved: 27/Mar/19 |
|
| Status: | Closed |
| Project: | Machine Learning |
| Component/s: | None |
| Affects Version/s: | None |
| Fix Version/s: | None |
| Type: | Bug | Priority: | Neutral |
| Reporter: | Cedric Reichenbach | Assignee: | Federico Grilli |
| Resolution: | Cannot Reproduce | Votes: | 1 |
| Labels: | None | ||
| Remaining Estimate: | 0d | ||
| Time Spent: | 3.5d | ||
| Original Estimate: | Not Specified | ||
| Issue Links: |
|
||||||||||||
| Template: |
|
||||||||||||
| Acceptance criteria: |
Empty
|
||||||||||||
| Task DoD: |
[ ]*
Doc/release notes changes? Comment present?
[ ]*
Downstream builds green?
[ ]*
Solution information and context easily available?
[ ]*
Tests
[ ]*
FixVersion filled and not yet released
[ ] 
Architecture Decision Record (ADR)
|
||||||||||||
| Bug DoR: |
[ ]*
Steps to reproduce, expected, and actual results filled
[ ]*
Affected version filled
|
||||||||||||
| Date of First Response: | |||||||||||||
| Epic Link: | Periscope improvements | ||||||||||||
| Sprint: | Foundation 7 | ||||||||||||
| Story Points: | 5 | ||||||||||||
| Description |
|
As observed by fgrilli while running performance load tests, training of ranking neural networks sometimes failes with the following exception: 019-02-06 17:38:33,726 ERROR gnolia.periscope.rank.ml.NeuralNetworkResultRanker: Failed to train ranking neural network java.lang.IllegalArgumentException: Unknown result with identifier: tours at info.magnolia.periscope.rank.ml.NeuralNetworkResultRanker.outputToArray(NeuralNetworkResultRanker.java:183) ~[magnolia-periscope-result-ranker-1.1-SNAPSHOT.jar:?] at info.magnolia.periscope.rank.ml.NeuralNetworkResultRanker.trainRanking(NeuralNetworkResultRanker.java:125) ~[magnolia-periscope-result-ranker-1.1-SNAPSHOT.jar:?] at info.magnolia.periscope.Periscope.resultPicked(Periscope.java:168) ~[magnolia-periscope-core-1.1-SNAPSHOT.jar:?] It looks like there's an inconsistency between assuring that a given label is in labels list and actually using that label for training. |
| Comments |
| Comment by Federico Grilli [ 27/Mar/19 ] |
|
Premise: I encountered this issue only once while running load tests in the scope of This time I wasn't able to reproduce the issue both manually and via magnolia-load-tests (due to DEV-1141).
|