[MGNLPER-6] Research: Adapt neural network to changing the size of the result set; improve performance Created: 27/Feb/18  Updated: 16/Apr/19  Resolved: 23/Nov/18

Status: Closed
Project: Periscope
Component/s: None
Affects Version/s: None
Fix Version/s: 1.0

Type: New Feature Priority: Neutral
Reporter: Ilgun Ilgun Assignee: Cedric Reichenbach
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
relates to MGNLPER-70 Physical memory usage is too high Closed
Template:
Acceptance criteria:
Empty
Epic Link: Periscope back-end MVP
Sprint: Basel 160, Basel 161
Story Points: 5

 Description   

Currently neural network doesn't adjust itself to changing result set however, we should be considering the fact that result set typically changes over time and thus this case should be handled properly.

Acceptance criteria / Research outcome:

  • Decision if we should set a fixed limit for the result set, such as 100,000 items.
    • Proposal what that limit should be.
    • Try if integer.maximum works.
    • Find a graceful fallback once all output units are "used up" (ideally forgetting old/irrelevant ones).
  • Improve performance, potentially by reducing the overall number of units (both in hidden layers and above-mentioned output layer).
  • Improve neural network storage performance
    • Debounce storage; only store once per time unit (default to e.g. 2 mins).
    • Assure proper synchronization on storage operations.
  • Timebox research to 5 SP.

Generated at Mon Feb 12 10:27:48 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.