[MGNLPER-17] Recognize typical marketing images reliably Created: 26/Mar/18  Updated: 07/May/19  Resolved: 26/Oct/18

Status: Closed
Project: Periscope
Component/s: None
Affects Version/s: None
Fix Version/s: 1.0

Type: Task Priority: Neutral
Reporter: Antti Hietala Assignee: Cedric Reichenbach
Resolution: Done Votes: 0
Labels: None
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Attachments: Zip Archive marketing-images.zip    
Issue Links:
Problem/Incident
causes IMGREC-40 Train custom neural network for image... Closed
Relates
relates to IMGREC-15 Bootstrap custom neural network Closed
Template:
Acceptance criteria:
Empty
Task DoR:
Empty
Date of First Response:
Epic Link: Periscope back-end MVP
Sprint: Basel 158
Story Points: 5

 Description   

User story:

As a digital marketer, I want to find the right image asset quickly. Image recognition helps me as long as it recognizes objects in typical marketing images and tags the assets accordingly. I expect common marketing subjects ("car", "dog", "person", "office", "beach" etc.) to be always recognized and tagged. I don't expect specialized subjects ("trilobite", "Cockerspaniel", "Daniel Radcliffe") to be always recognized but I am positively surprised if they are.

Internal story:

As a Magnolia Product Manager, I want local image recognition to be sufficiently complete and accurate so that it impresses evaluators. I want typical marketing subjects to be always recognized and tagged. For specialized images I expect the client to use an integrated image recognition service.

Business benefit / value: Reliable and accurate local image recognition makes a great first impression. It convinces evaluators.

Background: Local image recognition is currently limited to ImageNet 1000 synsets (synonym sets). This collection of labels does not represent typical marketing imagery. It is heavily biased towards animals ("African elephant", "hyena", "weasel") while common marketing subjects like "computer", "person" and "shoe" are missing. This means that a neural network pre-trained on Imagenet 1000 classifications does not recognize common marketing subjects.

Acceptance criteria:

  • Common marketing image subjects ("man", "woman", "child", "computer", "office", "beach") are recognized 90% of the time.
  • At least 3 tags are applied to common marketing subjects. If an easily recognizable image gets no tags it gives a poor first impression.
  • Tags are nouns. Exclude verbs, adjectives and adverbs.

Implementation proposal (optional, up to PD to decide):

Attachments:

  • marketing-images.zip – set of common marketing subjects for testing.


 Comments   
Comment by Antti Hietala [ 12/Oct/18 ]

Timebox research to 5 SP

Comment by Cedric Reichenbach [ 26/Oct/18 ]

Outcome after timeboxed effort: There are no useful-enough pretrained networks available.

The most useful one is probably TinyYOLO, which is pretty accurate and detects classes like "person" or "car", but still only supports 20 different classes. Also, it also detects (boundling-box) object locations, which we don't need here and just causes additional computation cost. Here a quick draft integration: https://git.magnolia-cms.com/users/creichenbach/repos/image-recognition/commits?until=refs%2Fheads%2FMGNLPER-17-more-relevant-tags&merges=include

See the linked follow-up issue for more infos about how to potentially proceed with a customly trained network.

Generated at Mon Feb 12 10:27:55 CET 2024 using Jira 9.4.2#940002-sha1:46d1a51de284217efdcb32434eab47a99af2938b.