Uploaded image for project: 'Solr Search Provider'
  1. Solr Search Provider
  2. MGNLEESOLR-156

Make crawler maxOutgoingLinksToFollow configurable

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Fixed
    • Normal
    • 5.5.5
    • None
    • None
    • 1

    Description

      Crawler4j by default limits parsed outgoing links from page to 5000, this doesn't have to be enough for example when using sitemap for all the urls to crawler.

      maxOutgoingLinksToFollow should be configurable from magnolia crawler configuration

      Checklists

        Acceptance criteria

        Attachments

          Issue Links

            Activity

              People

                mdivilek Milan Divilek
                mdivilek Milan Divilek
                Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                  Created:
                  Updated:
                  Resolved:

                  Checklists

                    Task DoD