Content Source Crawling issues

[Saurabh Jain]

Problem Statement

What are the possible causes of Crawl failure for a new Content Source?

Environment	Production
Reported product version	C'20
Resolved in version	C'20
Module	Content Source

Causes

1. Invalid credentials
2. Incorrect content source name
3. Incorrect client URL
4. Incorrect crawling Start Date
5. Overlap of crawling operations with patch deployment
6. Mismatch between the article fields and fields configured in the admin panel.
7. Expiration of password used to authenticate a content source

Solution

1. Ensure that the login credentials of the content source are the same as the ones used to connect the content source in the SU admin panel.
2. Check if the content source name contains any non-ASCII characters. Only ASCII characters should be used to set the content source name.
3. Only baseURL should be used while configuring content source in the SU admin panel.
4. The crawling start date should not be current or future date but should be in the past, from when you would like crawling to start.
5. If crawl fails in between, check with your CSM if there was any patch deployment happening while crawling was in progress.
6. If you have updated the article structure for one of your content sources by deleting a field, make sure to update the SearchUnify admin panel object fields for a given content source.
7. Content source Authentication - Check if the password of a user used for authentication of content source has been expired due to which content has not been auto-crawled as per frequency set on SU Admin panel.

For more information on how to configure a new content source please visit: https://docs.searchunify.com/Content/Content-Sources/Content-Source.htm