Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you may not be able to execute some actions.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Unable to crawl updated Sitemap

S

Saurabh Jain last edited by Arshi
Question

What are some of the reasons for crawlers not able to crawl updated sitemap?

Environment Production

Reported product version C'20

Resolved in version C'20

Module Search Client

Answer

Please make sure the below things are followed to correctly upload and crawl an updated Sitemap.
1. Make sure the structure of the sitemap is correct.
2. After uploading the sitemap, to enable the connect/save button, make sure the Client URL includes the domain location as shown below.
Reply Quote 0
1 Reply Last reply

Environment	Production
Reported product version	C'20
Resolved in version	C'20
Module	Search Client

1
Posts

66
Views

Suggested Topics

M

Unable to find articles in Search Results
Search Clients • Created • Last Reply Last reply • madhuri.tripathi

1

1
Votes

1
Posts

215
Views

M

Problem Statement What could be the possible reasons for not finding the article/s in the search results? Environment Production Reported product version Mamba'20 Resolved in version Mamba'20 Module Search Client Solution Any of the below is a possible reason for an article not showing up in the search results: Incorrect Data Category Tagged- The article is not tagged to a data category correctly. Data categories can be assigned when an article is created. Permission issues- The user might not have the permissions required to access the article. Crawling issue- The article has not yet been crawled and it's not available in the index. It happens when an article is newly published and crawling hasn't taken place after this addition. It will not be available in the search index in this scenario. To trigger recrawl and check the logs and status of crawling please refer to the below docs https://docs.searchunify.com/Content/Content-Sources/Recrawl.htm https://docs.searchunify.com/Content/Content-Sources/Status.htm https://docs.searchunify.com/Content/Content-Sources/Crawl-Logs.htm To check if an article is present in the index, please follow below steps: a) Click the document counts for a given content source. b) Select the date range when an article is crawled in SearchUnify index. c) Search the article using a search box. Stopwords- Keywords used to search the particular article/s are included in the stopwords section (NLP Manager > Manage Stopwords) in SearchUnify Admin. Advanced Search is performed- # used before the search query. Example "And" operator. The search ends up with no results due to these operators. Exclude List- Articles are not filtered in "Create Formula" options under Search client configurations in the Admin portal. Facet Selection- For the selected facets, the article/s in question isn't available. Missing content URLs in the sitemap XML - Sitemap XML needs to have all the content URLs to make sure SU crawlers are indexing all the content. Please review your sitemaps for missing content URL. Content Source Upgrade- Sometimes after the content source upgrade, auto crawlers fail to index the new content even though it is made searchable in the admin panel. Users are advised to manually recrawl the content source and see if missing content appears in search results. Frequency set to Never - If the frequency of crawl is set to never, new content will not be auto crawled, and hence some content will be unavailable in the search results. To set the frequency please follow the below steps: https://docs.searchunify.com/Content/Content-Sources/Edit.htm#!#SetUpCrawlFrequency Missing Keyword in any of the Content Sources - Please make sure the keyword that you are searching for is present in one of the content sources. Searchunify also allows you to partially search a keyword by appending #* at the beginning. For Example: If the exact keyword "CENG-50196" is not present in any of the content sources, you can partially search #*50196 to return all results that contain 50196. Disallowed Pages - If an article exists in the list of disallowed pages, crawlers will not index those pages. Please check the list of pages that are disallowed for indexing. Generally, customers maintain robot.txt file to maintain list of disallowed pages. URL present in Should not crawl list - For website type of Content Sources, make sure URL is not present in the "Should Not Crawl" list. Below are the steps to verify the same: a. Edit a website type content source and go to Rules Tab. b. Under filter tab, check "Should not crawl" list to ensure website URL is not present. Field not added in admin panel - If a new field is added to the content metadata to an existing content source in the SU admin panel, the field needs to be added to the SU object and made searchable by following the below steps: a. Open SU object where the field needs to be added b. Add the metadata field with a label for the object and click save. c. Make the field searchable
M

How to hide specific results from the crawled content without deleting?
Search Clients • Created • Last Reply Last reply • madhuri.tripathi

1

0
Votes

1
Posts

191
Views

M

Environment Production Reported product version Mamba'20 Resolved in version Mamba'20 Module Search Client Answer To hide specific document (in this case a Knowledge Article) from the crawled content sources, follow the steps below: Login to SearchUnify Admin panel. Navigate to search client and click Edit. Navigate to Content source tab > Select the content source and click on Create formula. Drag the ID field and enter the ID of an Article which you want to exclude from the search result in the Value Field. Enable parity and click Save. Please refer to this article for more details.
S

Incorrect metadata tag values
Search Clients • Created • Last Reply Last reply • Saurabh Jain

1

1
Votes

1
Posts

25
Views

S

Problem Statement: Why is the metatag seen in the search results not the same as the metatag on the article? How do I populate the "Created date" metadata field value on the search results with "Published date"? Can SearchUnify truncate metatag values in search results? Environment Production Reported product version M'21 Resolved in version M'21 Module Search Client Solution: The metatag field can be selected and populated while configuring the search client in the admin panel. Please check: https://docs.searchunify.com/Content/Search-Clients/Configure-the-Content-Sources.htm If metadata tags don’t show the correct value, it is advised to recrawl the content source so that the metadata tags pick the correct values. To manually recrawl the content source please follow: https://docs.searchunify.com/Content/Content-Sources/Recrawl.htm As of now, we cannot truncate the metatag value if it is occupying a bigger portion of the search results page. However, with M21, users will be able to preview the search results page with a bigger metadata field value. Please check: https://docs.searchunify.com/Content/Search-Clients/Configurations.htm#!#PreviewSearchResults
S

Incorrect or missing langugage tag
Search Clients • Created • Last Reply Last reply • Saurabh Jain

1

1
Votes

1
Posts

8
Views

S

Problem Statement: Why some of the documents are tagged as French in SearchUnify results when they are written and published in the English language? Users are not able to filter few articles based on language? When they select a language filter, corresponding articles are not shown in the search results? Why is the language filter for our doc site not coming? Environment Production Reported product version M'21 Resolved in version M'21 Module Search Client Causes: Content in the community is tagged to the French language. Articles are missing language metadata or not tagged to the correct language metadata. Language metadata information of the articles is not crawled Solution: Please check if the content present in your community is in the English language and not in the French language. SearchUnify crawlers use REST APIs to fetch content from different communities. Please click here to understand how to configure metadata for search results. If few articles are missing for a given language filter, please check whether articles are tagged and crawled with correct language metadata. For other causes of missing articles in the search result, please click here Make sure crawling is not stuck and all the content with correct language metadata is crawled.
S

Duplicate Search Results
Search Clients • duplicatesearch apps • Created • Last Reply Last reply • Saurabh Jain

1

0
Votes

1
Posts

48
Views

S

Problem Statement: Why are duplicate search results coming? Environment Production Reported product version C'20 Resolved in version C'20 Module Search Client Causes: Duplicate content sources (same content is present in multiple content sources). Updated the crawling configuration that returned the same articles with http:// and https:// protocols. Updated the title of an article and have not yet published the article. Solution: Remove the duplicate content from one of the content repositories. In case any dummy content source is added that contains duplicate content, it can be removed by the following steps- a) Login to SU Admin panel b) Go to Content Sources c) Click on the delete option to remove the content source If the crawling configuration is updated and it is returning duplicate articles with http:// and https:// protocols, you can filter out results so that only articles either from http:// or https:// are shown. To know more on how to filter results using the "Create Formula" method please refer: https://docs.searchunify.com/Content/Search-Clients/Create-Formula.htm If the title of an article is changed and crawlers are crawling both draft and published articles, search results will display both the articles with different article states.
S

No results are returned in SearchUnify
Search Clients • Created • Last Reply Last reply • Saurabh Jain

1

0
Votes

1
Posts

25
Views

S

Questions: What are some of the reasons for SearchUnify returning no results? Environment Production Reported product version C'20 Resolved in version C'20 Module Search Client Answers: Connection with the content source got failed. Reauthenticating the content source and recrawling the content source should solve the problem. User was deactivated in the given content source such as Salesforce, Lithium, Zendesk, etc. User needs to be reactivated to fetch search results from the given content source. All the results are filtered due to the Formula defined in the SU admin panel. Correct or remove the formula to get the search results. Performance issue that caused analytics data size to exceed the heap size. Please contact the SearchUnify support team to take further action. Incorrect Client URL. The Client URL needs to be corrected before trying to make a connection.
S

Cached Results
Search Clients • Created • Last Reply Last reply • Saurabh Jain

1

0
Votes

1
Posts

26
Views

S

Question: Does SU search API return the latest updated articles or the cached versions? Environment Production Reported product version C'20 Resolved in version C'20 Module Search Client Answer: SearchUnify search API does not return cached results. It matches the keywords to the searchable fields in the indexed documents and presents the user with results with the top score. With the release of M21, end users can now view multiple versions of the same article on the results page. Please review the below link for more info https://docs.searchunify.com/Content/Search-Clients/Merge-Results.htm
S

Private Group post is shown to all users
Search Clients • Created • Last Reply Last reply • Saurabh Jain

1

0
Votes

1
Posts

26
Views

S

Problem Statement Why are the posts from the private groups shown to all the users? Environment Production Reported product version C'20 Resolved in version C'20 Module Search Client Causes: The "Privacy Status" of the group is not set to hidden. Formula defined for private discussions in SearchClient is not correct The group has been moved from public to private and no recrawl is done Solution: Please ensure: The "Privacy Status" of the group is set to ‘hidden’ in order to exclude indexing of such groups by SearchUnify crawlers. If a formula defined to filter private discussions contains nested navigational topics, then it will display all the content to all the users as SearchUnify currently only supports navigational topics on discussions. As most of the feed items in a discussion group are tagged to visibility, you need to recrawl the data to filter out the private content.

Unable to crawl updated Sitemap

Question

Answer

Suggested Topics

Unable to find articles in Search Results
Search Clients • Created • Last Reply Last reply • madhuri.tripathi

How to hide specific results from the crawled content without deleting?
Search Clients • Created • Last Reply Last reply • madhuri.tripathi

Incorrect metadata tag values
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Incorrect or missing langugage tag
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Duplicate Search Results
Search Clients • duplicatesearch apps • Created • Last Reply Last reply • Saurabh Jain

No results are returned in SearchUnify
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Cached Results
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Private Group post is shown to all users
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Unable to crawl updated Sitemap

Question

Answer

Suggested Topics

Unable to find articles in Search Results Search Clients • Created • Last Reply Last reply • madhuri.tripathi

How to hide specific results from the crawled content without deleting? Search Clients • Created • Last Reply Last reply • madhuri.tripathi

Incorrect metadata tag values Search Clients • Created • Last Reply Last reply • Saurabh Jain

Incorrect or missing langugage tag Search Clients • Created • Last Reply Last reply • Saurabh Jain

Duplicate Search Results Search Clients • duplicatesearch apps • Created • Last Reply Last reply • Saurabh Jain

No results are returned in SearchUnify Search Clients • Created • Last Reply Last reply • Saurabh Jain

Cached Results Search Clients • Created • Last Reply Last reply • Saurabh Jain

Private Group post is shown to all users Search Clients • Created • Last Reply Last reply • Saurabh Jain

Unable to find articles in Search Results
Search Clients • Created • Last Reply Last reply • madhuri.tripathi

How to hide specific results from the crawled content without deleting?
Search Clients • Created • Last Reply Last reply • madhuri.tripathi

Incorrect metadata tag values
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Incorrect or missing langugage tag
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Duplicate Search Results
Search Clients • duplicatesearch apps • Created • Last Reply Last reply • Saurabh Jain

No results are returned in SearchUnify
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Cached Results
Search Clients • Created • Last Reply Last reply • Saurabh Jain

Private Group post is shown to all users
Search Clients • Created • Last Reply Last reply • Saurabh Jain