Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you may not be able to execute some actions.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Generate a Sitemap from URL (loc) and Last Modified (lastmod) Attributes

sugrokker last edited by sugrokker
Site Map Generator is an open source program used to create a customized site map with two attributes: <loc> and <lastmod>.
```
<url>
<loc>
https://docs.searchunify.com/Content/Content-Sources/Jive.htm
</loc>
<lastmod>2020-05-07</lastmod>
</url>
```
It offers several options as well. The most useful is probably -b which is used to specify the file types that shouldn’t be part of the sitemap.

Prerequisites
- Server (or PC) with Python 3 or higher installed
- Server (or PC) with PIP3 installed
Method
1. Go to ~~this page~~ (updated link) and copy the code.
2. Create a file named sitemap_gen.py.
```
~$ touch sitemap_gen.py
```
1. Paste the code into sitemap_gen.py.
2. Generate a sitemap.
```
~$ python3 sitemap_gen.py https://yourwebsite.com/
```
1. Depending on the size of the website, the command might continue to run for a few minutes. Once it has run its course, a file named sitemap.xml will be created.
Tips

To exclude certain file types from the sitemap, use the argument -b.
```
~$ python3 sitemap_gen.py -b png -b jpeg https://yourwebsite.com/
```
The above command puts all the images into the sitemap except for png and jpeg images.
Reply Quote 2
1 Reply Last reply

0
Replies

283
Views

Suggested Topics

M

Remove Archived Article from Search Results
Created • Last Reply Last reply • madhuri.tripathi

Content Sources contentsource delete articles total documents

0

1
Votes

0
Replies

197
Views

M

Problem Statement When will an archived article stop reflecting in the search results? Environment Production Reported product version Mamba'20 Resolved in version Mamba'20 Module Content Crawling frequency Cause The jobs running in the backend for deleting the Archived/Draft articles are different from the ones used for crawling content sources. Solution Archived articles are deleted from the content indexes as per the frequency set in the backend job. Currently, these jobs automatically delete archived articles only from Salesforce content source. Another option for admins is to manually delete an article from the index: Log in to the admin panel Go to Content source Under the Documents column, click on the number of documents reflecting for a particular content source. Search for the desired article and delete it.
S

Content Source crawling status and crawling logs
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling

0

0
Votes

0
Replies

122
Views

S

Problem Statement: How to find the number of documents crawled in a content source? How to find the crawling logs for the last crawling of a content source? How to manually start/stop crawling of a content source? How to setup Email notification Alert for Crawling Completion status of a content source? Environment Production Reported product version Mamba'20 Resolved in version Mamba'20 Module Alerts and Content Source Resolution Number of documents indexed in a content source, is shown up next to the name of the Content source (shown in the image) Crawling logs for the last crawl of a content source tab are accessible, on click of logs icon (shown in the image) Click on the Refresh icon to manually start and stop the crawling of a content source (shown in the image) Content source Indexing notifications are configurable from the Admin Panel. To set it up please follow these steps: a. Login into SearchUnify Admin Panel b. Navigate to "Alerts" section from the navigation list on the left c. Click on "Content Source Subscription" tab d. Click "Add Subscription Email", select the content source and enter the email ID to set the crawling alerts
S

Exclude pages from Indexing
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling exclude

0

1
Votes

0
Replies

53
Views

S

Question What is the best way to exclude multiple pages from search when they are not in the same folder structure? Environment Production Reported product version M'21 Resolved in version M'21 Module Content Sources Answer The best way to exclude the docs while crawling would be to add all the URLs to the "Should not crawl" section. Below are the steps: Navigate to Content Sources in Admin Panel Edit website type Content Source from which you want to exclude the docs. Navigate to Rule tab and click "By Filter" Add pages in "should not crawl" section. This will prevent these pages from being indexed and shown in search result pages. Note: These URLs will also be excluded by auto crawlers that are running after a given frequency. Also, this feature is only available for website type content source.
S

Content Source Crawling issues
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling

0

1
Votes

0
Replies

193
Views

S

Problem Statement What are the possible causes of Crawl failure for a new Content Source? Environment Production Reported product version C'20 Resolved in version C'20 Module Content Source Causes Invalid credentials Incorrect content source name Incorrect client URL Incorrect crawling Start Date Overlap of crawling operations with patch deployment Mismatch between the article fields and fields configured in the admin panel. Expiration of password used to authenticate a content source Solution Ensure that the login credentials of the content source are the same as the ones used to connect the content source in the SU admin panel. Check if the content source name contains any non-ASCII characters. Only ASCII characters should be used to set the content source name. Only baseURL should be used while configuring content source in the SU admin panel. The crawling start date should not be current or future date but should be in the past, from when you would like crawling to start. If crawl fails in between, check with your CSM if there was any patch deployment happening while crawling was in progress. If you have updated the article structure for one of your content sources by deleting a field, make sure to update the SearchUnify admin panel object fields for a given content source. Content source Authentication - Check if the password of a user used for authentication of content source has been expired due to which content has not been auto-crawled as per frequency set on SU Admin panel. For more information on how to configure a new content source please visit: https://docs.searchunify.com/Content/Content-Sources/Content-Source.htm
S

Restrict SU Search to a specific type of knowledge articles
Created • Last Reply Last reply • Saurabh Jain

Content Sources

0

0
Votes

0
Replies

59
Views

S

Question How can a user restrict SU crawlers to index only specific category of knowledge articles? Environment Production Reported product version C'20 Resolved in version C'20 Module Content Sources Answer To restrict search to a specific knowledge article, it is required to know "record type id" of that article type. Once it is known, below steps can be followed to configure it in SU admin panel: Click on Content source in SU admin panel Edit content source Click on "Rules" Tab Click on "Manage conditions" option for a given Salesforce object 5)Drag the record type ID field and enter the "record type ID" that you want to crawl.
How to Rename a Content Type?
Created • Last Reply Last reply • sugrokker

Content Sources content sources content type object rename change label

0

0
Votes

0
Replies

108
Views

Content Types are the objects used for storing data. Some platforms contain only one object while in others there is virtually no upper limit. YouTube has one content type, video. Dropbox supports two content types, paper and file. But in Salesforce, admins can create dozens of custom objects. Object names are defined by the platform in question but SearchUnify allows admins to change their labels. For instance, you can YouTube video as movie and Dropbox paper as doc. Change Content Type Label To change the label of a content type Go to Content Sources from the main navigation. Open any content source for editing. Open the Rules tab. In the Label column, find your content type, click the edit icon, and enter the new label. Save your settings. The content type will be renamed.
Content Sources Authentication — Cheat Sheet
Created • Last Reply Last reply • sugrokker

Content Sources content sources cheatsheet authentication

0

1
Votes

0
Replies

104
Views

A quick cheat sheet to help you prepare beforehand if you are planning to index multiple content sources. No Auth No content source privileges are needed, apart from the content source's URL. Select it to crawl public content. Basic Auth Indexes the data available to the user whose username and password are entered. Admins can either enter the credentials on the screen or in the pop-up window which appears when you click Connect. OAuth and Others High security authentication for some content sources, such as Khoros (Lithium) communities, Higher Logic communities, and ServiceNow. Usually requires an API token or the client ID and client password of an app. In this chart, you can map content sources with available authentication methods. Content Source No Auth Basic Auth OAuth or Others API Yes Yes No Azure DevOps No No Yes; Personal Access Token (PAT) Box No Yes No Confluence No Yes Yes; OAuth Docebo No No Yes; client ID and client secret Dropbox No Yes No GitHub No No Yes Google Drive No Yes No Helpscout No No Yes; API keys Higher Logic No No Yes; Username, password, and HLIAM key Jira Yes Yes Yes; OAuth Jira On-Premises No No Yes Jive No No Yes; client ID and client secret Lithium Yes No Yes; (1) Community User details and HTAccess; (2) API User details and HTAccess; (3) OAuth Litmos No No Yes; API key MadCap Flare No No Yes Marketo No No Yes; client ID and secret Microsoft Dynamics No No Yes; client ID, tenant ID, and client secret Mindtouch No Yes No Moodle No No Yes Saba No Yes No Salesforce No Yes No ServiceNow No Yes Yes; OAuth SharePoint No Yes No SkillJar No No Yes; API key Slack No Yes No SOLR Yes No No StackOverflow Yes No Yes; OAuth Vimeo No Yes No Website Yes Yes Yes; Form YouTube No Yes No Zendesk Yes Yes Yes; OAuth Zoomin Yes No No

Generate a Sitemap from URL (loc) and Last Modified (lastmod) Attributes

Prerequisites

Method

Tips

Suggested Topics

Remove Archived Article from Search Results
Created • Last Reply Last reply • madhuri.tripathi

Content Sources contentsource delete articles total documents

Content Source crawling status and crawling logs
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling

Exclude pages from Indexing
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling exclude

Content Source Crawling issues
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling

Restrict SU Search to a specific type of knowledge articles
Created • Last Reply Last reply • Saurabh Jain

Content Sources

How to Rename a Content Type?
Created • Last Reply Last reply • sugrokker

Content Sources content sources content type object rename change label

Content Sources Authentication — Cheat Sheet
Created • Last Reply Last reply • sugrokker

Content Sources content sources cheatsheet authentication

Generate a Sitemap from URL (loc) and Last Modified (lastmod) Attributes

Prerequisites

Method

Tips

Suggested Topics

Remove Archived Article from Search Results Created • Last Reply Last reply • madhuri.tripathi Content Sources contentsource delete articles total documents

Content Source crawling status and crawling logs Created • Last Reply Last reply • Saurabh Jain Content Sources content sources crawling

Exclude pages from Indexing Created • Last Reply Last reply • Saurabh Jain Content Sources content sources crawling exclude

Content Source Crawling issues Created • Last Reply Last reply • Saurabh Jain Content Sources content sources crawling

Restrict SU Search to a specific type of knowledge articles Created • Last Reply Last reply • Saurabh Jain Content Sources

How to Rename a Content Type? Created • Last Reply Last reply • sugrokker Content Sources content sources content type object rename change label

Content Sources Authentication — Cheat Sheet Created • Last Reply Last reply • sugrokker Content Sources content sources cheatsheet authentication

Remove Archived Article from Search Results
Created • Last Reply Last reply • madhuri.tripathi

Content Sources contentsource delete articles total documents

Content Source crawling status and crawling logs
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling

Exclude pages from Indexing
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling exclude

Content Source Crawling issues
Created • Last Reply Last reply • Saurabh Jain

Content Sources content sources crawling

Restrict SU Search to a specific type of knowledge articles
Created • Last Reply Last reply • Saurabh Jain

Content Sources

How to Rename a Content Type?
Created • Last Reply Last reply • sugrokker

Content Sources content sources content type object rename change label

Content Sources Authentication — Cheat Sheet
Created • Last Reply Last reply • sugrokker

Content Sources content sources cheatsheet authentication