Your browser does not seem to support JavaScript. As a result, your viewing experience will be diminished, and you may not be able to execute some actions.

Please download a browser that supports JavaScript, or enable it if it's disabled (i.e. NoScript).

Generate a Sitemap from URL (loc) and Last Modified (lastmod) Attributes

sugrokker last edited by sugrokker
Site Map Generator is an open source program used to create a customized site map with two attributes: <loc> and <lastmod>.
```
<url>
<loc>
https://docs.searchunify.com/Content/Content-Sources/Jive.htm
</loc>
<lastmod>2020-05-07</lastmod>
</url>
```
It offers several options as well. The most useful is probably -b which is used to specify the file types that shouldn’t be part of the sitemap.

Prerequisites
- Server (or PC) with Python 3 or higher installed
- Server (or PC) with PIP3 installed
Method
1. Go to this page and copy the code.
2. Create a file named sitemap_gen.py.
```
~$ touch sitemap_gen.py
```
1. Paste the code into sitemap_gen.py.
2. Generate a sitemap.
```
~$ python3 sitemap_gen.py https://yourwebsite.com/
```
1. Depending on the size of the website, the command might continue to run for a few minutes. Once it has run its course, a file named sitemap.xml will be created.
Tips

To exclude certain file types from the sitemap, use the argument -b.
```
~$ python3 sitemap_gen.py -b png -b jpeg https://yourwebsite.com/
```
The above command puts all the images into the sitemap except for png and jpeg images.
Reply Quote 1
1 Reply Last reply

1
Posts

229
Views

Suggested Topics

S

New Content Source Crawling issues
Content Sources • • Saurabh Jain

1

1
Votes

1
Posts

160
Views

S

Problem Statement What are the possible causes of Crawl failure for a new Content Source? Environment Production Reported product version C'20 Resolved in version C'20 Module Content Source Causes Invalid credentials Incorrect content source name Incorrect client URL Incorrect crawling Start Date Overlap of crawling operations with patch deployment Mismatch between the article fields and fields configured in the admin panel. Solution Ensure that the login credentials of the content source are the same as the ones used to connect the content source in the SU admin panel. Check if the content source name contains any non-ASCII characters. Only ASCII characters should be used to set the content source name. Only baseURL should be used while configuring content source in the SU admin panel. The crawling start date should not be current or future date but should be in the past, from when you would like crawling to start. If crawl fails in between, check with your CSM if there was any patch deployment happening while crawling was in progress. If you have updated the article structure for one of your content sources by deleting a field, make sure to update the SearchUnify admin panel object fields for a given content source. For more information on how to configure a new content source please visit: https://docs.searchunify.com/Content/Content-Sources/Content-Source.htm
Content Sources Authentication — Cheat Sheet
Content Sources • content sources cheatsheet authentication • • sugrokker

1

1
Votes

1
Posts

86
Views

A quick cheat sheet to help you prepare beforehand if you are planning to index multiple content sources. No Auth No content source privileges are needed, apart from the content source's URL. Select it to crawl public content. Basic Auth Indexes the data available to the user whose username and password are entered. Admins can either enter the credentials on the screen or in the pop-up window which appears when you click Connect. OAuth and Others High security authentication for some content sources, such as Khoros (Lithium) communities, Higher Logic communities, and ServiceNow. Usually requires an API token or the client ID and client password of an app. In this chart, you can map content sources with available authentication methods. Content Source No Auth Basic Auth OAuth or Others API Yes Yes No Azure DevOps No No Yes; Personal Access Token (PAT) Box No Yes No Confluence No Yes Yes; OAuth Docebo No No Yes; client ID and client secret Dropbox No Yes No GitHub No No Yes Google Drive No Yes No Helpscout No No Yes; API keys Higher Logic No No Yes; Username, password, and HLIAM key Jira Yes Yes Yes; OAuth Jira On-Premises No No Yes Jive No No Yes; client ID and client secret Lithium Yes No Yes; (1) Community User details and HTAccess; (2) API User details and HTAccess; (3) OAuth Litmos No No Yes; API key MadCap Flare No No Yes Marketo No No Yes; client ID and secret Microsoft Dynamics No No Yes; client ID, tenant ID, and client secret Mindtouch No Yes No Moodle No No Yes Saba No Yes No Salesforce No Yes No ServiceNow No Yes Yes; OAuth SharePoint No Yes No SkillJar No No Yes; API key Slack No Yes No SOLR Yes No No StackOverflow Yes No Yes; OAuth Vimeo No Yes No Website Yes Yes Yes; Form YouTube No Yes No Zendesk Yes Yes Yes; OAuth Zoomin Yes No No
S

Restrict SU Search to a specific type of knowledge articles
Content Sources • • Saurabh Jain

1

0
Votes

1
Posts

46
Views

S

Question How can a user restrict SU crawlers to index only specific category of knowledge articles? Environment Production Reported product version C'20 Resolved in version C'20 Module Content Sources Answer To restrict search to a specific knowledge article, it is required to know "record type id" of that article type. Once it is known, below steps can be followed to configure it in SU admin panel: Click on Content source in SU admin panel Edit content source Click on "Rules" Tab Click on "Manage conditions" option for a given Salesforce object 5)Drag the record type ID field and enter the "record type ID" that you want to crawl.
How to Rename a Content Type?
Content Sources • content sources content type object rename change label • • sugrokker

1

0
Votes

1
Posts

101
Views

Content Types are the objects used for storing data. Some platforms contain only one object while in others there is virtually no upper limit. YouTube has one content type, video. Dropbox supports two content types, paper and file. But in Salesforce, admins can create dozens of custom objects. Object names are defined by the platform in question but SearchUnify allows admins to change their labels. For instance, you can YouTube video as movie and Dropbox paper as doc. Change Content Type Label To change the label of a content type Go to Content Sources from the main navigation. Open any content source for editing. Open the Rules tab. In the Label column, find your content type, click the edit icon, and enter the new label. Save your settings. The content type will be renamed.
S

Content Source crawling status and crawling logs
Content Sources • • Saurabh Jain

1

0
Votes

1
Posts

112
Views

S

Problem Statement: How to find the number of documents crawled in a content source? How to find the crawling logs for the last crawling of a content source? How to manually start/stop crawling of a content source? How to setup Email notification Alert for Crawling Completion status of a content source? Environment Production Reported product version Mamba'20 Resolved in version Mamba'20 Module Alerts and Content Source Resolution Number of documents indexed in a content source, is shown up next to the name of the Content source (shown in the image) Crawling logs for the last crawl of a content source tab are accessible, on click of logs icon (shown in the image) Click on the Refresh icon to manually start and stop the crawling of a content source (shown in the image) Content source Indexing notifications are configurable from the Admin Panel. To set it up please follow these steps: a. Login into SearchUnify Admin Panel b. Navigate to "Alerts" section from the navigation list on the left c. Click on "Content Source Subscription" tab d. Click "Add Subscription Email", select the content source and enter the email ID to set the crawling alerts
M

Remove Archived Article from Search Results
Content Sources • contentsource delete articles total documents • • madhuri.tripathi

1

1
Votes

1
Posts

144
Views

M

Problem Statement When will an archived article stop reflecting in the search results? Environment Production Reported product version Mamba'20 Resolved in version Mamba'20 Module Content Crawling frequency Cause The jobs running in the backend for deleting the Archived/Draft articles are different from the ones used for crawling content sources. Solution Archived articles are deleted from the content indexes as per the frequency set in the backend job. Another option for admins is to manually delete an article from the index: Log in to the admin panel Go to Content source Under the Documents column, click on the number of documents reflecting for a particular content source. Search for the desired article and delete it.

Generate a Sitemap from URL (loc) and Last Modified (lastmod) Attributes

Prerequisites

Method

Tips

Suggested Topics

New Content Source Crawling issues Content Sources • • Saurabh Jain

Content Sources Authentication — Cheat Sheet Content Sources • content sources cheatsheet authentication • • sugrokker

Restrict SU Search to a specific type of knowledge articles Content Sources • • Saurabh Jain

How to Rename a Content Type? Content Sources • content sources content type object rename change label • • sugrokker

Content Source crawling status and crawling logs Content Sources • • Saurabh Jain

Remove Archived Article from Search Results Content Sources • contentsource delete articles total documents • • madhuri.tripathi

New Content Source Crawling issues
Content Sources • • Saurabh Jain

Content Sources Authentication — Cheat Sheet
Content Sources • content sources cheatsheet authentication • • sugrokker

Restrict SU Search to a specific type of knowledge articles
Content Sources • • Saurabh Jain

How to Rename a Content Type?
Content Sources • content sources content type object rename change label • • sugrokker

Content Source crawling status and crawling logs
Content Sources • • Saurabh Jain

Remove Archived Article from Search Results
Content Sources • contentsource delete articles total documents • • madhuri.tripathi