Installation and Setup

Installing the Connector

📘
Prerequisite

Sitefinity version 14.0.7700, 14.1.7800, 14.2.7900, 14.3.8000 , 14.4.8100.1 , 14.4.8100.2, 15.0.8200.1

Corresponding version of Sitefinity connector NuGet package:

Install from the following NuGet repository - https://developer.hawksearch.com/nuget

Install the NuGet from local repository - Place the NuGet package in a local folder on your workstation and set it as Package source in Visual Studio

Please refer to Lifecycle Policy - Progress Sitefinity to keep up with Supported and Retired Sitefinity versions from Progress .

🚧
Important
Disable 'Use cached controller container assemblies' option in Sitefinity Advanced Settings

Open Sitefinity Advanced Settings (your-website-url/Sitefinity/Administration/Settings/Advanced)

Select ‘Feather’ from the settings list

Uncheck 'Use cached controller container assemblies'

Click ‘Save Changes’ button

Steps to install the connector

Install the Hawksearch.Sitefinity NuGet package
Build your solution

🚧
Important
Modify HawkRoutesInitializer.cs - comment or remove the following two lines of code containing the config.MapHttpAttributeRoutes() and config.EnsureInitialized() located in the HawkRoutesInitializer.cs (App_Start folder) in case you already have them in your Global.asax.cs or anywhere else in your project.

Activation

Steps to Configure Sitefinity Connector

From Administration tab open the Module & Services (your-site-domain/Sitefinity/Administration/ModulesAndServices) and ensure that Hawksearch module is installed and active.
From the basic settings screen, open the Search section (your-site-domain/Sitefinity/Administration/Settings/Basic/Search) and specify Hawksearch as the selected search service in the system.
If the Sitefinity Web Security module (your-site-domain/Sitefinity/Administration/Settings/Basic/WebSecurityBasicSettings) is enabled then you must apply the following settings in the 'Trusted sources' screen:
1. In the 'Scripts' section add - *.hawksearch.net
2. In the 'Styles' section add - *.hawksearch.net
3. In the 'Connect sources' section (under 'Forms, frames, child sources, connect sources, plugins') add - *.hawksearch.net

Applying Settings

Remark
You can find the values for HawkSearchApiKey and HawkSearchClientId from the Hawksearch engine backend : Admin section → Account Info tab

Steps for Integrating Your Hawksearch Instance

From the ‘Advanced Settings’ (your-site-domain/Sitefinity/Administration/Settings/Advanced) screen, open the Hawksearch section and configure your Hawksearch server related settings.

Specify whether you are using V4 version or V4L version of Hawksearch,

For Hawksearch V4 (sample values):
1. HawkSearchApiKey: 2W113223-B3YU-42R5-A5T8-4WER5632G3S
2. HawkSearchIndex: https://indexing-dev.hawksearch.net/api/v2/indexing/
3. HawkSearchSearchingAPI: https://searchapi-dev.hawksearch.net/api/v2/search
4. HawkSearchBaseAPI: https://dev.hawksearch.net/api/v9
5. HawkSearchClientId: 3e3c44b0vn6688a0121r84rd446o5577
6. [Optional] ExportDirectory : C:\Projects\Hawk\Hawk Export
For Hawksearch V4L (sample values):
1. HawkSearchIndex: hawksearchenginename
2. HawkSearchApiKey: 2W113223-B3YU-42R5-A5T8-4WER5632G3S
3. HawkSearchBaseAPI: https://dev.hawksearch.net/api/v9HawkSearchClientId: 3e3c44b0vn6688a0121r84rd446o5577
4. HawkSearchEngineUrl: dev.hawksearch.net/sites/
5. ExportDirectory : C:\Projects\Hawk\Hawk Export
6. HawkSearchStagingDomain: dev.hawksearch.net

Settings Explanations

Goal
Provide descriptions of all of the Hawksearch configurations.

Versioning

Version
Specifies which version of Hawksearch is supposed to be used.

Document size

Document Size Limit
Controls the maximum size in KB of the documents that can be indexed. The default value is 200KB and the maximum is 4000KB. If you wish to index larger documents a custom search service must be implemented.

Logging information

Enable Trace logging when Indexing
Specifies whether the index requests and responses should be logged in Sitefinity’s Trace log file.

Enable Trace logging when Searching
Specifies whether the search requests and responses should be logged in Sitefinity’s Trace log file.

Deferred index

Deferred Index Activation

Controls when index can be used from Hawksearch widgets. If checked, index will be used after reindexing is finished. If unchecked, the index will be used immediately. Index deletion will be deferred by the the amount of time provided in Delete Interval field.

Delete interval

Controls the interval for deleting the index In seconds. Used when creating delete scheduled tasks.

Tracking

Enable Hawksearch Event tracking

Select whether the Hawksearch Event Tracking functionality should be enabled.

Provider configuration

Default provider

Specifies the default data provider. Can be accessed through the HawkManager class.

Export information

LastDeltaExportTime

Used to display information in the Hawksearch admin page about the last time the delta export was used.

LastFullExportTime

Used to display information in the Hawksearch admin page about the last time the full export was used.

LastFullExportDuration

Used to display information in the Hawksearch admin page about the duration of the last full export.

LastDeltaExportDuration

Used to display information in the Hawksearch admin page about the duration of the last delta export.

LastExportType

Specifies whether the last export was full or delta (empty if none of them was used).

Filter results

Filter by permissions

Choose if search results should be filtered by the permissions of the user when using the built-in Sitefinity search widgets.

Export directory for data files

ExportDirectory

Specifies the directory for the export of the files.

Scheduling

ExportScheduleType

Specifies how often a full export should be executed – choose between none, daily and wekly

ExportScheduleDayOfWeek

Specifies the day of the week on which the full export should be executed.

ExportScheduleTime

Specifies the time at which the full export should be executed

ExportDeltaIntervalInMinutes

Specifies the time between each Delta export in minutes

Delta Lifetime Interval

Controls for how long removed delta items will be persisted in the database. Enter 0 if you don't want to delete the removed items.

Task keys

ExportTaskKey

An ID used to mark scheduled full export tasks or deleting full export schedule. If there are more than 2 tasks with that ID further tasks will not be scheduled. The number of concurrent tasks can be modified through the QueuedExportTasksLimit setting.

ExportDeltaTaskKey

An ID used to mark scheduled delta export tasks or deleting delta export schedule. If there are more than 2 tasks with that ID further tasks will not be scheduled. The number of concurrent tasks can be modified throught the QueuedExportTasksLimit setting.

QueuedExportTasksLimit

Used to limit the number of concurrent export or delete tasks for full or delta export. Default value is 2

API keys and endpoints

HawkSearchApiKey

The key used to identify against the Hawksearch engine when indexing. Used In the Initialization of the Hawksearch client.

TrackingURL

The endpoint which exposes tracking information about user behavior e.g. what they click, how often etc.

AutocompleteURL

Hawksearch endpoint at which requests with search input are sent when using the Hawksearch widgets in order to have autocomplete functionality. Used In the Initialization of the Hawksearch client.

HawkSearchIndex

When using the V4 version this setting contains the URL for the Hawksearch Indexing API

When using the V4L version it contains the name of the index for the corresponding engine

Used In the Initialization of the Hawksearch client.

HawkSearchSearchingAPI

The endpoint at which search request are sent when using the Hawksearch widgets

Used In the Initialization of the Hawksearch client.

HawkSearchBaseAPI

The endpoint which exposes the Hawksearch Workbench

Used In the Initialization of the Hawksearch client.

HawkSearchClientID

The client Id used to Identify against the Hawksearch engine when making search requests

Used In the Initialization of the Hawksearch client.

HawkSearchStagingDomain

The URL for the Hawksearch staging domain.

HawkSearchProductionDomain

The URL for the Hawksearch production domain.

Run Rebuild on Server

Choose if Full or Delta export should trigger a rebuild on the Hawksearch engine.

Default value: True

IsTestMode

Specifies whether we are in test mode or not in order to use the proper staging domain or production domain.

Authentication

WindowsAuthLogin

Used to identify the client username against the engine when using Windows authentication

WindowsAuthPassword

Used to identify the client password against the engine when using Windows authentication

AuthorizationHeaderValue

Used to specify the authorization header value. Endpoints marked with the [Authorize] attribute check whether the request contains the specified header and are declined if not

Additional configurations

UseHawkSearchAPIAfterExport

Used to specify whether a rebuild of the index should be executed after the export of the files.

RenderDocumentLinksAsDetailPages

Specifies whether you should be forwarded to the detail page of the search result you have clicked on.

Publication Date Formatted

Used to specify whether this format [yyyy/MM/dd hh:mm] should be applied to the field Publication Date when getting Dynamic or Static type fields needed when accessing the index settings.

SearchProxyURL

When using V4L, the field must contain - your-site-domain/hawk/search/index

RenderRelatedDataIdentifiesAs

Specifies which related data field should be rendered when indexing, you can choose between “guid”, “title” and “url-name”.

ListPageUrl [Obsolete]

Specifies the URL for listing the result

External domain configuration
RunExportTaskOnSpecifiedDomainOnly

Used to specify whether the export tasks should be ran on a particular domain.

Host

Specifies the hostname where the export will to run.

Example: https://admin.mysite.com

Timeout

Specifies the web request timeout in seconds when running the export tasks on the specified domain.

AppKey

Specifies an identification key used to validate the user when the export should run on an external host.

FTP and SFTP configurations

PushExportFilesToFTP

Specifies whether the export files should be pushed over FTP or SFTP.

UseSFTP

Specifies whether the files should be pushed over SFTP instead of FTP. Must be used in addition to the above configuration.

FTP Host

Specifies the FTP host when creating the FTPConnectionString and connecting to the FTP.

SFTP Port

Specifies the SFTP port in the FTPConnectionString when transferring files over SFTP.

SFTP User

Specifies the FTP username when configuring the FTPConnectionString.

FTP Password

Specifies the FTP password when configuring the FTPConnectionString.

FTP Path

Specifies the file path on the FTP or SFTP host to place the exported files.

Multilingual functionality explained

Goal
Show how the multilingual functionality is working and what changes it makes to the indexing process.

How does Multilingual work?

The multilingual functionality is an easy way for sorting an filtering the content based on a language of the content items.

When enabling the multilingual and creating content on different languages, the items are indexed in a different languages, so the items can be divided based on the language.

What it does behind the scene is:

When the Reindex is triggered the UpdateIndex method is called. There is a check whether the multilingual field is selected or not.
If the field is selected, the method is called ProcessMultilingualFields(). There you loop through the fields of the document and a suffix is added to those that are different from language, permissions and denials.
Example:

As mentioned earlier to almost every field is added a language suffix. The reason is that there are three fields that do not need suffix: permissions, denials and language.

The permissions and denials are two fields that are added to the document, only when the Filter by permissions functionality is on.

The third field that remains unchanged is “language”. Its value carries language information (language abbreviation) and there is no need to add a suffix.

How multilingual works in the dashboard

In order the language options to work properly, Visitor targets should be set in the Workbench of the engine.

The Visibility target is a rule on how to divide the documents, based on a condition. There should be set as a parameter the “language“ and as value the chosen languages. For each language there should be a separate visitor target.

📘
Note
If the Multilingual is enabled, but there is only one language, there will be no menu with the language options, but the documents fields will still be indexed with the suffixes.

Extending the Multilingual functionality

Goal
This article is written for developers to show how the multilingual functionality can be extended. When indexing, with enabled multilingual, the fields of the document's are concatenated with a language suffix.

There might be some fields, that do no need to have this suffix and this article is going to explain

How to extend multilingual functionality

Currently, the basis of multilingual functionality is the addition of field extensions. Of course, users may want to have fields that do not have a suffix added. In the logic of multilingual there are 4 fields for which no suffix is added. These fields are - language, pageurl, permissions, denials. The addition of suffixes can be controlled by extending the method GetMultilingualDocuments

In the example below is shown how the field pageurl can be left without a suffix. The same will be with any other field. It should be just added in the check in the AddLanguageSuffix method (line 65).

🚧
In order for the indexing to work properly for multilingual and Filter by Permissions , it is necessary for these 4 fields (permissions, denials, language, pageurl) to remain without suffixes as in the sample below.

using System;
using System.Collections.Generic;
using System.Linq;
using Hawksearch.SDK.Indexing;
using Hawksearch.Search;
using Telerik.Sitefinity.Services;

namespace SitefinityWebApp.Custom
{
    public class CustomSearchService : HawksearchService
    {
        protected override List<SubmitDocument> GetMultilingualDocuments(IEnumerable<SubmitDocument> documents)
        {
            var documentList = new List<SubmitDocument>();
            var languages = SystemManager.CurrentContext.AppSettings.DefinedFrontendLanguages.Select(language => language.Name).ToList();

            foreach (var document in documents)
            {
                var language = this.GetDocumentLanguage(document, languages);

                if (!languages.Contains(language))
                {
                    continue;
                }

                this.AddLanguageSuffix(document, language);
                this.SetLanguageField(document, language);

                documentList.Add(document);
            }

            return documentList;
        }

        private void SetLanguageField(SubmitDocument document, string language)
        {
            var languageField = document.Fields.ToList().FirstOrDefault(f => f.Name == "Language");

            if (languageField != null)
            {
                if (string.IsNullOrWhiteSpace(languageField.Values.FirstOrDefault()))
                {
                    languageField.Values[0] = language;
                }
            }
        }

        private void AddLanguageSuffix(SubmitDocument document, string language)
        {
            foreach (var documentField in document.Fields)
            {
                var languageSuffix = "_" + language;

                if (documentField.Name == "Id")
                {
                    var id = documentField.Values.FirstOrDefault();

                    if (!string.IsNullOrWhiteSpace(id))
                    {
                        id += languageSuffix;
                        documentField.Values.Clear();
                        documentField.Values.Add(id);
                    }
                }
                else if (string.Equals(documentField.Name, "language", StringComparison.InvariantCultureIgnoreCase)
                         || string.Equals(documentField.Name, "pageurl", StringComparison.InvariantCultureIgnoreCase)
                         || string.Equals(documentField.Name, "permissions", StringComparison.InvariantCultureIgnoreCase)
                         || string.Equals(documentField.Name, "provider", StringComparison.InvariantCultureIgnoreCase)
                         || string.Equals(documentField.Name, "denials", StringComparison.InvariantCultureIgnoreCase))
                {
                }
                else
                {
                    documentField.Name += languageSuffix;
                }
            }
        }

        private string GetDocumentLanguage(SubmitDocument document, List<string> languages)
        {
            var language = string.Empty;
            var languageField = document.Fields.ToList().FirstOrDefault(f => f.Name == "Language");

            if (languageField != null)
            {
                language = languageField.Values.FirstOrDefault();

                if (languages.Count == 1 && string.IsNullOrWhiteSpace(language))
                {
                    language = languages.FirstOrDefault();
                }
            }

            return language;
        }
    }
}

🚧
Once you implement the code in Visual Studio , build your solution and you will also have to reindex the index you are using from Administrator → Search Indexes → Action → Reindex

📘
Expected results
Now if you Inspect your frontend page you should be able to find the language suffix fields you have added to your document in the XHR search → results → document fields

Process Taxonomies for Media Content

Due to Sitefinity's limitations, media content taxonomies cannot be resolved out of the box from the connector and a custom search service should be created for this.

This article provides an example of extending the search service and developing a logic for exporting media content taxonomies.

📘
Register Custom Search Service
In order to use your custom search service instead of the built-in one you need to register it in the backend.

Add Taxonomy Fields to the Index

Go to Sitefinity’s Search indexes backend page and open the the desired index (your-website-domain/Sitefinity/Administration/Search).
In the Advanced section of the search index editor add - Category, Tags
Add a Tag or a Category to one of your Media Content - Documents

📘
Setup search service
In order to add taxonomy fields to the index you need to create a custom search service which inherits the HawkseachService class and overrides the CreateIndex and UpdateIndex methods. Please refer to the code snippet below.

🚧
Field names should be exact, as the additional fields are case sensitive.

Process Media Content Taxonomies

using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Linq;
using Hawksearch.Search;
using Telerik.OpenAccess;
using Telerik.Sitefinity.Modules.Libraries;
using Telerik.Sitefinity.Publishing;
using Telerik.Sitefinity.Services.Search.Data;
using Telerik.Sitefinity.Taxonomies;
using Telerik.Sitefinity.Taxonomies.Model;

namespace SitefinityWebApp.Search
{
    public class CustomSearchService : HawksearchService
    {
        private const string DocumentType = "Telerik.Sitefinity.Libraries.Model.Document";

        public override void UpdateIndex(string name, IEnumerable<IDocument> documents)
        {
            var documentList = documents.ToList();
            var contentType = string.Empty;
            var doc = documentList.FirstOrDefault();

            if (doc != null && doc.Fields.FirstOrDefault(f => f.Name == "ContentType") != null)
            {
                var contentTypeField = doc.Fields.FirstOrDefault(f => f.Name == "ContentType").Value;

                if (contentTypeField != null)
                {
                    contentType = contentTypeField.ToString();
                }
            }

            if (string.Equals(contentType, DocumentType, StringComparison.InvariantCultureIgnoreCase))
            {
                var librariesManager = LibrariesManager.GetManager();
                var taxonomyManager = TaxonomyManager.GetManager();
                var mediaDocuments = librariesManager.GetDocuments().ToList();
                var taxonomies = taxonomyManager.GetTaxa<Taxon>().ToList();

                foreach (var document in documentList.Cast<Telerik.Sitefinity.Services.Search.Model.Document>())
                {
                    var id = document.Fields.FirstOrDefault(f => f.Name == "Id");
                    var documentId = Guid.Empty;

                    if (id != null)
                    {
                        documentId = Guid.Parse(id.Value.ToString());
                    }

                    var properties = TypeDescriptor.GetProperties(mediaDocuments.FirstOrDefault(m => m.Id == documentId));

                    foreach (PropertyDescriptor property in properties)
                    {
                        if (property != null)
                        {
                            if (property.PropertyType == typeof(TrackedList<Guid>))
                            {
                                var taxonomyIds = mediaDocuments.FirstOrDefault(m => m.Id == documentId).GetPropertyValue<TrackedList<Guid>>(property.Name);
                                var taxonomyNames = new List<string>();

                                foreach (var taxonomyId in taxonomyIds)
                                {
                                    var taxonName = string.Empty;
                                    var taxon = taxonomies.FirstOrDefault(t => t.Id == taxonomyId);

                                    if (taxon != null)
                                    {
                                        taxonName = taxon.Title;
                                    }

                                    taxonomyNames.Add(taxonName);
                                }

                                if (document.Fields.FirstOrDefault(f => f.Name == property.Name) != null)
                                {
                                    document.Fields.FirstOrDefault(f => f.Name == property.Name).Value = taxonomyNames.ToArray();
                                }
                            }
                        }
                    }
                }
            }

            base.UpdateIndex(name, documentList);
        }
    }
}

🚧
Once you implement the code in Visual Studio , build your solution and you will also have to reindex the index you are using from Administrator → Search Indexes → Action → Reindex

📘
Find the taxonomies you have added to your Media Content
Now if you Inspect your frontend page you should be able to find the tag or the category fields you have added to your document in the XHR search → results → document fields

Installation and Setup

Installing the Connector

📘
Prerequisite

🚧
Important

Steps to install the connector

🚧
Important

Activation

Steps to Configure Sitefinity Connector

Applying Settings

Steps for Integrating Your Hawksearch Instance

Settings Explanations

Versioning

Document size

Logging information

Deferred index

Tracking

Provider configuration

Export information

Filter results

Export directory for data files

Scheduling

Task keys

API keys and endpoints

Authentication

Additional configurations

FTP and SFTP configurations

Multilingual functionality explained

📘
Note

Extending the Multilingual functionality

🚧

🚧

📘
Expected results

Process Taxonomies for Media Content

📘
Register Custom Search Service

Add Taxonomy Fields to the Index

📘
Setup search service

🚧

Process Media Content Taxonomies

🚧

📘
Find the taxonomies you have added to your Media Content

Installing the Connector

📘Prerequisite

🚧Important

Steps to install the connector

🚧Important

Activation

Steps to Configure Sitefinity Connector

Applying Settings

Steps for Integrating Your Hawksearch Instance

Settings Explanations

Versioning

Document size

Logging information

Deferred index

Tracking

Provider configuration

Export information

Filter results

Export directory for data files

Scheduling

Task keys

API keys and endpoints

Authentication

Additional configurations

FTP and SFTP configurations

Multilingual functionality explained

📘Note

Extending the Multilingual functionality

🚧

🚧

📘Expected results

Process Taxonomies for Media Content

📘Register Custom Search Service

Add Taxonomy Fields to the Index

📘Setup search service

🚧

Process Media Content Taxonomies

🚧

📘Find the taxonomies you have added to your Media Content

📘
Prerequisite

🚧
Important

🚧
Important

📘
Note

📘
Expected results

📘
Register Custom Search Service

📘
Setup search service

📘
Find the taxonomies you have added to your Media Content