Elasticsearch search integration set up

Using an existing index

1. In the Select index section, click Use my index to connect to an existing Elasticsearch index. The Use my index option is selected as default in your Elasticsearch set up.

2. In the Use my index option, type the Elasticsearch index name.

3. You can see the Requirements not met error message when your Elasticsearch instance cannot connect to the ELSER semantic embeddings model or does not support the ELSER semantic embeddings model.

4. Click Next to go to the Enable conversational search (optional) section.

5. Conversational search is available only in the Plus and Enterprise plans of watsonx Assistant. In the Enable conversational search (optional) section, switch the Conversational Search toggle to on if you want to activate conversational search. If you don't want to activate conversational search, switch the toggle to off. For more information about conversational search, see conversational search.

6. Click Save and then Close.

Uploading documents to a new index

Before you upload documents, you must meet the following requirements:

• Upload document feature requires Elasticsearch 8.8 or above because it uses ELSER model and the ELSER model is available since Elasticsearch 8.8.

• Your Elasticsearch instance must have a paid or trial subscription, such as the Platinum Edition of IBM Cloud Databases for Elasticsearch or a Platinum or Enterprise subscription offered by Elastic.co.

• To use the ELSER model, you must have an Machine Learning (ML) node feature and a minimum of 4 GB memory in the ML node to deploy the model. For more information about ELSER requirements, see ELSER requirements.

• You can increase the number_of_allocations and threads_per_allocation to improve the throughput and latency of the ELSER model when you deploy it. For more information about start trained model deployment, see Start trained model.

To upload documents to a new index, use the following procedure:

1. In the Select index section of the Elasticsearch window, click Upload documents to a new index in your Elasticsearch instance.

   watsonx Assistant passes the uploaded documents to your Elasticsearch instance for storage, chunking, and indexing.

2. In the Configure result content section, provide the following fields to map the title, body, and URL to the search response:

   • Title

     Search result title. Use the title, name, or similar type of field from the collection as the search result title.

     You must select something for the title or no search result response is displayed in the Facebook and Slack integrations.

   • Body

     Search result description. Use an abstract, summary, or highlight field from the collection as the search result body.

     You must select something for the body or no search result response is displayed in the Facebook and Slack integrations.

   • URL

     This field can be populated with any footer content that you want to include at the end of the search result.

   When you configure the query body in the Advanced Elasticsearch Settings to search the nested documents, you must ensure that the Title, Body, and URL are from the fields of the internal documents in your Elasticsearch index. For more information about using nested queries, see Elasticsearch nested query.

3. Expand the Advanced Elasticsearch settings section to see the following text boxes:

   • Configure the filter array for Elasticsearch

     You define the filter as an array of objects so that you can create filters to arrange the content per the query body.

   • Configure the query body for Elasticsearch

     The query body is used to manipulate the user requests into a format that the search expects. It controls the query forms, search fields, filters, and query size. In the REST API, the query body is an object representing the POST body for the _search request to Elasticsearch. THe query body has a "$QUERY" token to represent the customer's query, and a "$FILTER" token to represent the array of filters that are defined either in the search settings or at the step level.

     By default, Elasticsearch integration uses keyword search. You can use advanced search such as semantic search with ELSER, KNN dense vector search, or nested queries to search the nested documents. For more information about using different types of query body examples, see Query body examples.

     For more information about the Elasticsearch _search API request body, see Elasticsearch search API request body.

   You cannot customize the query body in the assistant with an existing Elasticsearch configuration.

4. Use the Message, No results found and Connectivity issue tabs to customize different messages to share with users based on the successfulness of the search.

   Search result messages

   Tab	Scenario	Example message
   Message	Search results are returned	I found this information that might be helpful:
   No results found	No search results are found	I searched my knowledge base for information that might address your query, but did not find anything useful to share.
   Connectivity issue	I was unable to complete the search for some reason	I might have information that could help address your query, but am unable to search my knowledge base at the moment.

5. Switch the Conversational Search toggle to on if you want to activate conversational search. If you don't want to activate conversational search, switch the toggle to off.

6. Conversational search is available only in the Plus and Enterprise plans of watsonx Assistant. If you switch the Conversational Search toggle to on, you can see the citation titles in your assistant responses. For more information about conversational search, see conversational search.

7. Click Save to save your settings.

8. Click the Documents tab in the Elasticsearch window.

   The Documents tab is enabled only if you select Upload documents to a new index in your Elasticsearch instance option.

9. Click Upload button. In the Upload documents section, you can drag and drop your files or do a single click to upload documents directly to your assistant.

   You can upload up to 20 documents at a time. Each document file must not exceed 25MB. The total size of all documents must not exceed 100MB.

   

10. After you upload the documents, you can see the upload status of your documents in a table in the Elasticsearch window.

11. Status Ready indicates that your files are available for search.

12. If the status indicates Error, you can delete the file by clicking the three dots next to the Error and click Delete.

    

13. If you want to change the Elasticsearch instance credentials, click the Instance tab, edit the credentials, and then click Save. Skip this step if you do not want to change Elasticsearch instance credentials.

14. Click Save and then Close to end the Elasticsearch set up.