IBM Cloud Docs
Configuring an Enterprise Search 7.17 server with an Databases for Elasticsearch instance

Configuring an Enterprise Search 7.17 server with an Databases for Elasticsearch instance

Welcome to a guide on configuring a functional Enterprise Search 7.17 server integrated with an IBM Cloud® Databases for Elasticsearch instance. Elasticsearch is a powerful and versatile search and analytics engine that helps you store, search, and analyze large volumes of data quickly and in near-real-time. Elasticsearch is commonly used for log and event data analysis, full-text search, and various other use cases where data needs to be queried efficiently.

IBM Cloud® Databases for Elasticsearch is an Elasticsearch service that is offered by IBM Cloud that provides a managed and scalable solution for deploying and running Elasticsearch clusters. Databases for Elasticsearch simplifies the setup and maintenance of Elasticsearch, enabling you to focus on using the power of Elasticsearch without the complexities of managing the infrastructure. Kibana complements Elasticsearch by offering a flexible visualization platform. It allows you to explore, visualize, and share insights from your data, enabling you to create custom dashboards and visualizations to better understand your information.

Enterprise Search extends the capabilities of Databases for Elasticsearch to provide a unified search experience across various data sources, including documents, emails, databases, and more. It offers a seamless interface for users to find relevant information across disparate data silos, enhancing productivity and collaboration. By integrating Enterprise Search with your Databases for Elasticsearch instance, you gain a comprehensive search solution that uses the strengths of both platforms to efficiently discover insights from your data.

Prerequisites

Before you start the installation process, have the prerequisites in place:

Setting Up Elasticsearch and Kibana Instances

  1. Configure your Databases for Elasticsearch and Kibana instances.

  2. Add the following line in your Kibana.yml file:

    enterpriseSearch.host: ${ENTERPRISE_HOST_URL}
    

    ENTERPRISE_HOST_URL depends upon your installation.

  3. After successfully setting up the instances, obtain the following details:

  • ELASTICSEARCH_USERNAME
  • ELASTICSEARCH_PASSWORD
  • ELASTICSEARCH_HOST URL
  • KIBANA_HOST URL
  • CA Certificate

How to use our self-signed root certificate

  1. In your instance's Overview, find the Endpoints section.

  2. Copy the certificate information from the Endpoints panel, or use the Download Certificate option.

  3. Save the certificate to a file. You can use the that is provided or your own file name.

  4. If necessary, convert this file to PEM format by using the command:

    openssl x509 -in <sourcefile> -out <targetfile>.pem
    
  5. Provide the path to the certificate to the driver or client.

Configure Enterprise Search through Package

Configure Enterprise Search to integrate with your Databases for Elasticsearch instance.

  1. Download Enterprise Search.

  2. Extract the downloaded package to a suitable location.

  3. Edit the Configuration File: Locate the enterprise-search.yml file within the config directory of the extracted package. Edit this file to incorporate the necessary settings.

  4. Add configuration settings: Inside the enterprise-search.yml file, add the following lines:

    allow_es_settings_modification: true
    secret_management.encryption_keys:['AWESOME_SECRET']
    elasticsearch.username: ELASTICSEARCH_USERNAME
    elasticsearch.password: ELASTICSEARCH_PASSWORD
    elasticsearch.host: ELASTICSEARCH_HOST
    elasticsearch.ssl.enabled: true
    elasticsearch.ssl.certificate_authority: path/to/ca-cert
    kibana.external_url: KIBANA_HOST
    

    Replace the placeholders with the details obtained in the previous step.

  5. CA Certificate Placement: Ensure that the CA certificate you downloaded is placed in the specified path that is mentioned in the configuration.

  6. Start Enterprise Search: Open your terminal and navigate to the location where you extracted the package. Start Enterprise Search by running the following command:

    ./bin/enterprise-search
    
  7. To ensure that Enterprise Search keeps running, consider running it as a background process. Do this with &, using a command like:

    ./bin/enterprise-search &
    

    You can also use the nohup command:

    nohup <COMMAND> &
    

    <COMMAND> is the command that you want to run in the background.

Start Enterprise Search through Docker Run

In addition to the standard installation, start Enterprise Search by using Docker by following these steps:

  1. Place CA Certificate: Copy the CA certificate into certs/ca/ca.crt within your project folder.

  2. Add the following:

  3. Run Docker Command:

    Run the following command in your project folder to start Enterprise Search with Docker:

    docker run \
    --name "enterprise-search" \
    --network "elastic" \
    --publish "3002:3002" \
    --volume "./certs:/usr/share/enterprise-search/es-config/certs:ro" \
    --interactive \
    --tty \
    --rm \
    --env "allow_es_settings_modification=true" \
    --env "elasticsearch.ssl.enabled=true" \
    --env "elasticsearch.ssl.certificate_authority=/usr/share/enterprise-search/es-config/certs/ca/ca.crt" \
    --env "elasticsearch.host=${ELASTICSEARCH_HOST} \
    --env "elasticsearch.username=${ELASTICSEARCH_USERNAME}" \
    --env "elasticsearch.password=${ELASTICSEARCH_PASSWORD}" \
    --env "kibana.external_url=${KIBANA_HOST}" \
    --env "secret_management.encryption_keys=[${AWESOME_SECRET}]" \
    "docker.elastic.co/enterprise-search/enterprise-search:7.17.7"
    

    Replace placeholders with your actual values.

Starting Enterprise Search AND Kibana through Docker Compose

You can start Kibana and Enterprise Search together with docker-compose. Follow these steps:

  1. Place CA Certificate: Copy the CA certificate into certs/ca/ca.crt within your project folder.

  2. Add the following to the .env file in your project directory:

    ELASTIC_VERSION=7.17.7
    ELASTICSEARCH_HOSTS=${ELASTICSEARCH_HOST}
    ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD}
    ELASTICSEARCH_USERNAME=${ELASTICSEARCH_USERNAME}
    ENCRYPTION_KEYS=${AWESOME_SECRET}
    KIBANA_HOST=http://kibana:5601
    ENTERPRISE_SEARCH_HOST=http://enterprisesearch:3002
    
  3. Create a docker-compose file and add the following code to it:

    version: "2.2"
    services:
      kibana:
        image: docker.elastic.co/kibana/kibana:${ELASTIC_VERSION}
        volumes:
          - ./certs:/usr/share/kibana/config/certs
        ports:
          - 5601:5601
        environment:
          - SERVERNAME=kibana
          - ELASTICSEARCH_HOSTS=${ELASTICSEARCH_HOSTS}
          - ELASTICSEARCH_USERNAME=${ELASTICSEARCH_USERNAME}
          - ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD}
          - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt
          - ENTERPRISE_SEARCH_HOST=${ENTERPRISE_SEARCH_HOST}
    
      enterprisesearch:
        depends_on:
          kibana:
            condition: service_healthy
        image: docker.elastic.co/enterprise-search/enterprise-search:${ELASTIC_VERSION}
        volumes:
          - ./certs:/usr/share/enterprise-search/config/certs
        ports:
          - 3002:3002
        environment:
          - SERVERNAME=enterprisesearch
          - secret_management.encryption_keys=[${AWESOME_SECRET}]
          - allow_es_settings_modification=true
          - elasticsearch.host=${ELASTICSEARCH_HOSTS}
          - elasticsearch.username=${ELASTICSEARCH_USERNAME}
          - elasticsearch.password=${ELASTICSEARCH_PASSWORD}
          - elasticsearch.ssl.enabled=true
          - elasticsearch.ssl.certificate_authority=/usr/share/enterprise-search/config/certs/ca/ca.crt
          - kibana.external_url=${KIBANA_HOST}
    

The placeholders in the docker-compose.yml file are automatically picked up from the .env file. Make sure the .env file is updated with actual details.

If you already have Kibana up and running, make sure to update the enterprise host url in the kibana.yml config.

Verify Integration:

After Enterprise Search is up and running, access it through your Kibana dashboard.

Remember, this guide provides a high-level overview of the setup process. For more information, see the official documentation for more detailed information and troubleshooting assistance. Happy searching!