Configuring an Enterprise Search 7.17 server with an Databases for Elasticsearch instance
Welcome to a guide on configuring a functional Enterprise Search 7.17 server integrated with an IBM Cloud® Databases for Elasticsearch instance. Elasticsearch is a powerful and versatile search and analytics engine that helps you store, search, and analyze large volumes of data quickly and in near-real-time. Elasticsearch is commonly used for log and event data analysis, full-text search, and various other use cases where data needs to be queried efficiently.
IBM Cloud® Databases for Elasticsearch is an Elasticsearch service that is offered by IBM Cloud that provides a managed and scalable solution for deploying and running Elasticsearch clusters. Databases for Elasticsearch simplifies the setup and maintenance of Elasticsearch, enabling you to focus on using the power of Elasticsearch without the complexities of managing the infrastructure. Kibana complements Elasticsearch by offering a flexible visualization platform. It allows you to explore, visualize, and share insights from your data, enabling you to create custom dashboards and visualizations to better understand your information.
Enterprise Search extends the capabilities of Databases for Elasticsearch to provide a unified search experience across various data sources, including documents, emails, databases, and more. It offers a seamless interface for users to find relevant information across disparate data silos, enhancing productivity and collaboration. By integrating Enterprise Search with your Databases for Elasticsearch instance, you gain a comprehensive search solution that uses the strengths of both platforms to efficiently discover insights from your data.
Prerequisites
Before you start the installation process, have the prerequisites in place:
-
Have a working instance of Databases for Elasticsearch and Kibana.
-
Java 11: If your preferred installation method is through a package, verify that Java 11 is installed on your system.
Setting Up Elasticsearch and Kibana Instances
-
Configure your Databases for Elasticsearch and Kibana instances.
-
Add the following line in your Kibana.yml file:
enterpriseSearch.host: ${ENTERPRISE_HOST_URL}
ENTERPRISE_HOST_URL
depends upon your installation. -
After successfully setting up the instances, obtain the following details:
ELASTICSEARCH_USERNAME
ELASTICSEARCH_PASSWORD
ELASTICSEARCH_HOST URL
KIBANA_HOST URL
CA Certificate
How to use our self-signed root certificate
-
In your instance's Overview, find the Endpoints section.
-
Copy the certificate information from the Endpoints panel, or use the Download Certificate option.
-
Save the certificate to a file. You can use the that is provided or your own file name.
-
If necessary, convert this file to PEM format by using the command:
openssl x509 -in <sourcefile> -out <targetfile>.pem
-
Provide the path to the certificate to the driver or client.
Configure Enterprise Search through Package
Configure Enterprise Search to integrate with your Databases for Elasticsearch instance.
-
Download Enterprise Search.
-
Extract the downloaded package to a suitable location.
-
Edit the Configuration File: Locate the
enterprise-search.yml
file within the config directory of the extracted package. Edit this file to incorporate the necessary settings. -
Add configuration settings: Inside the
enterprise-search.yml
file, add the following lines:allow_es_settings_modification: true secret_management.encryption_keys:['AWESOME_SECRET'] elasticsearch.username: ELASTICSEARCH_USERNAME elasticsearch.password: ELASTICSEARCH_PASSWORD elasticsearch.host: ELASTICSEARCH_HOST elasticsearch.ssl.enabled: true elasticsearch.ssl.certificate_authority: path/to/ca-cert kibana.external_url: KIBANA_HOST
Replace the placeholders with the details obtained in the previous step.
-
CA Certificate Placement: Ensure that the CA certificate you downloaded is placed in the specified path that is mentioned in the configuration.
-
Start Enterprise Search: Open your terminal and navigate to the location where you extracted the package. Start Enterprise Search by running the following command:
./bin/enterprise-search
-
To ensure that Enterprise Search keeps running, consider running it as a background process. Do this with
&
, using a command like:./bin/enterprise-search &
You can also use the
nohup
command:nohup <COMMAND> &
<COMMAND>
is the command that you want to run in the background.
Start Enterprise Search through Docker Run
In addition to the standard installation, start Enterprise Search by using Docker by following these steps:
-
Place CA Certificate: Copy the CA certificate into
certs/ca/ca.crt
within your project folder. -
Add the following:
-
Run Docker Command:
Run the following command in your project folder to start Enterprise Search with Docker:
docker run \ --name "enterprise-search" \ --network "elastic" \ --publish "3002:3002" \ --volume "./certs:/usr/share/enterprise-search/es-config/certs:ro" \ --interactive \ --tty \ --rm \ --env "allow_es_settings_modification=true" \ --env "elasticsearch.ssl.enabled=true" \ --env "elasticsearch.ssl.certificate_authority=/usr/share/enterprise-search/es-config/certs/ca/ca.crt" \ --env "elasticsearch.host=${ELASTICSEARCH_HOST} \ --env "elasticsearch.username=${ELASTICSEARCH_USERNAME}" \ --env "elasticsearch.password=${ELASTICSEARCH_PASSWORD}" \ --env "kibana.external_url=${KIBANA_HOST}" \ --env "secret_management.encryption_keys=[${AWESOME_SECRET}]" \ "docker.elastic.co/enterprise-search/enterprise-search:7.17.7"
Replace placeholders with your actual values.
Starting Enterprise Search AND Kibana through Docker Compose
You can start Kibana and Enterprise Search together with docker-compose. Follow these steps:
-
Place CA Certificate: Copy the CA certificate into
certs/ca/ca.crt
within your project folder. -
Add the following to the
.env
file in your project directory:ELASTIC_VERSION=7.17.7 ELASTICSEARCH_HOSTS=${ELASTICSEARCH_HOST} ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD} ELASTICSEARCH_USERNAME=${ELASTICSEARCH_USERNAME} ENCRYPTION_KEYS=${AWESOME_SECRET} KIBANA_HOST=http://kibana:5601 ENTERPRISE_SEARCH_HOST=http://enterprisesearch:3002
-
Create a
docker-compose
file and add the following code to it:version: "2.2" services: kibana: image: docker.elastic.co/kibana/kibana:${ELASTIC_VERSION} volumes: - ./certs:/usr/share/kibana/config/certs ports: - 5601:5601 environment: - SERVERNAME=kibana - ELASTICSEARCH_HOSTS=${ELASTICSEARCH_HOSTS} - ELASTICSEARCH_USERNAME=${ELASTICSEARCH_USERNAME} - ELASTICSEARCH_PASSWORD=${ELASTICSEARCH_PASSWORD} - ELASTICSEARCH_SSL_CERTIFICATEAUTHORITIES=config/certs/ca/ca.crt - ENTERPRISE_SEARCH_HOST=${ENTERPRISE_SEARCH_HOST} enterprisesearch: depends_on: kibana: condition: service_healthy image: docker.elastic.co/enterprise-search/enterprise-search:${ELASTIC_VERSION} volumes: - ./certs:/usr/share/enterprise-search/config/certs ports: - 3002:3002 environment: - SERVERNAME=enterprisesearch - secret_management.encryption_keys=[${AWESOME_SECRET}] - allow_es_settings_modification=true - elasticsearch.host=${ELASTICSEARCH_HOSTS} - elasticsearch.username=${ELASTICSEARCH_USERNAME} - elasticsearch.password=${ELASTICSEARCH_PASSWORD} - elasticsearch.ssl.enabled=true - elasticsearch.ssl.certificate_authority=/usr/share/enterprise-search/config/certs/ca/ca.crt - kibana.external_url=${KIBANA_HOST}
The placeholders in the docker-compose.yml
file are automatically picked up from the .env
file. Make sure the .env
file is updated with actual details.
If you already have Kibana up and running, make sure to update the enterprise host url in the kibana.yml config.
Verify Integration:
After Enterprise Search is up and running, access it through your Kibana dashboard.
Remember, this guide provides a high-level overview of the setup process. For more information, see the official documentation for more detailed information and troubleshooting assistance. Happy searching!