ElasticSearch 7 installation or upgrade

Issues with Open Distro

We previously recommended Open Distro as part of our on-premise installation prerequisites for Elasticsearch. However, the Open Distro project has been deprecated and has not received security or feature updates since May 11, 2022.

The requirements for its replacement, OpenSearch, do not natively support a simple Windows installer. On Windows systems, OpenSearch requires a Docker deployment. Because Docker introduces significant operational overhead and infrastructure complexity, we do not officially support OpenSearch for on-premise installations.

Instead, we recommend that on-premise clients wishing to keep their environments secure and up to date migrate to Elasticsearch 7. This is the original version of the product and not an open-source variant.

Security considerations

Elasticsearch 7 in this recommended configuration will no longer use a username/password or HTTPS to access the cluster. Instead, Elasticsearch only needs to be accessed locally by the ThoughtFarmer application on http://localhost:9200 from the host server itself.

Setting up strict firewall rules to block all inbound external requests to port 9200 is sufficient to secure your cluster. Please apply any additional infrastructure hardening your organization requires.

If an encrypted HTTPS connection and credential authentication are strict requirements for your organization, you must either remain on your legacy Open Distro deployment or plan a migration to our fully managed Cloud environment. An archived version of Open Distro can be provided by our Support team upon request.

Installation instructions

1. Open Windows Services (services.msc) on your application server and locate the active Elasticsearch service.
2. Note the service identifier, which appears in brackets beside the friendly name (e.g., Elasticsearch 2.4.6 (es2)).
   
3. If the identifier in brackets is (elasticsearch-service-x64), you must remove it. Stop the service first, then open a Command Prompt (cmd) in Administrator mode and execute:

   • sc delete elasticsearch-service-x64
     

   • NOTE: Deleting the service means a rollback will require reinstalling the legacy service from scratch.
4. Refresh the Windows Services console and confirm the service has been successfully removed.
5. If the service identifier is anything else, simply change its Startup type to Disabled and stop the service. (You may skip this step entirely if this is a fresh installation on a new server).

Download and Extract Elasticsearch 7

1. Identify the latest release of Elasticsearch 7 by visiting the Elasticsearch Past Releases page. 
   1. NOTE: Note: Elasticsearch 8 or later is not supported by ThoughtFarmer. This document assumes version 7.17.29.
2. Download the Windows ZIP package for your chosen 7.x version.
3. Extract the contents of the ZIP package and copy the root folder directly to the root of your server directory (e.g., C:\elasticsearch-7.17.29).

Configure Environment Variables

1. Open your system environment variables configuration.
2. Locate your existing JAVA_HOME variable, copy its value, and create a new system variable named ES_JAVA_HOME using that identical path.

   1. Note: If this is a new server deployment and Java is missing, you must install a supported Java SDK first. We recommend Amazon Corretto 17.

3. Once the Java installation is complete, point your ES_JAVA_HOME system variable to the root directory of your JDK installation (e.g., C:\Program Files\Amazon Corretto\jdk17.0.8_8).

Install and Start the Windows Service

1. Open PowerShell in Administrator mode and navigate to your binaries directory:

2. cd c:/elasticsearch-7.17.29/bin

3. Execute the service installation script:

4. ./elasticsearch-service.bat install

5. Note: If an existing service named elasticsearch-service-x64 was left on the server and not uninstalled during Step 1, you must assign a unique SERVICE_ID variable (e.g., elasticsearch_7) before running the install script. Refer to the Elastic Official Windows Service Guide for details.
6. Start the newly installed service:

7. ./elasticsearch-service.bat start

8. Return to Windows Services and locate your new service. Ensure its Startup type is explicitly configured to Automatic.
9. Verify local access to the cluster by opening a browser or running a request to: http://localhost:9200/_cat/indices

ThoughtFarmer configuration instructions

1. Back up the Database: Back up your database's ConfigSetting table before making direct edits to accelerate recovery times if a rollback is needed. 
2. Open SQL Server Management Studio (SSMS) and execute the following query against your ThoughtFarmer database to update the search endpoint paths:

3. UPDATE ConfigSetting
   SET Value = 'http://localhost:9200'
   WHERE Name = 'elasticSearch.url';
   
   UPDATE ConfigSetting
   SET Value = ''
   WHERE Name = 'elasticSearch.password';
   
   UPDATE ConfigSetting
   SET Value = ''
   WHERE Name = 'elasticSearch.username';
   
   UPDATE ConfigSetting 
   SET Value = 'ES7X' 
   WHERE Name = 'elasticSearch.versions.enabled';

4. Recycle both the main ThoughtFarmer application pool and the formsapi application pool within IIS.
5. Log in to your ThoughtFarmer site as an administrator and navigate to Admin Panel > Search Settings.
6. Configure the following reindex parameters precisely:
   1. Select Delete index before rebuild.
   2. Select Reindex form content (ensure all form data checkboxes are checked).
   3. Click Rebuild now, with delete index, then click Continue.

7. Monitor the indexation process. Once completed, your items should exit the processing queue successfully.

8. Re-verify http://localhost:9200/_cat/indices. The output will now display both your primary ThoughtFarmer content indices and your activity indices.

9. Confirm that your system health check page returns a healthy status by navigating to: https://yourThoughtFarmerURL.com/status
    

Rollback plan

If unexpected complications occur during validation and you need to revert immediately to your previous working configurations, execute the following steps:

1. Open SSMS and execute the rollback query below to restore your original settings from your backup table:
   (Note: Verify that your backup table is named ConfigSettingBackup before executing).

2. UPDATE ConfigSetting
   SET Value = (SELECT Value FROM ConfigSettingBackup WHERE Name = 'elasticSearch.url')
   WHERE Name = 'elasticSearch.url';
   
   UPDATE ConfigSetting
   SET Value = (SELECT Value FROM ConfigSettingBackup WHERE Name = 'elasticSearch.password')
   WHERE Name = 'elasticSearch.password';
   
   UPDATE ConfigSetting
   SET Value = (SELECT Value FROM ConfigSettingBackup WHERE Name = 'elasticSearch.username')
   WHERE Name = 'elasticSearch.username';
   
   UPDATE ConfigSetting
   SET Value = (SELECT Value FROM ConfigSettingBackup WHERE Name = 'elasticSearch.versions.enabled')
   WHERE Name = 'elasticSearch.versions.enabled';
   

3. Open Windows Services, locate your newly installed Elasticsearch 7 service, and Disable it.
4. Locate your legacy Elasticsearch/Open Distro service, set its Startup type back to Automatic, and Start it.
5. Note: If you executed the sc delete command during earlier part of the installation, you must reinstall your archived version of the legacy service before starting it.
6. Recycle your IIS application pools to complete the rollback process.