HA Upgrade Guide

Overview

This document provides step-by-step instructions for upgrading the Motadata ServiceOps application to the latest version in a High Availability (HA) environment.

note

• From version v8.2, incremental upgrades are not required. You can upgrade directly from v8.2 to v8.6.0.
• Always perform a full database backup before upgrading.
• Starting from v8.6.0, you can upgrade ServiceOps using a single common installer across supported operating systems.

Supported OS Versions

• Ubuntu: 22, 24
• RedHat: 9.2, 9.4

Pre-Upgrade Checklist:

• Application Backup: Backup the application and filedb folder. Refer to the Application Backup documentation.
• Database Backup: Take a complete database backup. Refer to the DB Backup documentation.
• VM Snapshot: Take a snapshot of the virtual machine for recovery.

Upgrade Procedure

To upgrade the ServiceOps application for HA environment, perform the below steps:

Download the Release Build

Download the latest Common Installer build from the Motadata Installation and Upgrade Document.

Stop the HA Observer Service

Before upgrading, stop the HA Observer service to prevent automatic failover during the process.

Before stopping the Observer service, it is essential to monitor the logs to ensure that no swapping or synchronization process is in progress. Stopping the Observer service abruptly may lead to instability in the HA cluster.

When and How to Stop the Observer Service

You must stop the service within a 1-minute window once the current HA counter or process completes. Follow the below steps:

1. Monitor the HA Observer logs located at:

   /opt/HA/logs/

2. Use the following command to view the latest log activity:

   cd /opt/HA/logs

   Syntax: tail -f ha_<date>.log

   Example: tail -f ha_30_04_2025.log

   

3. As soon as the current counter process in the log ends and before the next one begins (ideally within a 1-minute window), stop the HA Observer service using the following commands:

   systemctl status ha
   systemctl stop ha
   systemctl status ha

caution

Do not stop the HA Observer service abruptly. Monitor the logs first to avoid HA state inconsistencies.

Ones the observer server service is stopped, you can now upgrade the HA environment.

Upgrade Master Application Instance

Prepare the Installer

1. Copy the MotadataServiceOpsCommonUpgrade installer to the Master ServiceOps application instance.

2. Log in to the terminal server with root privileges.

   sudo su

3. Stop the main server and analytics server services using the below command:

   systemctl stop ft-main-server.service ft-analytics-server.service

4. Check the service status using the below command:

   systemctl status ft-main-server.service ft-analytics-server.service

5. Grant execute permissions using the root user:

   chmod 777 service_desk_master_CI

Run the Installer

1. Run installer using the following command:

   ./MotadataServiceOpsCommonUpgrade_V860

The upgrade process will begin as shown below.

Once the upgrade process is completed successfully, the below screen will appear.

Post-Deployment Verification

1. Check the status of all the services.

   systemctl status ft-main-server.service
   systemctl status ft-analytics-server.service
   systemctl status ft-plugin-server.service (If system with any plugins)
   systemctl status ft-discovery-service.service (before v8.6.0)
   systemctl status ft-discovery-go.service (from v8.5.0 onwards)
   systemctl status ft-file-server.service (If File Server is deployed locally)
   systemctl status elasticsearch.service
   systemctl status nginx.service
   systemctl status postgresql.service
   systemctl status file-sync.service (If HA Architecture with Master Server)

2. Now, wait for at least 5 Minutes to ensure all services are up and running.

3. Log in to the Motadata ServiceDesk Portal and verify the application version from the Admin > Organization > Account > License Details tab.

4. Do the sanity check and verify functionality.

In case of upgrade failure,

• Collect the SSH Session Logs You can view the logs from the below location path:

  /opt/flotomate/main-server/logs/

It contains two folders: apollo (default tenant) and common

The apollo folder contains the following logs:

Admin	Discovery	Error	nsq
Agent	Elastic	Email	scheduler
csvimport	Apilogger	Events	service
Audit

The common folder contains the following logs:

Apilogger	error	Nsq	Service
Agent	events	Integration	shutdown

• Collect logs and evidence from these folders and share with Motadata QA/Dev team for troubleshooting.

Upgrade Stand by Application Instance

To upgrade the ServiceOps standby instance, repeat the steps from sections 2.3.1 to 2.3.3.

note

• After upgrade, check that all services are running.
• Then stop all application services except PostgreSQL using the below commands:

systemctl stop ft-main-server.service
systemctl stop ft-analytics-server

Start Observer Service

After both Master and Standby servers are upgraded, start the Observer Service using the below commands:

    systemctl status ha
    systemctl start ha
    systemctl status ha

Check High Availability Cluster Health Status

Once the upgrade is completed, verify that the HA setup is functioning correctly by ensuring the database and the filedb folder are properly synchronized.

Database Sync Check:

Check if the Master and Standby databases are in sync by comparing the request counts using the below command:

sudo -u postgres psql -d flotoitsmdb -tAc "SELECT COUNT(*) FROM apolo.request;"

tip

Create a test ticket on the Master server and confirm that the same record appears on the Standby to validate synchronization.

FileDB Sync Check:

In an HA setup, the filedb folder on the Master server is synchronized with the Standby server using rsync. This ensures that file attachments and other critical data remain consistent across both nodes.

1. Ensure the file-sync.service is running on the Master and stopped on the Standby.

2. Check the service status using the below command:

   systemctl status file-sync

3. Compare the number of files in the filedb directory on both Master and Standby using the below command:

find /opt/flotomate/main-server/filedb -type f | wc -l

The Master and Standby servers will have same filedb file count.

4. To verify, create a test file in the Master’s filedb folder and confirm it appears on the Standby node.