Upgrade Guide for Standalone Deployment

A comprehensive guide for upgrading ServiceOps servers across different deployment architectures with minimal downtime and maximum reliability.

In a standalone ServiceOps environment, all components, including the application server, database, and other services, are hosted on a single server. This consolidated setup simplifies the upgrade process, as all actions are performed on one machine. This guide provides a unified procedure to update all components seamlessly.

ServiceOps server upgrades can be performed on standalone servers with specific considerations for the environment. From version v8.2 onwards, incremental upgrades are not required - you can upgrade directly from v8.2 to the latest version.

Key Upgrade Features

• Single Installer: From v8.6.0, use a single common installer across all supported operating systems
• Direct Upgrades: Skip incremental versions (e.g., v8.2 to latest available release version directly)
• Cross-Platform Support: Ubuntu 24 and RedHat 9.4
• Minimal Downtime: Structured upgrade procedures to minimize service interruption

Pre-Upgrade Requirements

System Requirements

Supported Operating Systems:

• Ubuntu: 22, 24
• RedHat: 9.2, 9.4

Pre-Upgrade Checklist

Before initiating any upgrade, complete these essential steps:

• Application Backup: Backup the application and filedb folder
• Database Backup: Take a complete database backup
• VM Snapshot: Take a snapshot of the virtual machine for recovery
• Verify Disk Space: Ensure sufficient space for upgrade files and backups
• Review Release Notes: Check for any breaking changes or new requirements
• Schedule Maintenance Window: Plan upgrade during low-usage periods

Backup Requirements

Backup Requirements: Always perform complete backups before upgrading. Refer to the Backup Procedure for detailed steps.

Application Upgrade Steps

Step 1: Download the Release Build

1. Download the latest Common Upgrade Build from the Download Link

Step 2: Prepare the Installer

1. Copy the MotadataServiceOpsCommonUpgrade installer to the target machine

2. Login to the terminal server with root user rights:

   sudo su

3. Stop the main server and analytics server services:

   systemctl stop ft-main-server.service ft-analytics-server.service

   

4. Check the status of the services:

   systemctl status ft-main-server.service ft-analytics-server.service

   

5. Grant execute permissions for the installer file:

   chmod 777 MotadataServiceOpsCommonUpgrade_V860

Installer Filename

Replace MotadataServiceOpsCommonUpgrade_V860 with the actual filename of the installer you downloaded. The version number may vary.

Step 3: Run the Installer

1. Execute the installer:

   ./MotadataServiceOpsCommonUpgrade_V860

2. The upgrade procedure will begin automatically

3. Once you see the completion screen, the upgrade is successful

   

Database Upgrade Steps (Optional)

When to Upgrade the Database

In a standalone deployment, the database upgrade is optional. You should only perform these steps if a specific ServiceOps version requires a newer version of PostgreSQL.

Pre-Upgrade Checklist

• Check Disk I/O Speed: Ensure that the I/O speed is greater than 150 MB/s:

  dd if=/dev/zero of=/tmp/test1.img bs=1G count=1 oflag=dsync
• Check Available Disk Space:

  df -h
• Check Memory Availability:

  free -h
• Clear Cache:

  sudo sync; sudo sysctl -w vm.drop_caches=3
• Backup and Snapshot: Take a full database backup along with the config file and create VM snapshot.

Step 1: Stop All Services

Before upgrading the database, stop all running ServiceOps services to avoid conflicts.

1. Stop Services: Log in to the terminal and stop all services:

   sudo systemctl stop ft-analytics-server.service ft-main-server.service ft-chat-server.service ft-ai-server.service ft-file-server.service ft-plugin-server.service meshcentral.service

   

2. Verify Service Status: Verify that all services are stopped:

   sudo systemctl status ft-analytics-server.service ft-main-server.service ft-chat-server.service ft-ai-server.service ft-file-server.service ft-plugin-server.service meshcentral.service

   If any service is still running, repeat the stop command.

Step 2: Prepare for PostgreSQL Upgrade

After ensuring that all services are stopped, follow these steps to upgrade PostgreSQL:

1. Download Upgrade Tool: Download the MotadataDatabaseUpgrade file from the Download Links.

2. Login and Switch to Root: Log in to the server and switch to the root user.

3. Set Permissions: Provide the necessary execution permissions:

   chmod 777 MotadataPGDatabaseUpgrade

   

4. Run Installer: Execute the PostgreSQL upgrade installer:

   ./MotadataPGDatabaseUpgrade

   

5. Respond to Upgrade Prompts: The script will prompt for upgrade options. Enter y for the relevant version upgrade.

   Upgrade PostgreSQL 16 → 17? (y/n):
   Upgrade PostgreSQL 11 → 17? (y/n):

   

6. Wait for Completion: The upgrade may take some time. Do not interrupt the process.

Step 3: Verify PostgreSQL Upgrade

Once the upgrade is installed, check the PostgreSQL version to confirm it was successful.

1. Switch to Postgres User:

   sudo su
   su postgres

2. Open PostgreSQL Console and Check Version:

   psql

3. Check Cluster Status:

   pg_lsclusters

4. Check Version:

   select version();

   Ensure that the version displayed matches the expected upgraded version.

Step 4: Restart Services

Once the PostgreSQL version is upgraded successfully, restart all the previously stopped services:

sudo systemctl start ft-analytics-server.service ft-main-server.service ft-chat-server.service ft-ai-server.service ft-file-server.service ft-plugin-server.service meshcentral.service

Post-Upgrade Verification

1. Check the status of all services:

   systemctl status ft-main-server.service ft-analytics-server.service

2. Wait for at least 5 minutes to allow restart of all Motadata services

3. Login to the ServiceOps portal and verify the application version from Admin > Organization > Account > License Details

   

4. Perform sanity checks and verify the functionality.

System Health Checks

1. Service Status: Verify all services are running
2. Application Access: Test login and basic functionality
3. Database Connectivity: Confirm database connections
4. File Operations: Test file upload/download functionality
5. Agent Communication: Verify agent connectivity and discovery

Performance Monitoring

1. System Resources: Monitor CPU, memory, and disk usage
2. Response Times: Check application response times
3. Error Rates: Monitor error logs for any issues
4. User Feedback: Gather feedback from end users

Troubleshooting Upgrade Issues

Common Upgrade Failures

Click to expand Common Upgrade Failures

1. Service Startup Failures: Check service status and logs
2. Database Connection Issues: Verify PostgreSQL service and connectivity
3. File Permission Errors: Ensure proper file permissions and ownership
4. Insufficient Disk Space: Check available disk space before upgrade

Log Collection for Troubleshooting

Click to expand Log Collection for Troubleshooting

If upgrade fails, collect logs from the following locations:

Main Log Directory:

/opt/flotomate/main-server/logs/

Apollo Folder Logs (Default Tenant):

Log Category	Description	Purpose
Admin	Administrative operations	User management, permissions, system configuration
Discovery	Asset and service discovery	Network scanning, device detection, inventory updates
Error	Error tracking and debugging	Application errors, exceptions, failure logs
NSQ	Message queue operations	Inter-service communication, task processing
Shutdown	Service shutdown events	Graceful shutdown logs, cleanup operations
Agent	Agent communication logs	Agent connectivity, data collection, status updates
Elastic	Search and indexing logs	Elasticsearch operations, data indexing
Email	Email service operations	Email notifications, SMTP communication
Scheduler	Task scheduling logs	Automated job execution, cron operations
SMS	SMS service operations	Text message notifications, SMS gateway logs
CSV Import	Data import operations	CSV file processing, bulk data imports
API Logger	API request/response logs	REST API calls, authentication, rate limiting
Events	System event tracking	User actions, system events, audit trails
Service	Core service operations	Main application services, business logic
System	System-level operations	OS interactions, resource management
Audit	Security audit logs	Access attempts, security events, compliance

Common Folder Logs:

Log Category	Description	Purpose
API Logger	API request/response logs	REST API calls, authentication, rate limiting
Error	Error tracking and debugging	Application errors, exceptions, failure logs
NSQ	Message queue operations	Inter-service communication, task processing
Service	Core service operations	Main application services, business logic
Agent	Agent communication logs	Agent connectivity, data collection, status updates
Events	System event tracking	User actions, system events, audit trails
Integration	Third-party integrations	External system connections, API integrations
Shutdown	Service shutdown events	Graceful shutdown logs, cleanup operations

Recovery Procedures

Click to expand Recovery Procedures

1. Rollback to Snapshot: If upgrade fails, restore from VM snapshot
2. Database Restore: Restore from database backup if needed
3. Service Restart: Restart all services in proper order
4. Contact Support: Share collected logs with Motadata QA/Dev team

Related Topics

• Backward Compatibility - Understand compatibility aspects with previous versions of ServiceOps.
• PostgreSQL Upgrade Guide - Detailed instructions for upgrading your PostgreSQL database.
• High Availability Upgrade Guide - Comprehensive guide for upgrading ServiceOps in High Availability environments.
• Disaster Recovery Upgrade Guide - Step-by-step instructions for upgrading Data Center-Disaster Recovery setups.