Log Ingestion in ObserveOps
Log Ingestion is where you manage every device and source that sends logs to ObserveOps. Every device ObserveOps already monitors appears here automatically. You can also add new sources that are not part of your monitoring setup.
Sources and Ingestion Profiles
The Log Ingestion list works on two levels.
A source is the device or system sending logs, identified by its IP address or hostname. An ingestion profile is a specific type of log from that source. One source can have multiple ingestion profiles. For example, a single Linux server can send Nginx application logs, PostgreSQL database logs, and syslog — each managed as a separate ingestion profile under the same source.
Expand a source in the list to see all its ingestion profiles and the log types assigned to each.
How Sources Appear in Log Ingestion
Sources appear in the list in three ways.
Automatically from monitoring: Every device ObserveOps already monitors appears in Log Ingestion automatically. These sources may have no ingestion profiles yet. When you are ready to collect logs from them, click the three-dot menu on the source and select Add Logs.
Manually added: If a device is not in your monitoring setup, click Add Source to add it. When you add a source manually, you must configure at least one ingestion profile before saving. You cannot save an empty source.
Auto-detected from TCP/UDP traffic: When a device that is not part of your monitoring setup starts forwarding logs directly to ObserveOps on port 514 over TCP or UDP, ObserveOps detects the source IP and automatically creates an entry for it in the Log Ingestion list. You do not configure this source in advance. It appears as soon as logs arrive. Open it from the list and click Action > Edit to assign a log type and tags. Without a log type, the logs arrive but do not match any pipeline and are not parsed. The source status shows as Unknown until logs are flowing and a log type is assigned.
The Log Ingestion Screen
Go to Settings > Observability Pipeline > Log Ingestion.
The screen shows all sources. Each source is expandable to reveal its ingestion profiles. Use Expand All or Collapse All to show or hide all profiles at once. Use Add Columns to show optional columns in the view.
| Column | Description |
|---|---|
| SOURCE NAME | The name of the source or ingestion profile |
| IP | The IP address of the source |
| LOG TYPES | The log types assigned to this source for routing into the matching pipeline |
| HEALTH | The health of the log source based on when logs last arrived |
| LAST LOG RECEIVED | The timestamp of the most recent log from this source |
| ENABLE | Toggle to enable or disable receiving logs from this source |
| ACTION | Edit the ingestion profile for this source |
Add a Log Source
Click Add Source to connect a new source that is not already in your monitoring setup. A panel opens on the right with six source categories:
| Category | How Logs Arrive |
|---|---|
| Application | Via MotaAgent installed on the source server |
| Syslogs | Configured remotely via a log collection plugin that runs on the source device |
| Database | Via MotaAgent installed on the database server |
| Cloud | Via collection plugins that poll cloud provider APIs at a set interval |
| TCP/UDP | Auto-created when logs are forwarded to ObserveOps on port 514 |
| Events | Via MotaAgent (agent-based) or collection plugins (agent-less) |
To add more logs to an existing source, click the three-dot menu on that source and select Add Logs. The source IP is pre-filled and locked — you only need to configure the new ingestion profile.
- Application
- Syslogs
- Database
- Cloud
- TCP/UDP
- Events
Application logs arrive through MotaAgent installed on the source server.
| Field | Description |
|---|---|
| Ingestion Profile Name* | A unique name for this ingestion profile |
| Log Type* | The log type to assign to logs from this source — used to route logs into the matching pipeline |
| Agent* | Select the MotaAgent installed on the source server |
| Directory* | Full path to the directory where log files are stored |
| Include File Pattern* | The filename pattern to collect. Use *.log to collect all .log files, or use a specific prefix or suffix that matches your application's log file naming |
| Tag(s) | Tags to attach to all logs from this source for use in pipeline filters and Log Explorer |
| Source Time Zone | The time zone of the source server. Set this if the server's time zone differs from your ObserveOps instance. |
Multiline Log Configuration: Enable this section if your application writes a single log entry across multiple lines. Enter the file path and a regex pattern that matches the first line of each new log entry. ObserveOps uses this pattern to detect entry boundaries and stitch subsequent lines into a single log entry before processing.
MotaAgent supports Linux and Windows only. If MotaAgent is not yet deployed on the target server, install it first, then return to complete this profile.
Syslog sources are configured by running a log collection plugin directly on the remote device. The plugin sets up the device to forward its syslogs to ObserveOps automatically. You do not need to manually edit the syslog configuration on the client device.
You can configure a single device, an IP range, or a CSV list of IPs — up to 100 devices at a time.
| Field | Description |
|---|---|
| Ingestion Profile Name* | A unique name for this ingestion profile |
| Log Type* | The log type to assign to logs from this source |
| IP/Host | The IP address or hostname of the syslog source. Use the IP Range or CSV tabs to configure up to 100 devices at once. |
| Log Collection Plugin* | The plugin that runs on the remote device to configure syslog forwarding |
Profile Variables — these control how the collection plugin runs on the remote device:
| Variable | Description |
|---|---|
| action.type | Enable or disable the collection action on the remote device |
| destination.ip | The ObserveOps IP address where logs should be sent |
| destination.port | The ObserveOps port where logs should arrive |
| forwarding.rule | The protocol to use: UDP or TCP |
| Credential Profiles | The credentials the plugin uses to connect to the remote device |
| Tag(s) | Tags to attach to all logs from this source |
| Port | The port number for log collection |
After you save, ObserveOps shows a Syslog Configure Progress screen while the collection plugin runs on the target device.
When the plugin finishes, the Syslog Configure Result screen shows the outcome for each device.
| Column | Description |
|---|---|
| SOURCE NAME | The name of the ingestion profile just configured |
| IP | The IP address of the syslog source |
| STATUS | Whether the collection plugin ran successfully or failed |
| MESSAGE | The outcome description, or the error if it failed |
| CREDENTIAL PROFILE | The credentials used to connect to the device |
| TAGS | Tags attached to this source |
How syslog profiles appear in the list:
Syslog forwards logs over UDP or TCP, so it shares the ingestion profile type with TCP/UDP sources. When ObserveOps configures syslog on a source:
- If the source already has a UDP ingestion profile, the syslog log type is added to that existing profile.
- If the source has no UDP profile, syslog creates a new one and assigns the syslog log type to it.
Disabling syslog sources:
The ENABLE toggle stops ObserveOps from receiving logs from that source. It does not stop the remote device from sending them. The remote device keeps forwarding until syslog is disabled at its end.
To stop both sending and receiving: click Action on the source and select Disable Syslog. This runs the collection plugin again to remove the forwarding configuration from the remote device.
Database logs arrive through MotaAgent installed on the database server. The configuration is the same as Application sources — set the agent, directory path, file pattern, log type, and tags. Use the directory path where your database writes its log files.
Multiline Log Configuration: Enable this section if your database writes a single log entry across multiple lines. Enter the file path and a regex pattern that matches the first line of each new log entry. ObserveOps uses this pattern to detect entry boundaries and stitch subsequent lines into a single log entry before processing.
MotaAgent supports Linux and Windows only. If MotaAgent is not yet deployed on the database server, install it first before creating this profile.
Cloud sources use collection plugins that poll cloud provider APIs at a configured interval. No agent is required on the cloud source. ObserveOps fetches logs directly from the provider.
Supported providers: AWS, Azure, Cisco ACI, Office 365, VMware, and Windows.
When you select a provider, ObserveOps automatically selects the matching collection plugin for that provider.
| Field | Description |
|---|---|
| Ingestion Profile Name* | A unique name for this ingestion profile |
| Log Type* | The log type to assign to logs from this cloud source |
| Log Collection Plugin | Auto-selected based on the provider. Configure its variables to match your cloud environment. |
| Monitor / Hostname | If the cloud source is already a monitored device, select it from the monitor list. If not, enter a hostname to identify this source. |
| Collection Interval | How often ObserveOps runs the plugin to fetch new logs — for example, every 5 minutes |
| Credential Profile | The credentials the plugin uses to authenticate with the cloud provider API |
| Tag(s) | Tags to attach to all logs from this source |
After you save, the plugin runs at the configured interval and fetches new logs each time.
TCP/UDP sources are created automatically — you do not configure them from the Add Source flow. When a device forwards logs to ObserveOps on port 514 over TCP or UDP, ObserveOps detects the source IP and automatically creates a source entry and a matching ingestion profile.
After ObserveOps creates the profile, open it from the list and click Action > Edit to set the log type and tags. Without a log type, the logs arrive but do not match any pipeline and are not parsed.
ObserveOps automatically splits logs that arrive as multiple entries in a single TCP packet into separate log entries for processing.
Event sources collect Windows Events, vCenter events, and Cisco events — either through MotaAgent or via collection plugins.
Agent-based (Windows):
Select the MotaAgent installed on the Windows host. ObserveOps collects Windows Event Log entries using event IDs. You can add event sources and specify which event IDs to collect. Configure the JSON keys you want to extract, such as log level and event name.
| Field | Description |
|---|---|
| Agent* | The MotaAgent installed on the Windows host |
| Tag(s) | Tags to attach to all events collected from this source |
Agent-less:
Choose the event source type: Windows, vCenter, or Cisco. ObserveOps connects using a collection plugin and the credential profile you provide. The plugin runs at the configured collection interval to fetch events.
| Source Type | Events Collected |
|---|---|
| Windows | Windows Event Log entries using configured event IDs |
| vCenter | vCenter Server events including VM lifecycle, datastore, and host events |
| Cisco | Cisco network device event logs |
Silence Detection
Silence Detection on an ingestion profile tells ObserveOps when to mark the profile as Silent if logs stop arriving, and how to notify operators about it. Health status in the Log Ingestion list is based on this per-profile interval instead of a fixed platform-wide threshold.
The Silence Detection section is available at the bottom of the Add Log Source drawer while you create the profile, and in the Edit Profile drawer (opened from the Action three-dot icon on the profile row) while you edit it. The fields are the same in both drawers.
Health values shown in the Log Ingestion list:
| Health | What It Means |
|---|---|
| Healthy | Logs are arriving within the Expected Log Interval configured for the profile. |
| Silent | No logs have arrived within the Expected Log Interval window configured for the profile. |
| Unknown | The profile has not yet received any logs. |
Ingestion Profile Health
Set how long the profile can go without receiving logs before it is marked Silent.
| Field | Description |
|---|---|
| Expected Log Interval | Required. Enter a number and pick a unit — Minutes, Hours, or Days. The profile is marked Silent if no logs arrive within this window. Default is 1 Hour. |
| Business Hours (optional) | Restrict health evaluation to a business-hours window. Leave blank to evaluate 24×7. Default is 24*7. |
Notify on Silence
Turn Notify on Silence on to receive a notification when the profile enters the Silent state. The following fields appear when the toggle is on.
| Field | Description |
|---|---|
| Severity | Required. The alert severity assigned to the notification. |
| Notify | Required. Recipients that receive the notification. Enter @User, email address, /Handle, or #User Profile. |
Notify when Ingestion Resumes
Turn Notify when ingestion resumes on to send a recovery notification when logs start arriving again after a Silent period. This toggle is on by default and uses the same recipient list as Notify on Silence.
Re-notification
Turn Re-notification on to send repeating reminders while the profile remains Silent. The following fields appear when the toggle is on.
| Field | Description |
|---|---|
| Repeat Every | How often to resend the notification while the profile stays Silent. |
| Stop After | How long to keep resending before the reminders stop. |
Save the drawer to apply the Silence Detection settings. Select Reset to clear the drawer back to the current saved values.
Log Type Rules
Log Type Rules apply only to TCP/UDP ingestion profiles. Rules do not apply to Application, Database, Cloud, Syslog, or Events sources, where the log type is set explicitly during profile creation.
Log Type Rules automatically assign a log type to TCP/UDP sources based on their source type and vendor in monitoring. When ObserveOps already monitors a device, it knows the device's source type and vendor. These rules use that information to assign the correct log type automatically, so logs flow into the right pipeline without any manual setup.
Go to Settings > Observability Pipeline > Log Ingestion and expand the Log Type Rule section.
The table shows all active rules with the source type, vendor, and the log type that gets applied. You can add a custom rule for any new source type, vendor, or log type combination.
When a log arrives from a monitored source and a matching rule exists, ObserveOps assigns the log type before the log enters the pipeline. The log then flows into whichever pipeline matches that log type.