能否使用Splunk Universal Forwarder删除源服务器日志替代常规轮转？

Great question—let’s cut to the chase: Yes, you can configure Splunk UF to delete source log files after successfully processing and indexing them, but this isn’t the default behavior. It requires specific setup, and you’ll need to proceed carefully to avoid data loss. Here’s how to implement it, plus critical considerations to keep in mind:

Two Common Scenarios & Corresponding Configurations #

1. For Static/Rotated Log Files (No Active Writes) #

If you’re dealing with rotated log archives that are no longer being written to (e.g., app.log.1, app.log.2.gz), use Splunk’s batch input type. This is built explicitly for processing one-time files and automatically removing them after ingestion.

Add this snippet to your UF’s inputs.conf (typically located at $SPLUNK_HOME/etc/system/local/ or a custom app directory):

[batch:///path/to/your/rotated/logs/*.log.*]
disabled = false
index = your_target_index
sourcetype = your_app_sourcetype
move_policy = delete  # Deletes the file once fully processed
file_done_timeout = 30  # Waits 30s to confirm no new writes (adjust as needed)

• If you want an extra safety layer, replace move_policy = delete with move_policy = move to send processed files to a staging directory (e.g., /var/log/splunk_processed/) first, then manually delete them after verifying successful indexing.

2. For Active, Rotated Logs #

If you want Splunk to delete logs only after they’ve been rotated (leaving the active, currently-written log file untouched), use the monitor input with the cleanup parameter. This pairs well with tools like logrotate that rename active logs (e.g., app.log → app.log.1).

Add this to inputs.conf:

[monitor:///path/to/your/app/logs/app.log]
disabled = false
index = your_target_index
sourcetype = your_app_sourcetype
cleanup = true  # Deletes rotated versions of the log post-ingestion

• The cleanup flag only acts on rotated files that are no longer being written to. It won’t interfere with the active app.log file, so your application can keep logging without interruptions.

Critical Safety Checks #

Before enabling this in production, don’t skip these safeguards:

• Data Loss Risk: If the UF loses connectivity to the indexer before sending data, deleting the source file means that data is gone forever. Mitigate this by:
  • Enabling indexAndForward on the UF to temporarily store data locally if the indexer is unreachable.
  • Starting with move_policy = move instead of delete to stage files, and only deleting them after confirming they appear in your Splunk index.
• Log Rotation Compatibility: Ensure your app’s log rotation tool (e.g., logrotate) uses either:
  • copytruncate (copies log content then truncates the original), or
  • Renames the active log and creates a new one. This ensures rotated files are static and ready for Splunk to process.
• Test First: Always validate the configuration in a staging environment. Check that files are being deleted as expected, and confirm all data appears in your Splunk index without gaps.
• Permissions: The user running Splunk UF (usually splunk) needs write/delete permissions on the log files and directories. Missing permissions will trigger errors in $SPLUNK_HOME/var/log/splunk/splunkd.log.

Final Takeaway #

Deleting source logs after ingestion is absolutely feasible with Splunk UF, but it’s not a "set it and forget it" solution. Prioritize data safety, align the configuration with your log rotation workflow, and test rigorously before deploying to production.

内容的提问来源于stack exchange，提问作者Monizer