We add a second log file in JSON format that the Splunk Agent consumes and forwards to the SPlunk Server. This decouples Splunk from our applications in the event Splunk goes down or is overwhelmed.

if !Rails.env.test? && !Rails.env.development?
 SemanticLogger.add_appender(
 file_name: ‘log/splunk_json.log’,
 formatter: :json,
 filter: -> log { log.payload.try!(:[], :action) != ‘health_check’ },
 level: :info,
 metrics: true
 )
end

It does not specifically use the Splunk formatter in the Splunk Appender. Seems to work well for us. There is no reason we cannot spin off the Splunk format code in the Splunk Appender into a separate Formatter so that instead of formatter: :json above, it could be formatter: :spunk.

Try the :json format and lets see if we still need to spin off the Splunk Formatter code into a separate formatter class.