Blog

Image courtesy of xe-pOr-ex/ FreeDigitalPhotos.net

The following post is for Splunk administrators that are already somewhat familiar with the Splunk Deployment Server, and the deployment of configuration app packages. The scenario: As a sysadmin, no matter how much effort goes into planning and organization, there is the possibility that one day you will no longer be able to easily group your servers simply by their hostnames or IP addresses. It may be the result of an emergency change ticket that forced you to repurpose machines, or perhaps because of a group of legacy hosts that just can’t be decommissioned. As years pass, IP addresses free up, and in turn get re-allocated. Naming schemes get improved, while documentation gets written to map old names to new. In any case, sometimes the need to keep organized is sacrificed, or overlooked entirely.

In the event that a role in your serverclass.conf can not be universally defined by DNS name, IP address, or OS platform, there is another way. Enter the clientName. The clientName value can be added to the deploymentclient.conf file of any instance of Splunk. While it was intended to provide an alias for a host running a forwarder or indexer, it can essentially be set to any string you wish to use as an organizational hierarchy. Simply add the clientName value in any deployment client’s SPLUNK_HOME/etc/system/local/deploymentclient.conf, and upon restart, the value will seen by your deployment server when that client calls home.

For example, if your company has multiple departments that use Splunk, each with multiple OS platforms, and each group of platforms performing multiple roles, your clientName could be defined as: clientName = department_platform_role. To use a more specific example, the following may be more applicable: clientName = finance_linux_forwarder. Now, in your deployment server’s serverclass.conf, you can whitelist based on this value with the sample stanzas below:

#Packages for all finance dept hosts
[serverClass:finance]
stateOnClient = enabled
restartSplunkd = true
whitelist.0 = finance*
[serverClass:finance:app:finance_all_deploymentclient]

#Packages for all finance dept linux hosts
[serverClass:finance_linux]
stateOnClient = enabled
restartSplunkd = true
whitelist.0 = finance_linux*
[serverClass: finance_linux:app:finance_linux_forwarder_outputs]

#Special packages for all finance dept linux forwarders
[serverClass:finance_linux_forwarder]
stateOnClient = enabled
restartSplunkd = true
whitelist.0 = finance_linux_forwarder*
[serverClass:finance_linux_forwarder:app:finance_linux_specialapp]

This organizational hierarchy would allow the use of wildcards in the above serverclass.conf stanzas to deploy finance* app packages to all finance department hosts, finance_linux* app packages to all finance department hosts running linux, and so on, giving you a clean slate on which to organize. Of course, since every IT system is structured in a slightly different way, you may find the filterable values in your clientName string to be sorted differently, but this concept remains useful when all else fails.

As a parting thought: When setting your clientName, you may find it useful, while slightly redundant, to tag the end of your clientName string with a unique identifier (such as the actual hostname i.e. finance_linux_forwarder_hostalphabravo). This will allow you to filter for a single deployment client without having to divert from using clientName, and may also help in quicker identification of a client in a troubleshooting scenario.