Blog

Encryption of some fields when creating a modular input is sometimes needed when security becomes an issue. There are a few ways with the most common being the creation of a setup.xml file for the modular input schema. When using setup.xml however, you lose some of the modular input schemas built in to the python SDK. However, when not using setup.xml and still would like to use the python SDK to encrypt an input field it is still possible with a bit of work. In this writing I will go over one way of encrypting an input field using an additional field as an identifier.

To start off your modular input will seem the same as any other default modular input with a python script and the Splunk Python SDK within the bin folder of the app. The first thing within the script that you will want to include are that of the python packages going to be used. For this we are just using sys, os, splunklib.client, and splunklib.modularinput.

import splunklib.client as client
from splunklib.modularinput import *
import sys
import os

The next item that is needed is a global variable. This global varable is the input mask, which is what the password will be seen as from within the web ui and the inputs.conf after the input is initially run.

MASK = "--------"

The next segment of encrypting the input field can be divided into 3 different segments: encryption, masking, and de-encryption. The first segment is that of encrypting the input field. This is done by using Splunk’s built-in encryption method using the passwords.conf in storing the password and using the session key in order to connect to the splunk service. Using the python SDK we also then add the new password while using the username as an identifier for the password itself. If any other passwords are already stored within the instance with the same username it is deleted before adding the new input credentials.

def encrypt_password(self, username, password, session_key):
        args = {'token':session_key}
        service = client.connect(**args)
      
        try:
            for storage_password in service.storage_passwords:
                if storage_password.username == username:
                    service.storage_passwords.delete(username=storage_password.username)
                    break

            service.storage_passwords.create(password, username)

        except Exception as e:
            raise Exception, "An error occurred updating credentials. Please ensure your user account has admin_all_objects and/or list_storage_passwords capabilities. Details: %s" % str(e)

The second section was that of masking the password. This is useful in not being able to see either the encrypted or un-encrypted password from within the Splunk Web UI or within its corresponding inputs.conf. This is done by finding the corresponding input within the Splunk instance and manually updating the the input and refreshing its information via the python SDK methods.

def mask_password(self, session_key, username):
        try:
            args = {'token':session_key}
            service = client.connect(**args)
            kind, input_name = self.input_name.split("://")
            item = service.inputs.__getitem__((input_name, kind))
           
            kwargs = {
                "username": username,
                "password": self.MASK
            }
            item.update(**kwargs).refresh()
           
        except Exception as e:
            raise Exception("Error updating inputs.conf: %s" % str(e))

The final section is retrieving the encrypted password and unencrypting said password for use within the actual modular input when needed. This is done with the use of the python SDK to go through all stored paswords and finding the one with the correct username. Then returning the password after the encryption has been cleared.

def get_password(self, session_key, username):
        args = {'token':session_key}
        service = client.connect(**args)

        for storage_password in service.storage_passwords:
            if storage_password.username == username:
                return storage_password.content.clear_password

With these three methods the only thing needed now is to actually use them within the stream_events method. I’ve outlined a sample modular input stream_events that just goes through the inputs and masks/encrypts the necessary field using the correct method. It then finally logs the unencrypted value of the password to the _internal index. NOTE: In a real modular input you shouldn’t be logging this information. This is for development purposes only.

def stream_events(self, inputs, ew):
        self.input_name, self.input_items = inputs.inputs.popitem()
        session_key = self._input_definition.metadata['session_key']
        username = self.input_items['username']
        password   = self.input_items['password']
           
        try:
            if password != self.MASK:
                self.encrypt_password(username, password, session_key)
                self.mask_password(session_key, username)
               
            ew.log('INFO', 'USERNAME:' + username + ' PASSWORD:' + self.get_password(session_key, username))
           
        except Exception as e:
            ew.log('ERROR','There was an error when encrypting/masking the password: %s' %e)

With this information and layout you should now easily be able to add an encrypted field to any input.