Blog

Are you wondering where your Assets are?  Why you can't find your identities, perhaps? Are you on Windows? With the recent release of version 3.0, there has been huge improvements in the power of the ES app, and the ease of its use. The Assets and Identities are one of the cornerstones of the ES app, and there is a major change in the way these files operate in ES 3.0 compared to ES 2.4.

Asset management provides additional information about the source and targets of events. This information can be used to correlate multiple events to a single host, identify the location of the host, determine whether the host is subject to regulatory compliance, etc.
Identity correlation provides additional information about the users involved in events. This information can be used to correlate multiple events to a single person, identify the owner of a host, determine whether an individual is subject to special attention, and more.

The issue:

On multiple occasions, we have seen a variation of lookup errors on Windows servers regarding these expanded files. For example:

The lookup table 'asset_lookup_by_cidr' does not exist. It is referenced by configuration 'DhcpSrvLog'

We have seen this error with various lookup tables and various referenced configurations being named. There may also be an error naming a python file in association, or a permissions error regarding creating lookups or identities files. Essentially, the basic problem of not being able to create the expanded files manifests as one of multiple possible errors.

In the 3.0 version of the Splunk App for Enterprise Security, the assets.csv and identities.csv files are automatically expanded into a Splunk-compatible form each time that Splunk is restarted. The expansion process reoccurs at set intervals thereafter (the default interval is one hour). At search time, the "expanded" versions of these files are now used directly as Splunk lookup tables. This change significantly improves the performance of the lookups by eliminating the reliance on an external Python script to perform the lookup. You can still add information to the assets and identities files as you did previously. The information will be automatically "expanded" into the faster Splunk-compatible form.

Splunk automatically loads the assets list at search time. Splunk does not need to be restarted for the changes to take effect. However, to immediately see your changes after updating the CSV file, you may have to wait for the lookup_expander to run, or restart Splunk, or run the lookup_expander manually. The issue is in the renaming of lookup tables on Windows environments, which is actually implemented as a delete followed by a rename, and can occur out-of-order. This results in lookup tables being deleted from disk following the setup routine. Subsequent attempts to update or access these tables, of course, fail.

The solution:

To rectify this, you can copy all the files in the SA-IdentityManagement\lookups directory from their ".csv.default" versions to their ".csv" versions if the ".csv" versions do not already exist. For every file ending in a ".csv.default" suffix, there should be a corresponding ".csv" file.
DO NOT RERUN SETUP as this will only cause the issue to recur.

Some files commonly not expanded include:

• SA-IdentityManagement\lookups\identities_expanded.csv
• SA-IdentityManagement\lookups\assets_by_str.csv
• SA-IdentityManagement\lookups\assets_by_cidr.csv

After the files are copied, you may take any of the following actions to get the identity management working:

• Restart Splunk
• Disable and then re-enable one of the stanzas in the Manager > Settings > Data Inputs > Identity Management screen
• Run the identity_manager.py script manually:
  • %SPLUNK_HOME%\bin\splunk cmd splunkd print-modinput-config identity_manager | %SPLUNK_HOME%\bin\python
  • %SPLUNK_HOME%\etc\apps\SA-IdentityManagement\bin\identity_manager.py --username=admin --password=changeme

If this solution has been useful to you, please let us know  ...and if not, let us know why!  As always, if you need any help with your Splunk deployment, please reach out to us at info@function1.com