Blog

Organizations around the world and throughout many industries are continuously producing massive amounts of data that need to be analyzed, tracked, and safely stored. 

As we know, Splunk enables us to produce an insight through an analysis and through dashboards of 'Big Data’ applications and workloads. Splunk provides us with the most accurate results, yet an appropriate storage system is needed for those organizations that are managing and storing terabytes to petabytes of critical data. NetApp’s E-Series storage systems were designed to handle the high performance and reliability associated to these environments.

Recently, Function1 and NetApp collaborated to produce a lab setting in which NetApp’s E-Series storage systems would be compared with commodity storage systems, and compared within its own systems. Through this collaboration, NetApp’s storage solution was verified as a better option in comparison to commodity servers with internal disk drives. 

Through our collaboration with NetApp we designed and executed an array of tests to help us best understand how Splunk Enterprise would perform on E-Series storage systems. In our simulated enterprise environment, tests were objective, and findings and results were reported accurately. In these tests, Function1 provided the resources to install and configure Splunk, while reporting on the test data for performance and stress testing.  NetApp provided the hardware, which included and is not limited to: eight indexers, one search head, eight forwarders, and one cluster/master/license master.

As with any test scenario, a concise test methodology was developed and performed. Through this methodology, test results were gathered, analyzed, and represented in tables and graphs. The final results were documented in our recent technical reference architecture paper, "NetApp E-Series for Splunk".  

An example is provided below, highlighting the initial tests between E-Series and commodity servers with internal disk drives. Other examples and all other tables and graphs can be found within the technical reference architecture paper, a link is provided at the close of this blog post.

The graph below shows the comparison between NetApp’s E-Series (blue), and the commodity servers with internal disk (orange).

Screen Shot 2015-10-07 at 5.52.39 PM.png

Graphs similar to the graph above are available within the technical reference architecture paper. Through this mirrored test, we discovered that E-Series provides similar indexing performance rate (Kb/sec) in comparison to commodity servers with internal disk, and provides an average of 69% search performance gains compared to the commodity. 

Through these tests, it was determined that E-Series provides a similar indexing performance rate and a much better average search performance, in a much more efficient configuration. 

The results of the total tests are provided below:

•   E-Series provided similar, and arguably in comparison to commodity servers with internal disks (for Indexing Rate, Kb/sec)

•   E-Series provided an average of 69% search performance gains, by providing an average of 31% static search performance, and an average of 107% streams search performance gains, in comparison to the commodity servers with internal disk drives.

•   Under a Splunk Cluster Node Failure stress test, E-Series provided an average of 30% static performance gains in comparison to the commodity. 

Through our collaboration and an objective lab test, we determined that the E-Series Storage System provides consistent performance even with failed conditions. The E-Series system, indeed, produced impressive results in comparison to the commodity. 

For more information on this paper, or to read it in its entirety, please click here.