SPLUNK

Splunk:

- Splunk is a Log Analyzing and Monitoring tool which communicates with the different log files and stores file’s data in the form of events into local Indexes

- Splunk has the capability to show data in different forms of dashboard which is useful for the application users and higher leadership

Components of Splunk:

- Universal Forwarder (UF)

- Heavy Forwarder (HF)

- Indexer (IDX)

- Search Head (SH)

- Deployment Server (DS)

- License Master (LM)

Splunk Features:

- Reporting

- Monitoring

- Log Analysis

- Alerting

- Dashboard

Splunk Enterprise:

- Splunk Enterprise collects, analyzes and acts on the value of the data generated by technology infrastructure, security and business applications

- It gives the insights to drive operational performance and business results

Splunk Cloud:

- Splunk cloud delivers all the features of Splunk Enterprise, as a cloud-based service

- The platform provides access to Splunk Enterprise Security and the Splunk App for AWS and it enables centralized visibility across cloud, hybrid and on-premises environments

Splunk Light:

- Splunk Light is a solution for small IT environments that automates log search and analysis.

- It speeds troubleshooting by gathering real-time log data from your distributed applications and infrastructure in one place to enable powerful searches, dynamic dashboards, alerts and reporting for real-time analysis all at an attractive price well within the budget

Four stages of splunk include,

– Accepts any text data as input

– Parses the data into events [rows in the database tables]

– Stores events in indexes [table in the relational DB format]

– Searches and reports

{Forwarder}

– Collects data from data source & forwards to indexer

{Indexer}

– Receives data from data source and do indexes the data

– It validates the license before indexing

{Search Head}

– Do searching the data from indexers and provides a report

Best practices:-

– Do not run splunk as super-user

– Create a user account that is used to run splunk

+ For input, splunk must be able to access data sources

+ On *Nix, non-root accounts cannot access port < 1024

+ On Windows,

– Use a domain account if splunk has to connect to other servers

– Otherwise, use a local machine account that can run services

+ Make sure the splunk account can access scripts used for inputs and alerts

– Splunk searches depend on accurate time

+ Correct event time stamping is essential

– It is imperative that your splunk indexer and production servers have standardized time configuration

+ Clock skew between hosts can affect search results

--------------------------------------------------------------------------------------------------------------------

SPLUNK SCALES:

Data processing:-

Input:- → Indexer/HF/UF

- Data from network/file/scripted input

- Data broken into 64k blocks

- Annotation of each block with host/source/source type/character encoding

Parsing:- → Indexer/HF

- Event line breaking

- Aggregation for multiline event

- Regex replacement

- Event wise host/source/source type annotation

- Time stamping events

Indexing:- → Indexer

- Parsed event data written into disk/index

Search:- → Indexer/SH

- Search on indexed data using SPL

- Knowledge object binding