Top Answers to Splunk Interview Questions

1. Compare Splunk & Spark

Criteria Splunk Spark
Deployment area Collecting large amounts of machine generated data Iterative
applications & in-memory processing
Nature of tool Proprietary Open Source
Working mode Streaming mode Both streaming and batch mode
2. What is Splunk?
Splunk is Google for your machine data.It�s a software/Engine which can be used for
searching, visualizing, Monitoring, reporting, etc. of your enterprise data. Splunk
takes valuable machine data and turns it into powerful operational intelligence by
providing real? time insight to your data through charts, alerts, reports, etc.

3. What are common port numbers used by Splunk?

Below are common port numbers used by splunk,however you can change them if
required

Service Port number Used

Splunk Web Port: 8000
Splunk Management Port: 8089
Splunk Indexing Port: 9997
Splunk Index Replication Port 8080
Splunk network port 514 (Used to get data in from netwok port i.e. UDP data)
KV store 8191
Check out this Splunk Tutorial video:

Learn for free ! Subscribe to our youtube Channel.

4. What are components of splunk/splunk architecture?

Below are components of splunk:

1) Search head � provides GUI for searching

2) Indexer � indexes machine data

3) Forwarder -Forwards logs to Indexer

4) Deployment server -Manges splunk components in distributed environment

5. Which is latest splunk version in use?

Splunk 6.3

6. What is splunk indexer? what are stages of splunk indexing?

The indexer is the Splunk Enterprise component that creates and manages indexes.
The primary functions of an indexer are:

Indexing incoming data.

Searching the indexed data.
Picture
7. What is a splunk forwarder and What are types of splunk forwarder?
There are two types of splunk forwarder as below

a) universal forwarder(UF) -Splunk agent installed on non-Splunk system to gather

data locally, can�t parse or index data

b) Heavy weight forwarder(HWF) � full instance of splunk with advance

functionality.
� Generally works as a remote collector, intermediate forwarder, and possible data
filter because they parse data, they are not recommended for production systems

Go through the Splunk Course in London to get clear understanding of Splunk.

8. what are most important configuration files of splunk OR can you tell name of
few important configuration files in splunk?
props.conf

indexes.conf

inputs.conf

transforms.conf

server.conf

9. What are types of splunk licenses?

Enterprise license

Free license

Forwarder license

Beta license

Licenses for search heads (for distributed search)

Licenses for cluster members (for index replication)

10. What is Splunk app?

Splunk app is a container/directory of configurations, searches, dashboards, etc.
in Splunk.

11. Where does splunk default configuration is stored?

$splunkhome/etc/system/default
12. What features are not available in splunkfree ?
splunk free lacks these features:

authentication and scheduled searches/alerting

distributed search
forwarding in TCP/HTTP (to non-splunk)
deployment management
13. what happens if the license master is unreachable?
license slave will start a 24-hour timer, after which search will be blocked on the
license slave (though indexing continues). users Will not be able to search data in
that slave until it can reach license master again

14. what is summary index insplunk?

The Summary index is the default summary index (the index thatSplunk Enterprise
uses if you do not indicate another one).

If you plan to run a variety of summary index reports you may need to create
additional summary indexes.

Learn more about Splunk in this Splunk training in New York to get ahead in your
career!
15. What is splunk DB connect?
Splunk DB Connect is a generic SQL database plugin for Splunk that allows you to
easily integrate database information with Splunk queries and reports.

16. Can you write down a general regular expression for extracting ip address from
logs?
There are multiple ways we can extract IP address from logs. Below are few
examples.

Regular Expression for extracting IP address:

rex field=_raw "(?<ip_address>\d+\.\d+\.\d+\.\d+)"

OR

rex field=_raw "(?<ip_address>([0-9]{1,3}[\.]){3}[0-9]{1,3})"

17. What is difference between stats vs transaction command?
The transaction command is most useful in two specific cases :

Unique id (from one or more fields) alone is not sufficient to discriminate between
two transactions. This is the case
when the identifier is reused, for example web sessions identified by cookie/client
IP. In this case, time span or pauses are also used to segment the data into
transactions.
In other cases when an identifier is reused, say in DHCP logs, a particular message
may identify the beginning or end of a transaction.
When it is desirable to see the raw text of the events combined rather than
analysis on the constituent fields of the events.
In other cases, it�s usually better to use stats as the performance is higher,
especially in a distributed search environment.
Often there is a unique id and stats can be used.
18. How to troubleshoot splunk performance issues?
Answer to this question would be very wide but basically interviewer would be
looking for following keywords in interview :

Check splunkd.log for any errors

Check server performance issues i.e. cpu/memory usage,disk i/o etc
Install SOS (Splunk on splunk) app and check for warning and errors in dashboard
check number of saved searches currently running and their system resources
consumption
install Firebug, which is a firefox extension. After it�s installed and enabled,
log into splunk (using firefox), open firebug�s panels,
switch to the �Net� panel (you will have to enable it).The Net panel will show you
the HTTP requests and responses along with the time spent in each. This will give
you a lot of information quickly over which requests are hanging splunk for a few
seconds, and which are blameless. etc..

19. What are buckets? Explain splunk bucket lifecycle?

Splunk places indexed data in directories, called as �buckets�. It is physically a
directory containing events of a certain period.

A bucket moves through several stages as it ages :

Hot � Contains newly indexed data. Open for writing. One or more hot buckets for
each index.
Warm � Data rolled from hot. There are many warm buckets.
Cold � Data rolled from warm. There are many cold buckets.
Frozen � Data rolled from cold. The indexer deletes frozen data by default, but you
can also archive it. Archived data can later be thawed (Data in frozenbuckets is
not searchable)
By default, your buckets are located in:

$SPLUNK_HOME/var/lib/splunk/defaultdb/db
You should see the hot-db there, and any warm buckets you have. By default, Splunk
sets the bucket size to 10GB for 64bit systems and 750MB on 32bit systems.

Interested in learning Splunk? Click here to learn more in this Splunk Training!

20. What is the different between stats and eventstats commands?

Stats command generate summary statistics of all existing fields in your search
results and save them as values in new fields.

Eventstats is similar to the stats command, except that aggregation results are
added inline to each event and only if the

aggregation is pertinent to that event.

eventstats computes the requested statistics like stats, but aggregates them to the
original raw data.

21. Who are the biggest direct competitors to Splunk?

logstash, Loggly, Loglogic,sumo logic etc..

22. splunk licenses specify what?

how much data you can index per calendar day

23. how does splunk determine 1 day, from a licensing perspective?

midnight to midnight on the clock of the license master

24. how are forwarder licenses purchased ?

They are included with splunk, no need to purchase separately

Interested in learning Splunk? Click here to learn more in this Splunk Training in
Tornoto!

25. What is command for restarting just the splunk webserver?

splunk start splunkweb

26. What is command for restarting just the splunk daemon?

splunk start splunkd

27. What is command to check for running splunk processes on unix/Linux ?

ps aux | grep splunk

28. What is Command to enable splunk to boot start?

$SPLUNK_HOME/bin/splunk enable boot-start

29. How to disable Splunk boot start?

$SPLUNK_HOME/bin/splunk disable boot-start
Learn Complete Splunk training at Hyderabad in 26 Hrs.

30. What is sourcetype in Splunk?

Sourcetype is Splunk way of identifying data.

31. How to reset Splunk admin password?

It depends on the version of Splunk. If you are using Splunk 7.1 and above just
refer to the following steps:

First, you will have to stop your Splunk Enterprise

Now you will have to find the passwd file and rename it to passwd.bk
Then create a file named user-seed.conf in your
$SPLUNK_HOME/etc/system/local/
directory and in this file add the following command:

[user_info]

PASSWORD = NEW_PASSWORD
In the place of �NEW_PASSWORD�, just add your own new password

After that just start the Splunk Enterprise and use the new password to log in
In case you have created other users earlier and know their login details, copy and
paste their credentials from the passwd.bk file into the passwd file and restart
Splunk.

Now, if you are using the versions prior to 7.1, follow the below steps:

First, stop the Splunk Enterprise

Find the passwd file and rename it to passw.bk
Start Splunk Enterprise and log in using the default credentials of admin/changeme
Then, you will be asked to enter a new password for your admin account
In case you have created other users earlier and know their login details, copy and
paste their credentials from the passw.bk file into the passwd file and restart
Splunk.

32. How to disable splunk launch message?

Set value OFFENSIVE=Less in splunk_launch.conf

Learn more about Splunk launch message in Splunk tutorial.

33. How to clear splunk search history?

Delete the following file on Splunk server:

$splunk_home/var/log/splunk/searches.log
34. What is btool or how will you troubleshoot splunk configuration files?
splunk btool is a command line tool that helps us to troubleshoot configuration
file issues or just see what values are being used by your Splunk Enterprise
installation in existing environment

35. What is difference between splunk app and splunk add on?
Basiclly both contains preconfigured configuration and reports etc but splunk add
on do not have visual app. Splunk apps have preconfigured visual app

36. What is .conf files precedence in splunk?

File precedence is as follows :

System local directory � highest priority

App local directories

App default directories

System default directory � lowest priority

37. what is fishbucket or what is fishbucket index?

Its a directory or index at default location:
/opt/splunk/var/lib/splunk
It contains seek pointers and CRCs for the files you are indexing, so splunkd can
tell if it has read them already. We can access it through GUI by seraching for

index=_thefishbucket
Are you interested in learning Splunk course in Bangalore from Experts?

38. How do i exclude some events from being indexed by Splunk?

This can be done by defining a regex to match the necessary event(s) and send
everything else to nullqueue. Here is a basic example that will drop everything
except events that contain the string login In props.conf:

<code>[source::/var/log/foo]

# Transforms must be applied in this order

# to make sure events are dropped on the

# floor prior to making their way to the

# index processor

TRANSFORMS-set= setnull,setparsing

</code>
In transforms.conf:

[setnull] REGEX = . DEST_KEY = queue FORMAT = nullQueue

[setparsing]

REGEX = login

DEST_KEY = queue

FORMAT = indexQueue
39. How can i tell when splunk is finished indexing a log file?
By watching data from splunk�s metrics log in real time.

index="_internal" source="*metrics.log" group="per_sourcetype_thruput"

series="&lt;your_sourcetype_here&gt;" |

eval MB=kb/1024 | chart sum(MB)