LESSON-11

Hive installation with details illustration of commands

 The class will start - refreshing the previous class with QA…. (15)

Today’s topics: PLAN

1. What is cloudera, and relation with hadoop .. (15)

2. Hive installation with command explanation.. (90)
3. questions / answers.. .. .. . . .. .. .. .. .. .. . (30)

(1) Hadoop is an ecosystem of open source components to process, analyze and storing big data. ...

Cloudera is an open source platform, or we can say “package” to work on top of hadoop distribution
with a view to utilize hadoop components in an efficient and smooth way.

(2) Hive installation

1. su - <root user name> # Switch to root user, if not in root user

2. cd /usr/lib/ # to go under “lib” under “usr” at home directory

3. sudo mkdir hive # to make a directory with name “hive” with above hierarchy

4. # Check from file explorer, if the directory is created or not. And the hive is empty

5. cd # come out from the above hierarchy

6. wget http://archive.apache.org/dist/hive/hive-0.13.0/apache-hive-0.13.0-bin.tar.gz # downld hive

7. # check the zip file is in the list or not (Red) with command “ls”

8. sudo tar xvzf apache-hive-0.13.0-bin.tar.gz # unzip the file

9. # check the unzip file is in the list or not(blue) with command “ls”

10. sudo mv apache-hive-0.13.0-bin /usr/lib/hive # move the file to desired location i.e.”/usr/lib/hive”

11. # Using file explorer go to the path and check it is moved or not

12. su – hadoopuser # switched to hadoopuser

13. nano ~/.bashrc # set environment variable

14. # append the below text to the environmental variables---

# Set HIVE_HOME # this is only comment and optional to copy

export HIVE_HOME="/usr/lib/hive/apache-hive-0.13.0-bin"
PATH=$PATH:$HIVE_HOME/bin
1
export PATH

15. source ~/.bashrc

16. su - <root user name> # Switch to root user

17. cd /usr/lib/hive/apache-hive-0.13.0-bin/bin # go to desired directory i.e. “/usr/lib/hive/a….bin”

18. sudo nano hive-config.sh # open configure file

19. # Append the below text in the file

export HADOOP_HOME=/usr/local/hadoop

# just after the below text in the config file

#Allow alternate conf dir location.
HIVE_CONF_DIR="${HIVE_CONF_DIR:-$HIVE_HOME/conf"
export HIVE_CONF_DIR=$HIVE_CONF_DIR
export HIVE_AUX_JARS_PATH=$HIVE_AUX_JARS_PATH

1. su – hadoopuser # switched to hadoopuser

2. hadoop fs -mkdir /usr/ # Make directory in hdfs
3. hadoop fs -mkdir /usr/hive/
4. hadoop fs -mkdir /usr/hive/warehouse
5. hadoop fs -chmod g+w /usr/hive/warehouse # set access permission(read write table)

---Now run hive on hadoop user---

6. hive # hit enter

7. exit; # to exit from hive or quit;

(3) Questions and answers

Time = 15+15+90+30=150 minutes with 30 min break and buffer

Useful link (hive installation and config)

http://doctuts.readthedocs.io/en/latest/hive.html

Bucketing by Sandeep patil

https://www.youtube.com/watch?v=ObuCKKd_5Aw