创建路径及上传数据
不设置压缩运行: bin/yarn jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.3.6.jar \ wordcount mapreduce/wordcount/input mapreduce/wordcount/output
设置压缩运行: bin/yarn jar \ share/hadoop/mapreduce/hadoop-mapreduce-examples-2.5.0-cdh5.3.6.jar \ wordcount \ -Dmapreduce.map.output.compress=true \ -Dmapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec \ mapreduce/wordcount/input mapreduce/wordcount/output
Hive中不压缩执行: select e.ename ,e.sal ,d.dname from emp e join dept d on e.deptno = d.deptno;
设置Hive压缩: set mapreduce.map.output.compress=true ; set mapreduce.map.output.compress.codec=org.apache.hadoop.io.compress.SnappyCodec ; set hive.exec.compress.intermediate = true ; 执行:select e.ename ,e.sal ,d.dname from emp e join dept d on e.deptno = d.deptno;
数据的存储方式:列式存储,节省内存 Text>RCFile>Parquet>ORCFile
