Linux下kafka和zookeeper环境搭配

一、安装zookeeper 1.下载zookeeper-3.3.6.tar.gz 2.解压 到/usr/server/zookeeper    ---这里是你的zookeeper的安装目录 3.配置zoo.cfg cp -rf conf/zoo_sample.cfg conf/zoo.cfg vim zoo.cfg # The number of milliseconds of each tick tickTime=2000 # The number of ticks that the initial # synchronization phase can take initLimit=10 # The number of ticks that can pass between # sending a request and getting an acknowledgement syncLimit=5 # the directory where the snapshot is stored. dataDir=/data/zookeeper/data # 日志文件夹 dataLogDir=/data/zookeeper/logs # the port at which the clients will connect clientPort=2181 4.配置profile vim /etc/profile 在后面追加 # idea - zookeeper-3.3.6 config start - 2017-03-08 export ZOOKEEPER_HOME=/usr/server/zookeeper export PATH=$ZOOKEEPER_HOME/bin:$PATH export PATH # idea - zookeeper-3.3.6 config start - 2017-03-08 source /etc/profile  使生效 5.启动zookeeper bin/zkServer.sh start 如打印以下信息说明成功 ZooKeeper JMX enabled by default Using config: /usr/local/services/zookeeper/zookeeper-3.4.9/bin/../conf/zoo.cfg Starting zookeeper ... STARTED (1) 查询zookeeper状态 bin/zkServer.sh status (2)关闭zookeeper bin/zkServer.sh stop 如打印如下信息则表明成功关闭： ZooKeeper JMX enabled by default Using config: /usr/local/services/zookeeper/zookeeper-3.4.9/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED (3)重启zookeeper bin/zkServer.sh restart 如打印如下信息则表明重启成功： ZooKeeper JMX enabled by default Using config: /usr/local/services/zookeeper/zookeeper-3.4.9/bin/../conf/zoo.cfg ZooKeeper JMX enabled by default Using config: /usr/local/services/zookeeper/zookeeper-3.4.9/bin/../conf/zoo.cfg Stopping zookeeper ... STOPPED ZooKeeper JMX enabled by default Using config: /usr/local/services/zookeeper/zookeeper-3.4.9/bin/../conf/zoo.cfg Starting zookeeper ... STARTED 二、安装kafka 1.下载kafka_2.10-0.9.0.0 2.解压到/usr/server/kafka  ---这里是你的安装目录 3.启动 bin/kafka-server-start.sh config/server.properties (要挂到后台使用： bin/kafka-server-start.sh config/server.properties &) NOTE: 当Kafka broker启动时，它会在ZK上注册自己的IP和端口号，客户端就通过这个IP和端口号来连接。 在AWS这种IaaS环境下，由于java.net.InetAddress.getCanonicalHostName调用拿到的HostName是类似ip-192-168-45-160这样的只有内网才能访问到的主机名，所以默认注册到ZK上的IP是内网才能访问的内网IP。 此时就需要显示指定 host.name, .listeners参数，让注册到ZK上的IP是外网IP。 ############################# Socket Server Settings ############################# listeners=PLAINTEXT://192.168.45.160:9092 # The port the socket server listens on #port=9092 # Hostname the broker will bind to. If not set, the server will bind to all interfaces host.name=192.168.45.160 否则启动kafka时会报错： 17/03/08 16:50:42 INFO client.ClientUtils$: Fetching metadata from broker id:0,host:192.168.45.160,port:9092 with correlation id 0 for 1 topic(s) Set(shaw) 17/03/08 16:50:42 INFO producer.SyncProducer: Connected to 192.168.45.160:9092 for producing 17/03/08 16:50:42 INFO producer.SyncProducer: Disconnecting from 192.168.45.160:9092 17/03/08 16:50:44 INFO producer.SyncProducer: Connected to h0045160:9092 for producing 17/03/08 16:50:44 INFO producer.SyncProducer: Disconnecting from h0045160:9092 17/03/08 16:50:44 WARN async.DefaultEventHandler: Failed to send producer request with correlation id 2 to broker 0 with data for partitions [shaw,0] java.nio.channels.ClosedChannelException     at kafka.network.BlockingChannel.send(BlockingChannel.scala:100)     at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:73)     at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:72)