环境描述: 操作系统:CentOS6.4 master-nginx均衡器:192.168.0.202 backup-nginx均衡器:192.168.0.203 后端web1:192.168.0.204 后端web2:192.168.0.205 VTP:192.168.0.200 一、安装nginx
1、安装依赖包和创建用户和组 yum install -y gcc gcc-c++ make kernel-headers glibc-headers zlib-devel openssl openssl-devel pcre-devel groupadd nginx useradd -g nginx -s /sbin/nologin nginx 2、安装nginx tar zxvf nginx-1.4.2.tar.gz cd nginx-1.4.2 ./configure --perfix=/usr/local/nginx --user=nginx --group=nginx --with-http_ssl_module --with-http_gzip_static_module --with-http_stub_status_module make && make install 二、配置主配置文件
1、vi /usr/local/nginx/conf/nginx.conf
user nginx nginx; worker_processes 8; error_log logs/error.log; pid logs/nginx.pid; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; log_format main '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '"$http_user_agent" "$http_x_forwarded_for"'; access_log logs/access.log main; #上面是访问日志格式和记录位置 sendfile on; tcp_nopush on; keepalive_timeout 65; gzip on; upstream www.test.com { server 192.168.0.203:80 weight=2; server 192.168.0.204:80 weight=2; } server { listen 80; server_name www.test.com; location / { root html; index index.html index.htm; #请求转向mysvr 定义的服务器列表 proxy_pass http://www.test.com; proxy_headers_hash_max_size 51200; #设置头部哈希表的最大值,不能小于你后端服务器设置的头部总数。 proxy_headers_hash_bucket_size 6400; #设置头部哈希表大小 proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; include proxy.conf; } } 2、vi /usr/local/nginx/conf/proxy.conf
proxy_redirect off; proxy_set_header Host $host; #后端的Web服务器可以通过X-Forwarded-For获取用户真实IP proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; client_max_body_size 10m; #允许客户端请求的最大单文件字节数 client_body_buffer_size 128k; #缓冲区代理缓冲用户端请求的最大字节数, proxy_connect_timeout 90; #nginx跟后端服务器连接超时时间(代理连接超时) proxy_read_timeout 90; #连接成功后,后端服务器响应时间(代理接收超时) proxy_buffer_size 4k; #设置代理服务器(nginx)保存用户头信息的缓冲区大小 proxy_buffers 6 32k; #proxy_buffers缓冲区,网页平均在32k以下的话,这样设置 proxy_busy_buffers_size 64k; #高负荷下缓冲大小(proxy_buffers*2) proxy_temp_file_write_size 64k; #设定缓存文件夹大小,大于这个值,将从upstream服务器传 三、安装主从keepalived (配置基本一样,主配置文件稍作修改) 1、安装依赖包: yum install -y pcre pcre-devel openssl-devel lftp libnl-devel popt* 2、wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz tar zxvf keepalived-1.2.7.tar.gz cd keepalived-1.2.7 ./configure /usr/local/keepalived make && make install 3、keepalived配置成系统服务 cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/ cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/ mkdir /etc/keepalived/ cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/ cp /usr/local/keepalived/sbin/keepalived /usr/sbin/ 4、修改keepalived主配置
! Configuration File for keepalived global_defs { notification_email { test@sina.com #故障联系人 } notification_email_from admin@test.com #故障发送人 smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_MASTER #BACKUP上修改为LVS_BACKUP } vrrp_script chk_http_port { script "/usr/local/nginx/nginx_local.sh" #检测nginx脚本路径 interval 2 #监控时间/秒 } vrrp_instance VI_1 { state MASTER #BACKUP上修改为BACKUP interface eth0 virtual_router_id 51 priority 100 #BACKUP上修改为90 advert_int 1 authentication { auth_type PASS auth_pass 1111 } track_script { #使用上面检测脚本 chk_http_port } virtual_ipaddress { #VTP 192.168.0.203 192.168.0.204 } } 5、编写nginx_local.sh脚本
#!/bin/bash NGINX1=`netstat -tupln |grep 80 |wc -l` if [ $NGINX1 -eq 0 ] then /usr/local/nginx/sbin/nginx NGINX2=`netstat -tupln |grep 80 |wc -l` if [ $NGINX2 -eq 0 ] then service keepalived stop else echo "Nginx again running!!!" fi fi 语句说明:当NGINX1等于0,也就是nginx关闭时,尝试重启nginx服务,如果启动成功,NGINX1和NGINX2变量是一样作用,则显示Nginx again running!!!,如果启动失败,就关闭keepalived。 6、重新启动:service keepalived restart /usr/local/nginx/sbin/nginx -s reload
四、其他说明 实验中遇到一个问题,当keepalived或服务器没有DOWN时,主从服务器不会切换,所以就使用vrrp_script功能,写了个简单的检测脚本,nginx服务启动不起来时,就关闭keepalived,从而确保能主从切换。 1.常用命令: ipvsadm -ln #显示集群中服务器ip信息 ip add #显示VTP绑定在哪个服务器上
/usr/local/nginx/sbin/nginx #启动nginx /sbin/nginx -s stop #快速终止web服务 /sbin/nginx -s quit #有安排的结束web服务 /sbin/nginx -s reload #重新加载相关配置 echo '/usr/local/nginx/sbin/nginx' >> /etc/rc.local #设置开机启动
2.tail -f /var/log/messger(从日志中可知,主机出现故障后,备机立刻检测到,此时备机变为MASTER角色,并且接管了主机的虚拟IP资源,最后将虚拟IP绑定在eth0设备上) 3.当主LVS恢复后,会切换成主动服务器,如果Keepalived监控模块检测web故障恢复后,恢复的wen主机又将此节点加入集群系统中。