最终结构:
操作步骤:
注:步骤中所有命令均改为粗体黄色,注意的地方用绿色标出
磁盘分区:
首先,在机器上挂载 4 块 10GB 的磁盘(这里用 Azure 虚拟机来模拟环境,系统使用 CentOS 6.7):
[root@DanCentOS67 daniel]# fdisk -l
……
Disk /dev/sdc: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdd: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sde: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
Disk /dev/sdf: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000000
接着分别将每个磁盘分为2个分区,以 /dev/sdc 为例:
使用 fdisk 工具划出第一个分区 /dev/sdc1,大小约为 5GB(dev/sdc2 同理,Partition number 要改为 2,扇区的起止编号也不同,sdc2扇区编号是 654-1305):
[root@DanCentOS67 daniel]# fdisk /dev/sdc
WARNING: DOS-compatible mode is deprecated. It's strongly recommended to
switch off the mode (command 'c') and change display units to
sectors (command 'u').
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 653
Command (m for help): t
Selected partition 1
Hex code (type L to list codes): fd
Changed system type of partition 1 to fd (Linux raid autodetect)
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
使用 fdisk-l 查看分区结果:
[root@DanCentOS67 daniel]# fdisk -l
……
Disk /dev/sdc: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbd293e5b
Device Boot Start End Blocks Id System
/dev/sdc1 1 653 5245191 fd Linux raid autodetect
/dev/sdc2 654 1305 5237190 fd Linux raid autodetect
对剩下的3个磁盘(/dev/sdd,/dev/sde,/dev/sdf)进行同样的处理,处理后使用 fdisk -l 查看是否全部完成分区格式化:
[root@DanCentOS67 daniel]# fdisk -l
……
Disk /dev/sdc: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0xbd293e5b
Device Boot Start End Blocks Id System
/dev/sdc1 1 653 5245191 fd Linux raid autodetect
/dev/sdc2 654 1305 5237190 fd Linux raid autodetect
Disk /dev/sdd: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x13c704fe
Device Boot Start End Blocks Id System
/dev/sdd1 1 653 5245191 fd Linux raid autodetect
/dev/sdd2 654 1305 5237190 fd Linux raid autodetect
Disk /dev/sde: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x3273d383
Device Boot Start End Blocks Id System
/dev/sde1 1 653 5245191 fd Linux raid autodetect
/dev/sde2 654 1305 5237190 fd Linux raid autodetect
Disk /dev/sdf: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x6283a8a2
Device Boot Start End Blocks Id System
/dev/sdf1 1 653 5245191 fd Linux raid autodetect
/dev/sdf2 654 1305 5237190 fd Linux raid autodetect
到这里,就完成了倒数第二层结构。
RAID 5的配置:
这里采用 RAID5,在提高IO性能的同时保证数据安全。
首先加载raid5内核模块:
[root@DanCentOS67 daniel]# modprobe raid5
接着将 /dev/sdc1,/dev/sdc2,/dev/sdd1 合并为 /dev/md127:
[root@DanCentOS67 daniel]# mdadm --create /dev/md127 --level=5 --raid-devices=3 /dev/sd[cd]1 /dev/sdc2
mdadm: /dev/sdc1 appears to contain an ext2fs file system
size=5245188K mtime=Thu Jan 1 00:00:00 1970
mdadm: /dev/sdd1 appears to contain an ext2fs file system
size=5245188K mtime=Thu Jan 1 00:00:00 1970
mdadm: /dev/sdc2 appears to contain an ext2fs file system
size=5237188K mtime=Thu Jan 1 00:00:00 1970
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md127 started.
查看 /dev/md127 的状态:
[root@DanCentOS67 daniel]# mdadm --misc --detail /dev/md127
/dev/md127:
Version : 1.2
Creation Time : Thu Mar 9 12:37:43 2017
Raid Level : raid5
Array Size : 10465280 (9.98 GiB 10.72 GB)
Used Dev Size : 5232640 (4.99 GiB 5.36 GB)
Raid Devices : 3
Total Devices : 3
Persistence : Superblock is persistent
Update Time : Thu Mar 9 12:41:44 2017
State : clean
Active Devices : 3
Working Devices : 3
Failed Devices : 0
Spare Devices : 0
Layout : left-symmetric
Chunk Size : 512K
Name : DanCentOS67:127 (local to host DanCentOS67)
UUID : 1a86a744:b01bd085:ee05717d:8ba86e4a
Events : 18
Number Major Minor RaidDevice State
0 8 33 0 active sync /dev/sdc1
1 8 49 1 active sync /dev/sdd1
3 8 34 2 active sync /dev/sdc2
可以看到实际可用大小变为10GB左右,即 5GB*(3-1) = 10GB,具体原理可以参考RAID5的实现原理。
同理,将 /dev/sdd2,/dev/sde1,/dev/sde2 合并为 /dev/md126:
[root@DanCentOS67 daniel]# mdadm --create /dev/md126 --level=5 --raid-devices=3 /dev/sd[de]2 /dev/sde1
将 /dev/sdf1 和 /dev/sdf2 合并为 /dev/md125:
[root@DanCentOS67 daniel]# mdadm --create /dev/md125 --level=5 --raid-device=2 /dev/sdf1 /dev/sdf2
到这里RAID 5已经全部做完。
LVM配置:
首先,将 /dev/md127,/dev/md126,/dev/md125 创建为 PhysicalVolume(PV):
[root@DanCentOS67 daniel]# pvcreate /dev/md127 /dev/md126 /dev/md125
Physical volume "/dev/md127" successfully created
Physical volume "/dev/md126" successfully created
Physical volume "/dev/md125" successfully created
扫描 Physical Volume 的改动:
[root@DanCentOS67 daniel]# pvscan
PV /dev/md125 lvm2 [4.99 GiB]
PV /dev/md126 lvm2 [9.98 GiB]
PV /dev/md127 lvm2 [9.98 GiB]
Total: 3 [24.95 GiB] / in use: 0 [0 ] / in no VG: 3 [24.95 GiB]
查看每个 Volume 的详细信息:
[root@DanCentOS67 daniel]# pvdisplay
"/dev/md125" is a new physical volume of "4.99 GiB"
--- NEW Physical volume ---
PV Name /dev/md125
VG Name
PV Size 4.99 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID JWf4ZE-FOCZ-mQfh-3cIA-sbcg-19S6-x28jMy
"/dev/md126" is a new physical volume of "9.98 GiB"
--- NEW Physical volume ---
PV Name /dev/md126
VG Name
PV Size 9.98 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID DVxViy-miUM-6q7r-520o-SP2T-xBbX-xHj793
"/dev/md127" is a new physical volume of "9.98 GiB"
--- NEW Physical volume ---
PV Name /dev/md127
VG Name
PV Size 9.98 GiB
Allocatable NO
PE Size 0
Total PE 0
Free PE 0
Allocated PE 0
PV UUID URW7wO-pQLV-uXkP-oXEg-jOKc-fber-oenEj7
这里我们首先将 /dev/md127 和 /dev/md126 添加到 Volume Group 中:
[root@DanCentOS67 daniel]# vgcreate VolGroup1 /dev/md127 /dev/md126
Volume group "VolGroup1" successfully created
查看 Volume Group的使用情况:
[root@DanCentOS67 daniel]# vgdisplay
--- Volume group ---
VG Name VolGroup1
System ID
Format lvm2
Metadata Areas 2
Metadata Sequence No 1
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 0
Open LV 0
Max PV 0
Cur PV 2
Act PV 2
VG Size 19.95 GiB
PE Size 4.00 MiB
Total PE 5108
Alloc PE / Size 0 / 0
Free PE / Size 5108 / 19.95 GiB
VG UUID dsTlXU-D3Y5-Fqb3-wCOl-LHmj-7Qyf-hXU186
Volume Group 创建好后,我们在其上划分新的 Logical Volume:
[root@DanCentOS67 daniel]# lvcreate -l 2500 -n LogicalVol1 VolGroup1
Logical volume "LogicalVol1" created.
[root@DanCentOS67 daniel]# lvcreate -l 2608 -n LogicalVol2 VolGroup1
Logical volume "LogicalVol2" created.
前面我们 Volume Group中一共有 5108 个 PE,每个 PE(Physical Extent)的默认大小是 4.00 MiB,一共 19.95 GiB。我们为 LogicalVol1划分了 2500 个 PE,即 10000 MiB(9.77 GiB),为 LogicalVol2 划分了 2608 个 PE,即 10432MiB(10.19 Gib)。
查看 LogicalVolume 的划分情况:
[root@DanCentOS67 daniel]# lvdisplay
--- Logical volume ---
LV Path /dev/VolGroup1/LogicalVol1
LV Name LogicalVol1
VG Name VolGroup1
LV UUID 8w2ZhQ-hTSr-Sg0z-OJRv-h9aq-coZw-zeYCtI
LV Write Access read/write
LV Creation host, time DanCentOS67, 2017-03-09 13:37:56 +0000
LV Status available
# open 0
LV Size 9.77 GiB
Current LE 2500
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 4096
Block device 253:0
--- Logical volume ---
LV Path /dev/VolGroup1/LogicalVol2
LV Name LogicalVol2
VG Name VolGroup1
LV UUID JdQsXc-OMwk-A6ZB-e3vJ-rLPv-hNek-3yNVVg
LV Write Access read/write
LV Creation host, time DanCentOS67, 2017-03-09 13:38:08 +0000
LV Status available
# open 0
LV Size 10.19 GiB
Current LE 2608
Segments 2
Allocation inherit
Read ahead sectors auto
- currently set to 4096
Block device 253:1
fdisk -l 查看设备:
[root@DanCentOS67 daniel]# fdisk -l
……
Disk /dev/mapper/VolGroup1-LogicalVol1: 10.5 GB, 10485760000 bytes
255 heads, 63 sectors/track, 1274 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
Disk identifier: 0x00000000
Disk /dev/mapper/VolGroup1-LogicalVol2: 10.9 GB, 10938744832 bytes
255 heads, 63 sectors/track, 1329 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 1048576 bytes
Disk identifier: 0x00000000
使用 mkfs.ext4 工具将两个 LogicalVolume 的文件系统格式化为 ext4:
[root@DanCentOS67 daniel]# mkfs.ext4 /dev/mapper/VolGroup1-LogicalVol1
mke2fs 1.41.12 (17-May-2010)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=128 blocks, Stripe width=256 blocks
640848 inodes, 2560000 blocks
128000 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2621440000
79 block groups
32768 blocks per group, 32768 fragments per group
8112 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 39 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.
[root@DanCentOS67 daniel]# mkfs.ext4 /dev/mapper/VolGroup1-LogicalVol2
……
格式化后,将格式化得到的文件分区mount到目录中(如果要开机自动挂载可以修改 /etc/fstab):
[root@DanCentOS67 daniel]# mkdir /mnt/LV1
[root@DanCentOS67 daniel]# mkdir /mnt/LV2
[root@DanCentOS67 daniel]# mount /dev/mapper/VolGroup1-LogicalVol1 /mnt/LV1
[root@DanCentOS67 daniel]# mount /dev/mapper/VolGroup1-LogicalVol2 /mnt/LV2
到这里,LVM部分已经配置完成。
