ISCSI网络存储

ISCSI： ISCSI即Internet Small Computer System Interface，Internet小型计算机系统接口，又称为IP-SAN，是一种基于因特网即SCSI-3协议下的存储技术。也是一个在TCP/IP上进行数据块传输的标准。ISCSI可以实现在IP网络上SCSI协议，使其能够在诸如高速千兆以太网上进行快速的数据存取备份操作。磁盘接口：IDESATASC...

IT运维民工

522人浏览 · 2019-11-19 23:09:55

IT运维民工 · 2019-11-19 23:09:55 发布

ISCSI：

ISCSI即Internet Small Computer System Interface，Internet小型计算机系统接口，又称为IP-SAN，是一种基于因特网即SCSI-3协议下的存储技术。也是一个在TCP/IP上进行数据块传输的标准。ISCSI可以实现在IP网络上SCSI协议，使其能够在诸如高速千兆以太网上进行快速的数据存取备份操作。

磁盘接口：

IDE
SATA
SCSI（10000r/m）
FC（光纤）

存储网络：

DAS：直接连接存储，管理成本低，直接依附在服务器上，存储共享受到限制，占用CPU。
NAS：网络连接区域（samba，nfs），通过局域网在多个文件服务器达到共享，集中管理数据，可以跨平台使用，只能用于局域网。
SAN：存储局域网络，通过光纤网络连接服务器和存储设备，基于SCSI，ip实现存储共享，服务器和存储设备是分开的，处于独立的状态，可以同时服务多个服务器，但是实施复杂，管理成本高。

优点：

接口应用广泛。
多任务。
带宽大。
CPU占用率低。
热插拔。

组成：

sscsi target：标签，用来描述块设备。
网络设备：交换机，路由器，tcp/ip网络。
iscsi initiator：iscsi发起端。
硬件：ISCSI HDA卡。

工作流程：

ISCSI系统由SCSI适配器发送一个SCSI命令。
命令封装到TCP/IP包中并发送以太网络。
接收方从TCP/IP包中抽取SCSI命令并执行相关操作。
把返回的SCSI命令和数据封装储到TCP/IP包中，将他们发回到发送方。
系统提取出数据或命令，并把他们传回SCSI子系统。
通过ISCSI实现存储共享，创建IQN用于识别启动器和目标，然后建立tpg共享存储组，设置相应的访问控制权限，然后在共享存储组添加lun逻辑单元（存储设备），然后建立实现共享portals入口（ip，port）。

概念：

TPG：共享存储组，某个特定ISCSI目标要侦听的接口IP地址和TCP端口的集合。可以将目标配置添加到TPG以协调多个LUN设置。
ACL：通过权限控制列表，一种使用节点IQN（通常是启动器名称）来验证启动器的访问权限的访问控制。
IQN：ISCSI限定名称，全球唯一名称，用于以强制命名格式来识别启动器和目标。
LUN：逻辑单元号，带有编号的块设备，连接到目标且通过目标来使用，可以有一个或多个LUN连接到单个目标，但通过一个目标提供一个LUN。
portals入口：目标或启动器上用于建立的IP地址和端口，默认3260端口。

部署ISCSI：

Host	网络信息
ISCSI	ens33：192.168.43.23
Client1	ens33：192.168.43.176

ISCSI：

[root@localhost ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x7d5004d1.

Command (m for help): n
Partition type:
    p   primary (0 primary, 0 extended, 4 free)
    e   extended
Select (default p): p
Partition number (1-4, default 1): 1
First sector (2048-20971519, default 2048): 
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-20971519, default 20971519): +2048M
Partition 1 of type Linux and of size 2 GiB is set

Command (m for help): n
Partition type:
    p   primary (1 primary, 0 extended, 3 free)
    e   extended
Select (default p): p
Partition number (2-4, default 2): 2
First sector (4196352-20971519, default 4196352): 
Using default value 4196352
Last sector, +sectors or +size{K,M,G} (4196352-20971519, default 20971519): +2048M
Partition 2 of type Linux and of size 2 GiB is set

Command (m for help): n
Partition type:
    p   primary (2 primary, 0 extended, 2 free)
    e   extended
Select (default p): p
Partition number (3,4, default 3): 3
First sector (8390656-20971519, default 8390656): 
Using default value 8390656
Last sector, +sectors or +size{K,M,G} (8390656-20971519, default 20971519): +2048M
Partition 3 of type Linux and of size 2 GiB is set

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x7d5004d1

    Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048     4196351     2097152   83  Linux
/dev/sdb2         4196352     8390655     2097152   83  Linux
/dev/sdb3         8390656    12584959     2097152   83  Linux

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@localhost ~]# rpm -qa | grep target
selinux-policy-targeted-3.13.1-102.el7.noarch
targetcli-2.1.fb41-3.el7.noarch
[root@localhost ~]# systemctl start target
[root@localhost ~]# systemctl enable target
Created symlink from /etc/systemd/system/multi-user.target.wants/target.service to /usr/lib/systemd/system/target.service.
[root@localhost ~]# systemctl stop firewalld
[root@localhost ~]# setenforce 0
[root@localhost ~]# targetcli 
Warning: Could not load preferences file /root/.targetcli/prefs.bin.
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 0]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk .....................................................................    ........................... [Storage Objects: 0]
o- iscsi ............................................................................................................ [Targets: 0]
o- loopback ......................................................................................................... [Targets: 0]
#backstores：指定添加的存储设备类型。
#block：块设备，磁盘分区，逻辑卷。
#fileio：文件的数据，相当于虚拟磁盘。
#pscsi：真实的物理scsi。
#ramdisk：内存盘。
/> backstores/block create disk /dev/sdb1 
Created block storage object disk using /dev/sdb1.
/> iscsi/ create iqn.2019-11.com.server.www:disk
Created target iqn.2019-11.com.server.www:disk.
Created TPG 1.
Global pref auto_add_default_portal=true
Created default portal listening on all IPs (0.0.0.0), port 3260.
/> iscsi/iqn.2019-11.com.server.www:disk/tpg1/acls create iqn.2019-11.com.client.www:client1
Created Node ACL for iqn.2019-11.com.client.www:client1
/> iscsi/iqn.2019-11.com.server.www:disk/tpg1/luns create /backstores/block/disk 
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2019-11.com.client.www:client1
/> iscsi/iqn.2019-11.com.server.www:disk/tpg1/portals/ delete 0.0.0.0 3260
Deleted network portal 0.0.0.0:3260
/> iscsi/iqn.2019-11.com.server.www:disk/tpg1/portals/ create 192.168.43.23 3260
Using default IP port 3260
Created network portal 192.168.43.23:3260.
/> ls
o- / ......................................................................................................................... [...]
o- backstores .............................................................................................................. [...]
| o- block .................................................................................................. [Storage Objects: 1]
| | o- disk ........................................................................... [/dev/sdb1 (10.0GiB) write-thru activated]
| o- fileio ................................................................................................. [Storage Objects: 0]
| o- pscsi .................................................................................................. [Storage Objects: 0]
| o- ramdisk ................................................................................................ [Storage Objects: 0]
o- iscsi ............................................................................................................ [Targets: 1]
| o- iqn.2019-11.com.server.www:disk ................................................................................... [TPGs: 1]
|   o- tpg1 ............................................................................................... [no-gen-acls, no-auth]
|     o- acls .......................................................................................................... [ACLs: 1]
|     | o- iqn.2019-11.com.client.www:client1 ................................................................... [Mapped LUNs: 1]
|     |   o- mapped_lun0 .................................................................................. [lun0 block/disk (rw)]
|     o- luns .......................................................................................................... [LUNs: 1]
|     | o- lun0 ......................................................................................... [block/disk (/dev/sdb1)]
|     o- portals .................................................................................................... [Portals: 1]
|       o- 192.168.43.23:3260 ............................................................................................... [OK]
o- loopback ......................................................................................................... [Targets: 0]
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

IQN格式：

iqn.YYYY-MM.com.reversed.domain[:optional_string]

iqn：表示此名称使用域为标识符。
YYYY-MM：表示拥有域名的年份份。
com.reversed.domain：表示拥有此iSCSI组织的逆向域名。
optional_string：以冒号为前缀的可选字符串，全球唯一，由域所有者分配。

Client1：

[root@localhost ~]# rpm -qa | grep iscsi
iscsi-initiator-utils-6.2.0.873-35.el7.x86_64
libiscsi-1.9.0-7.el7.x86_64
iscsi-initiator-utils-iscsiuio-6.2.0.873-35.el7.x86_64
[root@localhost ~]# vim /etc/iscsi/initiatorname.iscsi 
修改：
InitiatorName=iqn.2019-11.com.client.www:client1
[root@localhost ~]# systemctl restart iscsid
[root@localhost ~]# systemctl enable iscsid
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
[root@localhost ~]# iscsiadm -m discovery -p 192.168.43.23:3260 -t sendtargets
192.168.43.23:3260,1 iqn.2019-11.com.server.www:disk
#-m：模式。
#discovery：查找。
#-p：IP和端口。
#-t：类型。
#sendtargets：将在服务器寻找可用的
[root@localhost ~]# iscsiadm -m node -T iqn.2019-11.com.server.www:disk -l
Logging in to [iface: default, target: iqn.2019-11.com.server.www:disk, portal: 192.168.43.23,3260] (multiple)
Login to [iface: default, target: iqn.2019-11.com.server.www:disk, portal: 192.168.43.23,3260] successful.
#node：节点。
#-T：指定连接的目标。
#-l：登入。
#-u：登出。
#--op：操作 ，update：更新，create：创建，delete：删除。
#-n：指定名字，设置开机自启连接需要的使用。
#-v automatic：自动挂载。
[root@localhost ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000aedd1

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200    41943039    19921920   8e  Linux LVM

Disk /dev/mapper/cl-root: 18.2 GB, 18249416704 bytes, 35643392 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/cl-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes

设置开机自动连接：

[root@localhost ~]# iscsiadm -m node -T iqn.2019-11.com.server.www:disk --op update -n node.startup -v automatic

格式化和挂载：

[root@localhost ~]# mkdir /client1
[root@localhost ~]# mkfs.ext4 /dev/sdb 
mke2fs 1.42.9 (28-Dec-2013)
/dev/sdb is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=1024 blocks
131072 inodes, 524288 blocks
26214 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=536870912
16 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
    32768, 98304, 163840, 229376, 294912

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (16384 blocks): done
Writing superblocks and filesystem accounting information: done 
[root@localhost ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/cl-root   17G   15G  2.3G  87% /
devtmpfs             1.9G     0  1.9G   0% /dev
tmpfs                1.9G   84K  1.9G   1% /dev/shm
tmpfs                1.9G  9.0M  1.9G   1% /run
tmpfs                1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1           1014M  173M  842M  18% /boot
tmpfs                378M   16K  378M   1% /run/user/42
tmpfs                378M     0  378M   0% /run/user/0
/dev/sdb             2.0G  6.0M  1.8G   1% /client1

LVS：

Host	网络信息
ISCSI	ens33：192.168.43.23
Client1	ens33：192.168.43.176
Client2	ens33：192.168.43.241

ISCSI：

[root@localhost ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): t
Partition number (1-3, default 3): 2
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): t
Partition number (1-3, default 3): 3
Hex code (type L to list all codes): 8e
Changed type of partition 'Linux' to 'Linux LVM'

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x7d5004d1

    Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048     4196351     2097152   83  Linux
/dev/sdb2         4196352     8390655     2097152   8e  Linux LVM
/dev/sdb3         8390656    12584959     2097152   8e  Linux LVM

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@localhost ~]# pvcreate /dev/sdb2 /dev/sdb3
Physical volume "/dev/sdb2" successfully created.
Physical volume "/dev/sdb3" successfully created.
[root@localhost ~]# vgcreate vg /dev/sdb2 /dev/sdb3 -s 16M
Volume group "vg" successfully created
[root@localhost ~]# lvcreate -L 1G -n lv vg
Logical volume "lv" created.
[root@localhost ~]# pvs
PV         VG Fmt  Attr PSize  PFree   
/dev/sda2  cl lvm2 a--  29.00g       0 
/dev/sdb2  vg lvm2 a--   1.98g 1008.00m
/dev/sdb3  vg lvm2 a--   1.98g    1.98g
[root@localhost ~]# lvs
LV   VG Attr       LSize  Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
root cl -wi-ao---- 26.00g                                                    
swap cl -wi-ao----  3.00g                                                    
lv   vg -wi-ao----  1.00g                                                    
[root@localhost ~]# vgs
VG #PV #LV #SN Attr   VSize  VFree
cl   1   2   0 wz--n- 29.00g    0 
vg   2   1   0 wz--n-  3.97g 2.97g
[root@localhost ~]# targetcli
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> backstores/block/ create lvm /dev/mapper/vg-lv 
Created block storage object lvm using /dev/mapper/vg-lv.
/> iscsi/ create iqn.2019-11.com.server.www:lvm
Created target iqn.2019-11.com.server.www:lvm.
Created TPG 1.
Default portal not created, TPGs within a target cannot share ip:port.
/> iscsi/iqn.2019-11.com.server.www:lvm/tpg1/acls create iqn.2019-11.com.client.www:client1
Created Node ACL for iqn.2019-11.com.client.www:client1
/> iscsi/iqn.2019-11.com.server.www:lvm/tpg1/acls create iqn.2019-11.com.client.www:client2
Created Node ACL for iqn.2019-11.com.client.www:client2
/> iscsi/iqn.2019-11.com.server.www:lvm/tpg1/luns create /backstores/block/lvm 
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2019-11.com.client.www:client2
Created LUN 0->0 mapping in node ACL iqn.2019-11.com.client.www:client1
/> iscsi/iqn.2019-11.com.server.www:lvm/tpg1/portals create 192.168.43.23 3260
Using default IP port 3260
Created network portal 192.168.43.23:3260.
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

Client1：

[root@localhost ~]# iscsiadm -m discovery -p 192.168.43.23:3260 -t sendtargets
192.168.43.23:3260,1 iqn.2019-11.com.server.www:disk
192.168.43.23:3260,1 iqn.2019-11.com.server.www:lvm
[root@localhost ~]# iscsiadm -m node -T iqn.2019-11.com.server.www:lvm -l
Logging in to [iface: default, target: iqn.2019-11.com.server.www:lvm, portal: 192.168.43.23,3260] (multiple)
Login to [iface: default, target: iqn.2019-11.com.server.www:lvm, portal: 192.168.43.23,3260] successful.
[root@localhost ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000aedd1

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200    41943039    19921920   8e  Linux LVM

Disk /dev/mapper/cl-root: 18.2 GB, 18249416704 bytes, 35643392 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/cl-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes


Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes

[root@localhost ~]# mkdir /lvm
[root@localhost ~]# mkfs.ext4 /dev/sdc
mke2fs 1.42.9 (28-Dec-2013)
/dev/sdc is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=1024 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

[root@localhost ~]# mount /dev/sdc /lvm
[root@localhost ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/cl-root   17G   15G  2.3G  87% /
devtmpfs             1.9G     0  1.9G   0% /dev
tmpfs                1.9G   84K  1.9G   1% /dev/shm
tmpfs                1.9G  9.0M  1.9G   1% /run
tmpfs                1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1           1014M  173M  842M  18% /boot
tmpfs                378M   16K  378M   1% /run/user/42
tmpfs                378M     0  378M   0% /run/user/0
/dev/sdb             2.0G  6.0M  1.8G   1% /client1
/dev/sdc             976M  2.6M  907M   1% /lvm

Client2：

[root@localhost ~]# iptables -F
[root@localhost ~]# iptables-save 
[root@localhost ~]# setenforce 0
[root@localhost ~]# rpm -qa | grep iscsi
iscsi-initiator-utils-6.2.0.873-35.el7.x86_64
libiscsi-1.9.0-7.el7.x86_64
iscsi-initiator-utils-iscsiuio-6.2.0.873-35.el7.x86_64
[root@localhost ~]# vim /etc/iscsi/initiatorname.iscsi 
修改：
InitiatorName=iqn.2019-11.com.client.www:client2
[root@localhost ~]# systemctl restart iscsid
[root@localhost ~]# systemctl enable iscsid
Created symlink from /etc/systemd/system/multi-user.target.wants/iscsid.service to /usr/lib/systemd/system/iscsid.service.
[root@localhost ~]# iscsiadm -m discovery -p 192.168.43.23:3260 -t sendtargets
192.168.43.23:3260,1 iqn.2019-11.com.server.www:disk
192.168.43.23:3260,1 iqn.2019-11.com.server.www:lvm
[root@localhost ~]# iscsiadm -m node -T iqn.2019-11.com.server.www:lvm -l
Logging in to [iface: default, target: iqn.2019-11.com.server.www:lvm, portal: 192.168.43.23,3260] (multiple)
Login to [iface: default, target: iqn.2019-11.com.server.www:lvm, portal: 192.168.43.23,3260] successful.
[root@localhost ~]# mkdir /lvm
[root@localhost ~]# fdisk -l

磁盘 /dev/sda：32.2 GB, 32212254720 字节，62914560 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理)：512 字节 / 512 字节
I/O 大小(最小/最佳)：512 字节 / 512 字节
磁盘标签类型：dos
磁盘标识符：0x0009bc17

   设备 Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200    62914559    30407680   8e  Linux LVM

磁盘 /dev/mapper/cl-root：27.9 GB, 27913093120 字节，54517760 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理)：512 字节 / 512 字节
I/O 大小(最小/最佳)：512 字节 / 512 字节


磁盘 /dev/mapper/cl-swap：3221 MB, 3221225472 字节，6291456 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理)：512 字节 / 512 字节
I/O 大小(最小/最佳)：512 字节 / 512 字节


磁盘 /dev/sdb：1073 MB, 1073741824 字节，2097152 个扇区
Units = 扇区 of 1 * 512 = 512 bytes
扇区大小(逻辑/物理)：512 字节 / 512 字节
I/O 大小(最小/最佳)：512 字节 / 4194304 字节

[root@localhost ~]# mkfs.ext4 /dev/sdb
mke2fs 1.42.9 (28-Dec-2013)
/dev/sdb is entire device, not just one partition!
无论如何也要继续? (y,n) y
文件系统标签=
OS type: Linux
块大小=4096 (log=2)
分块大小=4096 (log=2)
Stride=0 blocks, Stripe width=1024 blocks
65536 inodes, 262144 blocks
13107 blocks (5.00%) reserved for the super user
第一个数据块=0
Maximum filesystem blocks=268435456
8 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376

Allocating group tables: 完成                            
正在写入inode表: 完成                            
Creating journal (8192 blocks): 完成
Writing superblocks and filesystem accounting information: 完成

[root@localhost ~]# mount /dev/sdb /lvm/
[root@localhost ~]# df -h
文件系统             容量  已用  可用 已用% 挂载点
/dev/mapper/cl-root   26G   14G   13G   54% /
devtmpfs             1.9G     0  1.9G    0% /dev
tmpfs                1.9G  128K  1.9G    1% /dev/shm
tmpfs                1.9G  9.0M  1.9G    1% /run
tmpfs                1.9G     0  1.9G    0% /sys/fs/cgroup
/dev/sda1           1014M  173M  842M   18% /boot
tmpfs                378M   20K  378M    1% /run/user/0
/dev/sr0             4.1G  4.1G     0  100% /run/media/root/CentOS 7 x86_64
/dev/sdb             976M  2.6M  907M    1% /lvm

RAID10：

Host	网络信息
ISCSI	ens33：192.168.43.23
Client1	ens33：192.168.43.176

ISCSI：

[root@localhost ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): n
Partition type:
    p   primary (3 primary, 0 extended, 1 free)
    e   extended
Select (default e): e
Selected partition 4
First sector (12584960-20971519, default 12584960): 
Using default value 12584960
Last sector, +sectors or +size{K,M,G} (12584960-20971519, default 20971519): 
Using default value 20971519
Partition 4 of type Extended and of size 4 GiB is set

Command (m for help): n
All primary partitions are in use
Adding logical partition 5
First sector (12587008-20971519, default 12587008): 
Using default value 12587008
Last sector, +sectors or +size{K,M,G} (12587008-20971519, default 20971519): +50M
Partition 5 of type Linux and of size 50 MiB is set

Command (m for help): N
All primary partitions are in use
Adding logical partition 6
First sector (12691456-20971519, default 12691456): 
Using default value 12691456
Last sector, +sectors or +size{K,M,G} (12691456-20971519, default 20971519): +50M
Partition 6 of type Linux and of size 50 MiB is set

Command (m for help): n
All primary partitions are in use
Adding logical partition 7
First sector (12795904-20971519, default 12795904): 
Using default value 12795904
Last sector, +sectors or +size{K,M,G} (12795904-20971519, default 20971519): +50M
Partition 7 of type Linux and of size 50 MiB is set

Command (m for help): n
All primary partitions are in use
Adding logical partition 8
First sector (12900352-20971519, default 12900352): 
Using default value 12900352
Last sector, +sectors or +size{K,M,G} (12900352-20971519, default 20971519): +50M
Partition 8 of type Linux and of size 50 MiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@localhost ~]# fdisk /dev/sdb
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): t
Partition number (1-8, default 8): 5
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): t
Partition number (1-8, default 8): 6
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): t
Partition number (1-8, default 8): 7
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): t
Partition number (1-8, default 8): 8
Hex code (type L to list all codes): fd
Changed type of partition 'Linux' to 'Linux raid autodetect'

Command (m for help): p

Disk /dev/sdb: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x7d5004d1

    Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048     4196351     2097152   83  Linux
/dev/sdb2         4196352     8390655     2097152   8e  Linux LVM
/dev/sdb3         8390656    12584959     2097152   8e  Linux LVM
/dev/sdb4        12584960    20971519     4193280    5  Extended
/dev/sdb5        12587008    12689407       51200   fd  Linux raid autodetect
/dev/sdb6        12691456    12793855       51200   fd  Linux raid autodetect
/dev/sdb7        12795904    12898303       51200   fd  Linux raid autodetect
/dev/sdb8        12900352    13002751       51200   fd  Linux raid autodetect

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@localhost ~]# mdadm -C /dev/md1 -ayes -l1 -n2 /dev/sdb[5,6]
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.
[root@localhost ~]# mdadm -C /dev/md2 -ayes -l1 -n2 /dev/sdb[7,8]
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md2 started.
[root@localhost ~]# mdadm -C /dev/md10 -ayes -l0 -n2 /dev/md1 /dev/md2
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md10 started.
[root@localhost ~]# targetcli
targetcli shell version 2.1.fb41
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> backstores/block/ create raid10 /dev/md10
Created block storage object raid10 using /dev/md10.
/> iscsi/ create iqn.2019-11.com.server.www:raid10 
Created target iqn.2019-11.com.server.www:raid10.
Created TPG 1.
Default portal not created, TPGs within a target cannot share ip:port.
/> iscsi/iqn.2019-11.com.server.www:raid10/tpg1/acls  create iqn.2019-11.com.client.www:client1
Created Node ACL for iqn.2019-11.com.client.www:client1
/> iscsi/iqn.2019-11.com.server.www:raid10/tpg1/luns create /backstores/block/raid10
Created LUN 0.
Created LUN 0->0 mapping in node ACL iqn.2019-11.com.client.www:client1
/> iscsi/iqn.2019-11.com.server.www:raid10/tpg1/portals create 192.168.43.23 3260
Using default IP port 3260
Created network portal 192.168.43.23:3260.
/> saveconfig
Last 10 configs saved in /etc/target/backup.
Configuration saved to /etc/target/saveconfig.json

Client1：

[root@localhost ~]# iscsiadm -m discovery -p 192.168.43.23:3260 -t sendtargets
192.168.43.23:3260,1 iqn.2019-11.com.server.www:lvm
192.168.43.23:3260,1 iqn.2019-11.com.server.www:disk
192.168.43.23:3260,1 iqn.2019-11.com.server.www:raid10
[root@localhost ~]# iscsiadm -m node -T iqn.2019-11.com.server.www:raid10 -l
Logging in to [iface: default, target: iqn.2019-11.com.server.www:raid10, portal: 192.168.43.23,3260] (multiple)
Login to [iface: default, target: iqn.2019-11.com.server.www:raid10, portal: 192.168.43.23,3260] successful.
[root@localhost ~]# fdisk -l

Disk /dev/sda: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000aedd1

    Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200    41943039    19921920   8e  Linux LVM

Disk /dev/mapper/cl-root: 18.2 GB, 18249416704 bytes, 35643392 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/cl-swap: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 2147 MB, 2147483648 bytes, 4194304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes


Disk /dev/sdc: 1073 MB, 1073741824 bytes, 2097152 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 4194304 bytes


Disk /dev/sdd: 101 MB, 101711872 bytes, 198656 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 524288 bytes / 524288 bytes
[root@localhost ~]# mkdir /raid10
[root@localhost ~]# mkfs.ext4 /dev/sdd
mke2fs 1.42.9 (28-Dec-2013)
/dev/sdd is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
Stride=512 blocks, Stripe width=512 blocks
24856 inodes, 99328 blocks
4966 blocks (5.00%) reserved for the super user
First data block=1
Maximum filesystem blocks=33685504
13 block groups
8192 blocks per group, 8192 fragments per group
1912 inodes per group
Superblock backups stored on blocks: 
	8193, 24577, 40961, 57345, 73729

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (4096 blocks): done
Writing superblocks and filesystem accounting information: done 

[root@localhost ~]# mount /dev/sdd /raid10/
[root@localhost ~]# df -h
Filesystem           Size  Used Avail Use% Mounted on
/dev/mapper/cl-root   17G   15G  2.3G  87% /
devtmpfs             1.9G     0  1.9G   0% /dev
tmpfs                1.9G   84K  1.9G   1% /dev/shm
tmpfs                1.9G  9.1M  1.9G   1% /run
tmpfs                1.9G     0  1.9G   0% /sys/fs/cgroup
/dev/sda1           1014M  173M  842M  18% /boot
tmpfs                378M   16K  378M   1% /run/user/42
tmpfs                378M     0  378M   0% /run/user/0
/dev/sdb             2.0G  6.0M  1.8G   1% /client1
/dev/sdc             976M  2.6M  907M   1% /lvm
/dev/sdd              90M  1.6M   82M   2% /raid10

CSDN学习社区

CSDN联合极客时间，共同打造面向开发者的精品内容学习社区，助力成长！

更多推荐