LINUX.ORG.RU
Форум — Admin  

multipath -f map in use

 


0

1

Всем привет.

Имеется сервер, подключенный к хранилищу по FC.

Подскажите, как правильно удалить multipath связи?

Последовательность действий такая:

на хосте удаляю устройства 

echo 1 > /sys/block/sdd/device/delete
echo 1 > /sys/block/sdb/device/delete


 # multipath -f 3600000e00d1100000011362900150000
3600000e00d1100000011362900150000: map in use

 # multipath -ll
3600000e00d1100000011362900150000 dm-4 
size=20G features='0' hwhandler='0' wp=rw


В лог сыпятся вот такие ошибки.

Apr 13 12:43:46 node3 kernel: monitor/80d00d8: sending ioctl 5401 to a partition!
Apr 13 12:43:46 node3 kernel: blk_update_request: I/O error, dev dm-4, sector 0
Apr 13 12:43:46 node3 kernel: blk_update_request: I/O error, dev dm-4, sector 41942912
Apr 13 12:43:46 node3 kernel: blk_update_request: I/O error, dev dm-4, sector 41943024


Последнее исправление: ha-ha (всего исправлений: 1)

На multipath-девайсе создан раздел? Если да, то его тоже надо удалить.

kpartx -d /dev/mapper/mpatha
multipath -f mpatha

или 

dmsetup remove /dev/mapper/mpathap1
dmsetup remove /dev/mapper/mpatha
и только потом уже 
echo 1 > /sys/block/sdd/device/delete

bigbit ★★★★★
()
Ответ на: комментарий от router

Да, ничего не смонтировано:

cat /proc/mounts | grep /dev/mapper
/dev/mapper/3600000e00d1100000011362900100000p2 / ext4 rw,relatime,data=ordered 0 0
/dev/mapper/3600000e00d1100000011362900100000p1 /boot ext4 rw,relatime,data=ordered 0 0
ha-ha
() автор топика
Ответ на: комментарий от bigbit

kpartx -d /dev/mapper/3600000e00d1100000011362900150000 - делаю - ничего не происходит.

multipath -f 3600000e00d1100000011362900150000 - по прежнему ругается.

Удалить не получается:

# dmsetup remove /dev/mapper/3600000e00d1100000011362900150000 

device-mapper: remove ioctl on 3600000e00d1100000011362900150000  failed: Device or resource busy
Command failed
Причем ничего не держит:

# dmsetup info /dev/mapper/3600000e00d1100000011362900150000 

Name:              3600000e00d1100000011362900150000
State:             ACTIVE
Read Ahead:        8192
Tables present:    LIVE
Open count:        7
Event number:      16
Major, minor:      253, 4
Number of targets: 1
UUID: mpath-3600000e00d1100000011362900150000

lsof | grep «253,4» - пустой вывод

# ll /dev/mapper/ 

total 0
lrwxrwxrwx 1 root root       7 Apr 13 14:29 3600000e00d1100000011362900100000 -> ../dm-0
lrwxrwxrwx 1 root root       7 Apr 13 14:29 3600000e00d1100000011362900100000p1 -> ../dm-1
lrwxrwxrwx 1 root root       7 Apr 13 14:29 3600000e00d1100000011362900100000p2 -> ../dm-2
lrwxrwxrwx 1 root root       7 Apr 13 14:29 3600000e00d1100000011362900100000p3 -> ../dm-3
lrwxrwxrwx 1 root root       7 Apr 13 15:20 3600000e00d1100000011362900150000 -> ../dm-4
lrwxrwxrwx 1 root root       8 Apr 13 14:29 3600000e00d11000000113629001d0000 -> ../dm-12
lrwxrwxrwx 1 root root       7 Apr 13 12:00 79b6de6f--d873--417e--8510--48f903aad78c-ids -> ../dm-9
lrwxrwxrwx 1 root root       8 Apr 13 11:58 79b6de6f--d873--417e--8510--48f903aad78c-inbox -> ../dm-10
lrwxrwxrwx 1 root root       7 Apr 13 12:00 79b6de6f--d873--417e--8510--48f903aad78c-leases -> ../dm-8
lrwxrwxrwx 1 root root       8 Apr 13 11:58 79b6de6f--d873--417e--8510--48f903aad78c-master -> ../dm-11
lrwxrwxrwx 1 root root       7 Apr 13 12:00 79b6de6f--d873--417e--8510--48f903aad78c-metadata -> ../dm-5
lrwxrwxrwx 1 root root       7 Apr 13 11:58 79b6de6f--d873--417e--8510--48f903aad78c-outbox -> ../dm-6
lrwxrwxrwx 1 root root       7 Apr 13 11:58 79b6de6f--d873--417e--8510--48f903aad78c-xleases -> ../dm-7
lrwxrwxrwx 1 root root       8 Apr 13 12:08 80d00d8b--9f29--46e7--ba11--099e60657583-ids -> ../dm-17
lrwxrwxrwx 1 root root       8 Apr 13 12:08 80d00d8b--9f29--46e7--ba11--099e60657583-inbox -> ../dm-18
lrwxrwxrwx 1 root root       8 Apr 13 12:13 80d00d8b--9f29--46e7--ba11--099e60657583-leases -> ../dm-16
lrwxrwxrwx 1 root root       8 Apr 13 12:08 80d00d8b--9f29--46e7--ba11--099e60657583-master -> ../dm-19
lrwxrwxrwx 1 root root       8 Apr 13 12:13 80d00d8b--9f29--46e7--ba11--099e60657583-metadata -> ../dm-13
lrwxrwxrwx 1 root root       8 Apr 13 12:08 80d00d8b--9f29--46e7--ba11--099e60657583-outbox -> ../dm-14
lrwxrwxrwx 1 root root       8 Apr 13 12:08 80d00d8b--9f29--46e7--ba11--099e60657583-xleases -> ../dm-15
crw------- 1 root root 10, 236 Apr  6 15:16 control

# lvs 

  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 0: Input/output error
  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 21474770944: Input/output error
  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 21474828288: Input/output error
  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 536805376: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 536862720: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 134152192: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 134209536: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 1073676288: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 1073733632: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 2147418112: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 2147475456: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 134152192: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 134209536: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 134152192: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 134209536: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 1073676288: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 1073733632: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 4096: Input/output error
  LV                                   VG                                   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  40a089b6-53e8-43be-aa67-871dfaa91e48 80d00d8b-9f29-46e7-ba11-099e60657583 -wi------- 128.00m                                                    
  6430d5de-0980-4b9a-a111-0572194096df 80d00d8b-9f29-46e7-ba11-099e60657583 -wi------- 128.00m                                                    
  ids                                  80d00d8b-9f29-46e7-ba11-099e60657583 -wi-ao---- 128.00m                                                    
  inbox                                80d00d8b-9f29-46e7-ba11-099e60657583 -wi-a----- 128.00m                                                    
  leases                               80d00d8b-9f29-46e7-ba11-099e60657583 -wi-a-----   2.00g                                                    
  master                               80d00d8b-9f29-46e7-ba11-099e60657583 -wi-a-----   1.00g                                                    
  metadata                             80d00d8b-9f29-46e7-ba11-099e60657583 -wi-a----- 512.00m                                                    
  outbox                               80d00d8b-9f29-46e7-ba11-099e60657583 -wi-a----- 128.00m                                                    
  xleases                              80d00d8b-9f29-46e7-ba11-099e60657583 -wi-a-----   1.00g

ha-ha
() автор топика
Ответ на: комментарий от a15

# pvdisplay 

  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 0: Input/output error
  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 21474770944: Input/output error
  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 21474828288: Input/output error
  /dev/mapper/3600000e00d1100000011362900150000: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 536805376: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 536862720: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/metadata: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 134152192: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 134209536: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/outbox: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 1073676288: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 1073733632: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/xleases: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 2147418112: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 2147475456: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/leases: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 134152192: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 134209536: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/ids: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 134152192: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 134209536: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/inbox: read failed after 0 of 4096 at 4096: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 0: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 1073676288: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 1073733632: Input/output error
  /dev/79b6de6f-d873-417e-8510-48f903aad78c/master: read failed after 0 of 4096 at 4096: Input/output error
  --- Physical volume ---
  PV Name               /dev/mapper/3600000e00d11000000113629001d0000
  VG Name               80d00d8b-9f29-46e7-ba11-099e60657583
  PV Size               1.00 TiB / not usable 384.00 MiB
  Allocatable           yes 
  PE Size               128.00 MiB
  Total PE              8189
  Free PE               8148
  Allocated PE          41
  PV UUID               7TcQVM-zoBJ-t928-Br32-NMU5-aBt5-K8OAGR

ha-ha
() автор топика
Ответ на: комментарий от ha-ha 
dmsetup suspend /dev/mapper/3600000e00d1100000011362900150000
dmsetup info /dev/mapper/3600000e00d1100000011362900150000
dmsetup clear /dev/mapper/3600000e00d1100000011362900150000
dmsetup wipe_table /dev/mapper/3600000e00d1100000011362900150000
mutipath -f /dev/mapper/3600000e00d1100000011362900150000
a15
()
Ответ на: комментарий от bigbit

Немного не так: LVM создавались не руками, а через некий WEBUI и также удалялись. А руками пришлось полезть в консоль, потому что возникли серьёзные проблемы из за этих ошибок.

ha-ha
() автор топика
Вы не можете добавлять комментарии в эту тему. Тема перемещена в архив.
Admin