Convert ext3 to ext4 filesystem on Debian Wheezy

老調重彈了,最近又再做這件事是因為Proxmox VE預設是用ext3當作檔案系統,某一台機器因為有點問題,灌Proxmox的時候一開始沒打算要當正式系統,也就沒特別改設定,但灌好後因為懶了 … 就直接上線了 … 偶然發現這個問題,順便重新作一下筆記怎麼樣轉到ext4,轉到ext4的好處以及效能差異就不用多說了,幾乎是完剩ext2、ext3,好檔案系統,不轉嗎?

如果懶的話,最簡單的方法就是去改fstab,直接把ext3掛成ext4,效能就可以有所提升,但只能使用到那些不需要修改到檔案系統的功能,並且可以重新用ext3的方式掛載,而要有比較完整的轉換則要做以下動作:

  1. 把要做轉換的分區umount(如果是root filesystem就進single user mode或用其他系統開機吧),DEV自行替換成對應代號
    # umount /dev/DEV
  2. 先做一次fsck
    # e2fsck -fyv /dev/DEV

  3. 沒問題的話就可以把ext4的功能打開了,這邊要注意 … 這是不可逆的操作,並且要和下個動作連貫完成
    # tune2fs -O extents,uninit_bg,dir_index /dev/DEV

  4. 調整完後要修一下(這邊會看到找到錯誤是正常現象)
    # e2fsck -fyvDC0 /dev/DEV

  5. 調整fstab的掛載格式
    -> 自己去fstab理面把該分區的ext3改成ext4吧!

  6. 重組
    # e4defrag -c -v /dev/DEV
    -> ext4開始使用extents來取代傳統的block mapping,在對大檔案操作時能有顯著的效能提升,而這部份是在剛剛的轉換沒有做的(tuen2fs只有把feature打開而以),轉換完成後新增的檔案自然是都會使用到這樣的功能,但原先已經存在於filesystem上的檔案必須要到下次被寫入時才會以新的方式寫入,但系統上有不少檔案是久久才會寫一次但可能經常要被讀取的,在被重新寫入以前便無法享受到這樣的效能改善,所以這邊透過e4defrag來幫我們做這件事!

過程中如果有出現錯誤的話就先解掉再往下一步走,像fsck這種工作做完一次還可以做第二次確保都沒問題了再往下走,過程並不複雜,剛好也可以順便檢查檔案系統有沒有問題,比較麻煩的大概就是root filesystem因為要unmount所以系統必須停機,但換來的效能是值得的!

(我之前硬碟壞掉就會出現fsck永遠修不完的現象,每次抓到的錯誤都不一樣,修完一次下次還是有得修,那很明顯是硬碟壞了)

Online disk defragment on Linux

Though we rarely defragment the filesystem on unix-like os, sometimes we still need it.

The demonstrate is on LinuxMint 17.1 Cinnamon 64-bit which is based on Ubuntu 14.04 LTS.

Okay, let’s check the fragmentation first!

Here is a very easy method to check the fragment on ext4 online:
$ sudo fsck -fn /dev/sdXY, replace XY for yourself, for example, $ sudo fsck -fn /dev/sda1

The output will look like this:

fsck from util-linux 2.20.1
e2fsck 1.42.5 (29-Jul-2012)
Warning! /dev/sda1 is mounted.
Warning: skipping journal recovery because doing a read-only filesystem check.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sda1: 247/121920 files (0.4% non-contiguous), 66062/487424 blocks

(BTW, this method could also work on ext2, ext3)

and on xfs:
$ xfs_db -c frag -r /dev/sdXY, for example,$ xfs_db -c frag -r /dev/sda1

The output will look like this:

actual 297843, ideal 286076, fragmentation factor 3.95%

And defrag a partition now!

This is the command to defrag ext4 online:
$ sudo e4defrag -v /dev/sdXY, for example, $ sudo e4defrag -v /dev/sda1
(BTW, device /dev/sdXY can be replaced by a regular file or a directory, this is supported in e2fsprogs 1.42.9)

The output will look like this:

ext4 defragmentation for device(/dev/sda1)
[2/238]/boot/System.map-3.2.0-4-686-pae: 100% [ OK ]
[3/238]/boot/vmlinuz-3.16-0.bpo.2-686-pae: 100% [ OK ]
[5/238]/boot/grub/video_cirrus.mod: 100% [ OK ]
[6/238]/boot/grub/usb_keyboard.mod: 100% [ OK ]
[8/238]/boot/grub/locale/pl.mo: 100% [ OK ]
[9/238]/boot/grub/locale/nl.mo: 100% [ OK ]

.
.
.

[223/238]/boot/grub/password_pbkdf2.mod: 100% [ OK ]
[224/238]/boot/grub/videotest.mod: 100% [ OK ]
[225/238]/boot/grub/ntldr.mod: 100% [ OK ]
[226/238]/boot/grub/udf.mod: 100% [ OK ]
[227/238]/boot/grub/halt.mod: 100% [ OK ]
[228/238]/boot/grub/minix.mod: 100% [ OK ]
[229/238]/boot/grub/gfxterm.mod: 100% [ OK ]
[230/238]/boot/config-3.2.0-4-686-pae: 100% [ OK ]
[231/238]/boot/memtest86+_multiboot.bin: 100% [ OK ]
[232/238]/boot/memtest86+.bin: 100% [ OK ]
[233/238]/boot/System.map-3.16-0.bpo.2-686-pae: 100% [ OK ]
[234/238]/boot/initrd.img-3.2.0-4-686-pae: 100% [ OK ]
[235/238]/boot/initrd.img-3.16-0.bpo.2-686-pae: 100% [ OK ]
[237/238]/boot/config-3.16-0.bpo.2-686-pae: 100% [ OK ]
[238/238]/boot/vmlinuz-3.2.0-4-686-pae: 100% [ OK ]

Success: [ 234/238 ]
Failure: [ 4/238 ]

And here is the command to defrag xfs online:
$ sudo xfs_fsr -v /dev/sdXY, for example, $ sudo xfs_fsr -v /dev/sda1
(device /dev/sdXY by a file, but a directory is not supported yet in xfs_fsr version 3.1.9)

/ start inode=0
ino=1020
extents before:2 after:1 DONE ino=1020
ino=2000
extents before:3 after:1 DONE ino=2000
ino=2270
extents before:2 after:1 DONE ino=2270
ino=2272
extents before:2 after:1 DONE ino=2272
ino=2497
extents before:2 after:1 DONE ino=2497
ino=2498
extents before:2 after:1 DONE ino=2498
.
.
.

Not very hard, but useful, if the data on the target to defrag is important, I’ll respectfully suggest you to backup first, in case there is an accident like losing electrical power.

Error: Read-only file system on Linux

Today I found that there is a strange situation, in certain mount point on my web server (Debian GNU/Linux 7.8), every process with write behavior will get messages like this (That’s the result from command touch test):

touch: cannot touch `test’: Read-only file system

I don’t think I mount it as read-only, so I check it with(the mount point is /srv):

$ mount | grep srv

And I got:

/dev/sdb1 on /srv type ext4 (ro,relatime,data=ordered)

Very strange … there was something wrong, could I remount it with r/w? I tried:

$ sudo mount -o remount,rw,relatime,data=ordered /srv

Hmmmm …

mount: cannot remount block device /dev/sdb1 read-write, is write-protected

Okay, seems I need to check the filesystem …

$ sudo umount /srv
$ sudo fsck.ext4 /dev/sdb1

The process didn’t take a very long time, just within few mins, and that’s the result:

e2fsck 1.42.5 (29-Jul-2012)
/dev/sdb1: recovering journal
/dev/sdb1 contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/sdb1: 796453/122077184 files (3.8% non-contiguous), 321192367/488281498 blocks

Let’s mount it again:

$ sudo mount -o rw,relatime,data=ordered /srv 

Oh, I didn’t get any error message this time, let’s check it again:

$ mount | grep srv

And … it’s good again

/dev/sdb1 on /srv type ext4 (rw,relatime,data=ordered)

Okay, nice … it’s very easy this time, I’ve got some disks with damaged partition before, event I can’t not recognize its partition type, though I fixed it, but I didn’t record the process that time, haha.