URL: https://linuxfr.org/forums/linux-debian-ubuntu/posts/probleme-de-perf-en-zfs Title: Problème de perf en zfs Authors: Ecran Plat Date: 2018年06月07日T16:07:02+02:00 License: CC By-SA Tags: zfs Score: 2 Bonjour, ici au taf je suis entrain de changer les serveurs de backups, un dans chaque bâtiment et ils se réplique entre heu grace à zfs. Les machines sont deux dell poweredge R540 avec carte Perc H330 et 6 disques de 4To SAS 10k tours/min chacune. Bon j'ai passé les cartes raid en hba, installé proxmox 5.2 (la dernière version) avec l'option raidz1 de zfs, j'ai un rpool de 18.4To. Je met mes deux proxmox dans un cluster dédié au backup (l'infra tourne avec un autre cluster de 4 serveurs proxmox). Je construit mon premier container qui va faire tourner un rsnapshot (il y aura un container rsnapshot pour les sauvegardes de fichiers, un container nfs pour que les serveurs du cluster proxmox principale) J'active la réplication zfs du dataset de mon container sur sont sistership situé dans l'autre bâtiment, ça fonctionne bien. Je me dis bon je vais activé la déduplication. Je fait un zfs set dedup=on rpool/data/subvol-100-disk-1 je lance mon premier rsnapshot, ça commence à copier et je rentre à la maison. Ce matin constat que c'est super lent, par contre le dataset en face ce met à jours tous les 15min. J'arrête et je commence à douter des accès disques. Permier truc je fait un zfs set dedup=off rpool/data/subvol-100-disk-1 ça ne change rien. Je fait un dans le dataset du système qui n'a jamais eux de déduplication: /usr/bin/time dd if=/dev/zero of=sometestfile bs=1024 count=300000 et il me répond: 307200000 bytes (307 MB, 293 MiB) copied, 2.82471 s, 109 MB/s 109MB/s c'est 4 fois moins que mon plus vieux serveurs en prod et même mon proxmox qui tourne dans ma cave avec deux disques sata les moins chère en raid1 (sous zfs) va deux fois plus vite. La je suis largué, que faire ? Remettre la carte en raid5 et exposer un seul disque virtuel au proxmox et lui installer zfs dessus (je veux zfs pour les snapshots à distance) ? Est ce raidz1 qui est mauvais (équivalent du raid 5) ? Faut 'il les mettre en raidz2 ou raidz3 ? Mon but est d'avoir le plus grand stockage disponible. La sécurité est moindre parce que les serveurs sont redondant, donc protéger contre la perte d'un disque est suffisant. Je vous met ci joint le résultat d'un zpool status: root@soyouz:~# zpool status pool: rpool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM rpool ONLINE 0 0 0 raidz1-0 ONLINE 0 0 0 sda2 ONLINE 0 0 0 sdb2 ONLINE 0 0 0 sdc2 ONLINE 0 0 0 sdd2 ONLINE 0 0 0 sde2 ONLINE 0 0 0 sdf2 ONLINE 0 0 0 errors: No known data errors root@soyouz:~# un zfs get all: root@soyouz:~# zfs get all NAME PROPERTY VALUE SOURCE rpool type filesystem - rpool creation Tue Jun 5 12:04 2018 - rpool used 324G - rpool available 16.6T - rpool referenced 153K - rpool compressratio 1.49x - rpool mounted yes - rpool quota none default rpool reservation none default rpool recordsize 128K default rpool mountpoint /rpool default rpool sharenfs off default rpool checksum on default rpool compression on local rpool atime off local rpool devices on default rpool exec on default rpool setuid on default rpool readonly off default rpool zoned off default rpool snapdir hidden default rpool aclinherit restricted default rpool createtxg 1 - rpool canmount on default rpool xattr on default rpool copies 1 default rpool version 5 - rpool utf8only off - rpool normalization none - rpool casesensitivity sensitive - rpool vscan off default rpool nbmand off default rpool sharesmb off default rpool refquota none default rpool refreservation none default rpool guid 9698752250673717501 - rpool primarycache all default rpool secondarycache all default rpool usedbysnapshots 0B - rpool usedbydataset 153K - rpool usedbychildren 324G - rpool usedbyrefreservation 0B - rpool logbias latency default rpool dedup off default rpool mlslabel none default rpool sync standard local rpool dnodesize legacy default rpool refcompressratio 1.00x - rpool written 153K - rpool logicalused 462G - rpool logicalreferenced 40K - rpool volmode default default rpool filesystem_limit none default rpool snapshot_limit none default rpool filesystem_count none default rpool snapshot_count none default rpool snapdev hidden default rpool acltype off default rpool context none default rpool fscontext none default rpool defcontext none default rpool rootcontext none default rpool relatime off default rpool redundant_metadata all default rpool overlay off default rpool/ROOT type filesystem - rpool/ROOT creation Tue Jun 5 12:04 2018 - rpool/ROOT used 1.33G - rpool/ROOT available 16.6T - rpool/ROOT referenced 153K - rpool/ROOT compressratio 1.71x - rpool/ROOT mounted yes - rpool/ROOT quota none default rpool/ROOT reservation none default rpool/ROOT recordsize 128K default rpool/ROOT mountpoint /rpool/ROOT default rpool/ROOT sharenfs off default rpool/ROOT checksum on default rpool/ROOT compression on inherited from rpool rpool/ROOT atime off inherited from rpool rpool/ROOT devices on default rpool/ROOT exec on default rpool/ROOT setuid on default rpool/ROOT readonly off default rpool/ROOT zoned off default rpool/ROOT snapdir hidden default rpool/ROOT aclinherit restricted default rpool/ROOT createtxg 6 - rpool/ROOT canmount on default rpool/ROOT xattr on default rpool/ROOT copies 1 default rpool/ROOT version 5 - rpool/ROOT utf8only off - rpool/ROOT normalization none - rpool/ROOT casesensitivity sensitive - rpool/ROOT vscan off default rpool/ROOT nbmand off default rpool/ROOT sharesmb off default rpool/ROOT refquota none default rpool/ROOT refreservation none default rpool/ROOT guid 14098958435902672841 - rpool/ROOT primarycache all default rpool/ROOT secondarycache all default rpool/ROOT usedbysnapshots 0B - rpool/ROOT usedbydataset 153K - rpool/ROOT usedbychildren 1.33G - rpool/ROOT usedbyrefreservation 0B - rpool/ROOT logbias latency default rpool/ROOT dedup off default rpool/ROOT mlslabel none default rpool/ROOT sync standard inherited from rpool rpool/ROOT dnodesize legacy default rpool/ROOT refcompressratio 1.00x - rpool/ROOT written 153K - rpool/ROOT logicalused 1.95G - rpool/ROOT logicalreferenced 40K - rpool/ROOT volmode default default rpool/ROOT filesystem_limit none default rpool/ROOT snapshot_limit none default rpool/ROOT filesystem_count none default rpool/ROOT snapshot_count none default rpool/ROOT snapdev hidden default rpool/ROOT acltype off default rpool/ROOT context none default rpool/ROOT fscontext none default rpool/ROOT defcontext none default rpool/ROOT rootcontext none default rpool/ROOT relatime off default rpool/ROOT redundant_metadata all default rpool/ROOT overlay off default rpool/ROOT/pve-1 type filesystem - rpool/ROOT/pve-1 creation Tue Jun 5 12:04 2018 - rpool/ROOT/pve-1 used 1.33G - rpool/ROOT/pve-1 available 16.6T - rpool/ROOT/pve-1 referenced 1.33G - rpool/ROOT/pve-1 compressratio 1.71x - rpool/ROOT/pve-1 mounted yes - rpool/ROOT/pve-1 quota none default rpool/ROOT/pve-1 reservation none default rpool/ROOT/pve-1 recordsize 128K default rpool/ROOT/pve-1 mountpoint / local rpool/ROOT/pve-1 sharenfs off default rpool/ROOT/pve-1 checksum on default rpool/ROOT/pve-1 compression on inherited from rpool rpool/ROOT/pve-1 atime off inherited from rpool rpool/ROOT/pve-1 devices on default rpool/ROOT/pve-1 exec on default rpool/ROOT/pve-1 setuid on default rpool/ROOT/pve-1 readonly off default rpool/ROOT/pve-1 zoned off default rpool/ROOT/pve-1 snapdir hidden default rpool/ROOT/pve-1 aclinherit restricted default rpool/ROOT/pve-1 createtxg 9 - rpool/ROOT/pve-1 canmount on default rpool/ROOT/pve-1 xattr on default rpool/ROOT/pve-1 copies 1 default rpool/ROOT/pve-1 version 5 - rpool/ROOT/pve-1 utf8only off - rpool/ROOT/pve-1 normalization none - rpool/ROOT/pve-1 casesensitivity sensitive - rpool/ROOT/pve-1 vscan off default rpool/ROOT/pve-1 nbmand off default rpool/ROOT/pve-1 sharesmb off default rpool/ROOT/pve-1 refquota none default rpool/ROOT/pve-1 refreservation none default rpool/ROOT/pve-1 guid 6978925590103990493 - rpool/ROOT/pve-1 primarycache all default rpool/ROOT/pve-1 secondarycache all default rpool/ROOT/pve-1 usedbysnapshots 0B - rpool/ROOT/pve-1 usedbydataset 1.33G - rpool/ROOT/pve-1 usedbychildren 0B - rpool/ROOT/pve-1 usedbyrefreservation 0B - rpool/ROOT/pve-1 logbias latency default rpool/ROOT/pve-1 dedup off default rpool/ROOT/pve-1 mlslabel none default rpool/ROOT/pve-1 sync standard inherited from rpool rpool/ROOT/pve-1 dnodesize legacy default rpool/ROOT/pve-1 refcompressratio 1.71x - rpool/ROOT/pve-1 written 1.33G - rpool/ROOT/pve-1 logicalused 1.95G - rpool/ROOT/pve-1 logicalreferenced 1.95G - rpool/ROOT/pve-1 volmode default default rpool/ROOT/pve-1 filesystem_limit none default rpool/ROOT/pve-1 snapshot_limit none default rpool/ROOT/pve-1 filesystem_count none default rpool/ROOT/pve-1 snapshot_count none default rpool/ROOT/pve-1 snapdev hidden default rpool/ROOT/pve-1 acltype off default rpool/ROOT/pve-1 context none default rpool/ROOT/pve-1 fscontext none default rpool/ROOT/pve-1 defcontext none default rpool/ROOT/pve-1 rootcontext none default rpool/ROOT/pve-1 relatime on temporary rpool/ROOT/pve-1 redundant_metadata all default rpool/ROOT/pve-1 overlay off default rpool/data type filesystem - rpool/data creation Tue Jun 5 12:04 2018 - rpool/data used 312G - rpool/data available 16.6T - rpool/data referenced 166K - rpool/data compressratio 1.49x - rpool/data mounted yes - rpool/data quota none default rpool/data reservation none default rpool/data recordsize 128K default rpool/data mountpoint /rpool/data default rpool/data sharenfs off default rpool/data checksum on default rpool/data compression on inherited from rpool rpool/data atime off inherited from rpool rpool/data devices on default rpool/data exec on default rpool/data setuid on default rpool/data readonly off default rpool/data zoned off default rpool/data snapdir hidden default rpool/data aclinherit restricted default rpool/data createtxg 8 - rpool/data canmount on default rpool/data xattr on default rpool/data copies 1 default rpool/data version 5 - rpool/data utf8only off - rpool/data normalization none - rpool/data casesensitivity sensitive - rpool/data vscan off default rpool/data nbmand off default rpool/data sharesmb off default rpool/data refquota none default rpool/data refreservation none default rpool/data guid 6882223753380654234 - rpool/data primarycache all default rpool/data secondarycache all default rpool/data usedbysnapshots 0B - rpool/data usedbydataset 166K - rpool/data usedbychildren 312G - rpool/data usedbyrefreservation 0B - rpool/data logbias latency default rpool/data dedup off default rpool/data mlslabel none default rpool/data sync standard inherited from rpool rpool/data dnodesize legacy default rpool/data refcompressratio 1.00x - rpool/data written 166K - rpool/data logicalused 459G - rpool/data logicalreferenced 44K - rpool/data volmode default default rpool/data filesystem_limit none default rpool/data snapshot_limit none default rpool/data filesystem_count none default rpool/data snapshot_count none default rpool/data snapdev hidden default rpool/data acltype off default rpool/data context none default rpool/data fscontext none default rpool/data defcontext none default rpool/data rootcontext none default rpool/data relatime off default rpool/data redundant_metadata all default rpool/data overlay off default rpool/data/subvol-100-disk-1 type filesystem - rpool/data/subvol-100-disk-1 creation Wed Jun 6 16:36 2018 - rpool/data/subvol-100-disk-1 used 312G - rpool/data/subvol-100-disk-1 available 1.65T - rpool/data/subvol-100-disk-1 referenced 312G - rpool/data/subvol-100-disk-1 compressratio 1.49x - rpool/data/subvol-100-disk-1 mounted yes - rpool/data/subvol-100-disk-1 quota none default rpool/data/subvol-100-disk-1 reservation none default rpool/data/subvol-100-disk-1 recordsize 128K default rpool/data/subvol-100-disk-1 mountpoint /rpool/data/subvol-100-disk-1 default rpool/data/subvol-100-disk-1 sharenfs off default rpool/data/subvol-100-disk-1 checksum on default rpool/data/subvol-100-disk-1 compression on inherited from rpool rpool/data/subvol-100-disk-1 atime off inherited from rpool rpool/data/subvol-100-disk-1 devices on default rpool/data/subvol-100-disk-1 exec on default rpool/data/subvol-100-disk-1 setuid on default rpool/data/subvol-100-disk-1 readonly off default rpool/data/subvol-100-disk-1 zoned off default rpool/data/subvol-100-disk-1 snapdir hidden default rpool/data/subvol-100-disk-1 aclinherit restricted default rpool/data/subvol-100-disk-1 createtxg 19158 - rpool/data/subvol-100-disk-1 canmount on default rpool/data/subvol-100-disk-1 xattr sa received rpool/data/subvol-100-disk-1 copies 1 default rpool/data/subvol-100-disk-1 version 5 - rpool/data/subvol-100-disk-1 utf8only off - rpool/data/subvol-100-disk-1 normalization none - rpool/data/subvol-100-disk-1 casesensitivity sensitive - rpool/data/subvol-100-disk-1 vscan off default rpool/data/subvol-100-disk-1 nbmand off default rpool/data/subvol-100-disk-1 sharesmb off default rpool/data/subvol-100-disk-1 refquota 1.95T received rpool/data/subvol-100-disk-1 refreservation none default rpool/data/subvol-100-disk-1 guid 15077408641230762542 - rpool/data/subvol-100-disk-1 primarycache all default rpool/data/subvol-100-disk-1 secondarycache all default rpool/data/subvol-100-disk-1 usedbysnapshots 0B - rpool/data/subvol-100-disk-1 usedbydataset 312G - rpool/data/subvol-100-disk-1 usedbychildren 0B - rpool/data/subvol-100-disk-1 usedbyrefreservation 0B - rpool/data/subvol-100-disk-1 logbias latency default rpool/data/subvol-100-disk-1 dedup off received rpool/data/subvol-100-disk-1 mlslabel none default rpool/data/subvol-100-disk-1 sync standard inherited from rpool rpool/data/subvol-100-disk-1 dnodesize legacy default rpool/data/subvol-100-disk-1 refcompressratio 1.49x - rpool/data/subvol-100-disk-1 written 0 - rpool/data/subvol-100-disk-1 logicalused 459G - rpool/data/subvol-100-disk-1 logicalreferenced 459G - rpool/data/subvol-100-disk-1 volmode default default rpool/data/subvol-100-disk-1 filesystem_limit none default rpool/data/subvol-100-disk-1 snapshot_limit none default rpool/data/subvol-100-disk-1 filesystem_count none default rpool/data/subvol-100-disk-1 snapshot_count none default rpool/data/subvol-100-disk-1 snapdev hidden default rpool/data/subvol-100-disk-1 acltype posixacl received rpool/data/subvol-100-disk-1 context none default rpool/data/subvol-100-disk-1 fscontext none default rpool/data/subvol-100-disk-1 defcontext none default rpool/data/subvol-100-disk-1 rootcontext none default rpool/data/subvol-100-disk-1 relatime off default rpool/data/subvol-100-disk-1 redundant_metadata all default rpool/data/subvol-100-disk-1 overlay off default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ type snapshot - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ creation Thu Jun 7 16:00 2018 - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ used 0B - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ referenced 312G - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ compressratio 1.49x - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ devices on default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ exec on default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ setuid on default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ createtxg 34699 - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ xattr sa inherited from rpool/data/subvol-100-disk-1 rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ version 5 - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ utf8only off - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ normalization none - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ casesensitivity sensitive - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ nbmand off default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ guid 3960069371916206380 - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ primarycache all default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ secondarycache all default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ defer_destroy off - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ userrefs 0 - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ mlslabel none default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ refcompressratio 1.49x - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ written 312G - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ clones - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ logicalreferenced 459G - rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ acltype posixacl inherited from rpool/data/subvol-100-disk-1 rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ context none default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ fscontext none default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ defcontext none default rpool/data/subvol-100-disk-1@__replicate_100-0_1528380003__ rootcontext none default rpool/swap type volume - rpool/swap creation Tue Jun 5 12:04 2018 - rpool/swap used 8.50G - rpool/swap available 16.6T - rpool/swap referenced 89.5K - rpool/swap compressratio 1.15x - rpool/swap reservation none default rpool/swap volsize 8G local rpool/swap volblocksize 4K - rpool/swap checksum on default rpool/swap compression zle local rpool/swap readonly off default rpool/swap createtxg 12 - rpool/swap copies 1 local rpool/swap refreservation 8.50G local rpool/swap guid 4371355181042701426 - rpool/swap primarycache metadata local rpool/swap secondarycache none local rpool/swap usedbysnapshots 0B - rpool/swap usedbydataset 89.5K - rpool/swap usedbychildren 0B - rpool/swap usedbyrefreservation 8.50G - rpool/swap logbias throughput local rpool/swap dedup off default rpool/swap mlslabel none default rpool/swap sync always local rpool/swap refcompressratio 1.15x - rpool/swap written 89.5K - rpool/swap logicalused 30K - rpool/swap logicalreferenced 30K - rpool/swap volmode default default rpool/swap snapshot_limit none default rpool/swap snapshot_count none default rpool/swap snapdev hidden default rpool/swap context none default rpool/swap fscontext none default rpool/swap defcontext none default rpool/swap rootcontext none default rpool/swap redundant_metadata all default rpool/swap com.sun:auto-snapshot false local root@soyouz:~# Je vous remercie d'avance pour votre aide

AltStyle によって変換されたページ (->オリジナル) /