issue189:disques_durs
Différences
Ci-dessous, les différences entre deux révisions de la page.
Prochaine révision | Révision précédente | ||
issue189:disques_durs [2023/01/29 20:31] – créée d52fr | issue189:disques_durs [2023/02/02 11:36] (Version actuelle) – auntiee | ||
---|---|---|---|
Ligne 1: | Ligne 1: | ||
- | Like most users, I do change my main working computer from time to time. When doing so, it may be a good time to start out afresh and give some thought to our hard disk setup. Even if cloud storage is widely used, limits on amounts of storage available (or its monthly fee), and network bandwidth to get our files up into the cloud, mean most users actually keep a sizable portion of their files on their local drives. Security considerations may also be a factor to consider, even with smaller files. In this article, I would like to set out several ideas that a more or less technical user could consider, strategies that may be of help both for data deduplication on your local drive, or for extending a drive with a limited capacity we are growing out of. | + | **Like most users, I do change my main working computer from time to time. When doing so, it may be a good time to start out afresh and give some thought to our hard disk setup. Even if cloud storage is widely used, limits on amounts of storage available (or its monthly fee), and network bandwidth to get our files up into the cloud, mean most users actually keep a sizable portion of their files on their local drives. Security considerations may also be a factor to consider, even with smaller files. In this article, I would like to set out several ideas that a more or less technical user could consider, strategies that may be of help both for data deduplication on your local drive, or for extending a drive with a limited capacity we are growing out of. |
Why mirror data? | Why mirror data? | ||
- | Making multiple copies of all files that are written to our computer’s hard drives is a technique used in all server environments; | + | Making multiple copies of all files that are written to our computer’s hard drives is a technique used in all server environments; |
- | Perhaps things are not as critical with modern SSD technology as they were at one point with rotational drives and their moving disks. The Mean Time Between Failures (MTBF) is a commonly-used metric that seems to have gone up with the switch to solid-state technology. But still, common sense tells us it is perhaps best not to entrust all our data to a single copy on our internal drive without making any backups. Fine… but there still are obstacles. In practice, not all backups are made on a schedule and kept up-to-date. At times, large files may never even get backed up at all -- neither to the cloud because they are too large, nor to a local external drive because we would need to think of plugging it in and performing the backup. Since, in the immortal words of Murphy’s Law, anything that can go wrong will do so and at the worst possible time, it can make sense to have multiple copies of all our data, made automatically, on our daily driver computer. | + | Comme la plupart des utilisateurs, |
- | Clearly, making several copies of our files on a single disk would not be very useful at all. In the event of that disk having a catastrophic physical failure, we would lose access to all data it contained and the fact that it held several copies of each file would be moot. So, our goal must include the use of more than one physical hard drive on our computer. And, since messing around with the hard drives on our daily driver is probably not a great idea, what better time to take a look at the possibilities than when acquiring a new (or new-to-us) machine? | + | Pourquoi mettre les données en miroir |
- | Why just mirror data? | + | Faire plusieurs copies de tous les fichiers qui sont écrits sur les disques durs de notre ordinateur est une technique utilisée dans tous les environnements de serveur ; chaque fois qu'un fichier est modifié, des copies sont écrites sur plus d'un disque en utilisant l'un des différents schémas. L' |
+ | |||
+ | |||
+ | **Perhaps things are not as critical with modern SSD technology as they were at one point with rotational drives and their moving disks. The Mean Time Between Failures (MTBF) is a commonly-used metric that seems to have gone up with the switch to solid-state technology. But still, common sense tells us it is perhaps best not to entrust all our data to a single copy on our internal drive without making any backups. Fine… but there still are obstacles. In practice, not all backups are made on a schedule and kept up-to-date. At times, large files may never even get backed up at all -- neither to the cloud because they are too large, nor to a local external drive because we would need to think of plugging it in and performing the backup. Since, in the immortal words of Murphy’s Law, anything that can go wrong will do so and at the worst possible time, it can make sense to have multiple copies of all our data, made automatically, | ||
+ | |||
+ | Clearly, making several copies of our files on a single disk would not be very useful at all. In the event of that disk having a catastrophic physical failure, we would lose access to all data it contained and the fact that it held several copies of each file would be moot. So, our goal must include the use of more than one physical hard drive on our computer. And, since messing around with the hard drives on our daily driver is probably not a great idea, what better time to take a look at the possibilities than when acquiring a new (or new-to-us) machine? | ||
+ | |||
+ | Avec la technologie SSD moderne, les choses ne sont peut-être pas aussi critiques qu' | ||
+ | |||
+ | Il est clair que faire plusieurs copies de nos fichiers sur un seul disque ne serait pas du tout utile. En cas de défaillance physique catastrophique de ce disque, nous perdrions l' | ||
+ | |||
+ | |||
+ | **Why just mirror data? | ||
The term “mirrored drive” implies the use of exactly two drives of a certain capacity. All data is written to disk in two copies, one on each disk. So, in essence, we are buying (and probably paying for) twice the disk space, and then using only the capacity of a single unit giving us a disk use efficiency of a mere 50%. | The term “mirrored drive” implies the use of exactly two drives of a certain capacity. All data is written to disk in two copies, one on each disk. So, in essence, we are buying (and probably paying for) twice the disk space, and then using only the capacity of a single unit giving us a disk use efficiency of a mere 50%. | ||
- | In server rooms, this would be considered wasteful, since techniques such as RAID-5, RAID-6 or others allow higher efficiencies -- given a sufficient number of disks. For instance, with an 8-disk setup and using single parity, a RAID-5 would give us a useful space of 7 times the capacity of a single disk, while buying eight times that worth of space. 7/8 or 87,5% disk usage efficiency, while retaining the ability to survive the death of any single disk without losing any data is not a bad result. However, the immediate objection is that while it may be practical to house a large number of disks in a server rack, things are a lot more complicated in a typical laptop, or even some desktop computers with slim form factors. Even the smaller grade of commercially-available Network Attached Storage (NAS) units tend to have receptacles or bays for only two hard drives, thus limiting us to mirroring or – in technical parlance -- RAID-1. | + | In server rooms, this would be considered wasteful, since techniques such as RAID-5, RAID-6 or others allow higher efficiencies -- given a sufficient number of disks. For instance, with an 8-disk setup and using single parity, a RAID-5 would give us a useful space of 7 times the capacity of a single disk, while buying eight times that worth of space. 7/8 or 87,5% disk usage efficiency, while retaining the ability to survive the death of any single disk without losing any data is not a bad result. However, the immediate objection is that while it may be practical to house a large number of disks in a server rack, things are a lot more complicated in a typical laptop, or even some desktop computers with slim form factors. Even the smaller grade of commercially-available Network Attached Storage (NAS) units tend to have receptacles or bays for only two hard drives, thus limiting us to mirroring or – in technical parlance -- RAID-1.** |
- | On most desktop machines, the motherboard will have at least two SATA interfaces, where two rotational drives or two SSD drives with SATA interfaces may be plugged in. Even on many laptops, a second SATA hard drive may often be installed. If the laptop comes with an optical drive, a possibility would be to replace this little-used piece of equipment with a hard drive carrier that just plugs in the same space the DVD used to take. This is actually quite useful, mainly with laptops with the larger form factors. | + | Pourquoi se contenter d'une mise en miroir des données ? |
- | Modern technology can also give us the possibility of using M2 interface SSD drives. These take up quite a bit less space than rotational drives with the SATA interface. Many laptop motherboards either come equipped with a disk drive with this interface, or an available M2 slot on the motherboard. However, most laptops will only have a single M2 interface, so some creativity may be necessary to combine, for instance, one SSD on the M2 interface with a second one on a SATA interface. This setup would not be advisable from the standpoint of speed, but should be doable if data redundancy is the primary objective and maximizing throughput is not (an SSD drive should be plenty fast, even on a SATA interface). | + | Le terme « disque miroir » implique l' |
- | On the other hand, desktop machines can use a PCI-Express adapter (extension card) to add an M2 interface to an existing machine. Some adapters are for a single M2 drive, while others even have space for two M2 interfaces thus allowing the user to build a two-drive system on a single adapter. What is more, the cost can be quite moderate, with some dual M2 interface adapters available for as little as 17 USD. However, maybe ensure your computer can boot off this extension card before acquiring one. | + | Dans les salles de serveurs, cela serait considéré comme un gaspillage, puisque des techniques telles que RAID-5, RAID-6 ou autres permettent des efficacités plus élevées, à condition d' |
- | As a final thought, it may be useful to remember that using two partitions situated on the same physical disk would not be a good idea: not only will access times be lengthened since all data needs to be written twice to the very same disk thus creating a bottleneck, but if that unit goes out physically, both partitions will be lost at once thus rendering mirroring completely useless as implemented. To sum up: mirroring requires the use of (at least) two physical disks or drives. | ||
- | Why extend an existing disk (striping) instead of mirroring? | + | **On most desktop machines, the motherboard will have at least two SATA interfaces, where two rotational drives or two SSD drives with SATA interfaces may be plugged in. Even on many laptops, a second SATA hard drive may often be installed. If the laptop comes with an optical drive, a possibility would be to replace this little-used piece of equipment with a hard drive carrier that just plugs in the same space the DVD used to take. This is actually quite useful, mainly with laptops with the larger form factors. |
+ | |||
+ | Modern technology can also give us the possibility of using M2 interface SSD drives. These take up quite a bit less space than rotational drives with the SATA interface. Many laptop motherboards either come equipped with a disk drive with this interface, or an available M2 slot on the motherboard. However, most laptops will only have a single M2 interface, so some creativity may be necessary to combine, for instance, one SSD on the M2 interface with a second one on a SATA interface. This setup would not be advisable from the standpoint of speed, but should be doable if data redundancy is the primary objective and maximizing throughput is not (an SSD drive should be plenty fast, even on a SATA interface).** | ||
+ | |||
+ | Sur la plupart des ordinateurs de bureau, la carte mère dispose d'au moins deux interfaces SATA, sur lesquelles il est possible de brancher deux disques rotatifs ou deux disques SSD avec des interfaces SATA. Et même, sur de nombreux ordinateurs portables, un deuxième disque dur SATA peut souvent être installé. Si l' | ||
+ | |||
+ | La technologie moderne nous offre également la possibilité d' | ||
+ | |||
+ | |||
+ | **On the other hand, desktop machines can use a PCI-Express adapter (extension card) to add an M2 interface to an existing machine. Some adapters are for a single M2 drive, while others even have space for two M2 interfaces thus allowing the user to build a two-drive system on a single adapter. What is more, the cost can be quite moderate, with some dual M2 interface adapters available for as little as 17 USD. However, maybe ensure your computer can boot off this extension card before acquiring one. | ||
+ | |||
+ | As a final thought, it may be useful to remember that using two partitions situated on the same physical disk would not be a good idea: not only will access times be lengthened since all data needs to be written twice to the very same disk thus creating a bottleneck, but if that unit goes out physically, both partitions will be lost at once thus rendering mirroring completely useless as implemented. To sum up: mirroring requires the use of (at least) two physical disks or drives.** | ||
+ | |||
+ | En revanche, les machines de bureau peuvent utiliser un adaptateur PCI-Express (carte d' | ||
+ | |||
+ | Pour finir, il peut être utile de rappeler que l' | ||
+ | |||
+ | |||
+ | **Why extend an existing disk (striping) instead of mirroring? | ||
For some people, mirroring data may not even be necessary for the user’s workflow. This would be the case when the user actually generates most data in the cloud – or on a company server, and the local hard drive is seen more as an archive to have a local copy of files in case the remote server is no longer available or some space needs to be freed up. Another scenario would be the “perfect” user who has one copy of each file locally, one copy on a remote server, and a third on cold storage, for instance on an archived external disk or optical media. Finally, we may at some times need a large amount of disk space to store files for a relatively short period of time, after which this large amount of data will be discarded. This is usually the scenario found while editing video, for instance, where the final product would certainly be kept, but perhaps not most of the original material if it is no longer relevant. | For some people, mirroring data may not even be necessary for the user’s workflow. This would be the case when the user actually generates most data in the cloud – or on a company server, and the local hard drive is seen more as an archive to have a local copy of files in case the remote server is no longer available or some space needs to be freed up. Another scenario would be the “perfect” user who has one copy of each file locally, one copy on a remote server, and a third on cold storage, for instance on an archived external disk or optical media. Finally, we may at some times need a large amount of disk space to store files for a relatively short period of time, after which this large amount of data will be discarded. This is usually the scenario found while editing video, for instance, where the final product would certainly be kept, but perhaps not most of the original material if it is no longer relevant. | ||
- | In these cases, would it not make more sense to use several drives to extend the total amount of space available, while retaining just a single copy of each file. In essence, we are trading some data security for available space. In the same way, the concept of mirroring deals with combining several hard drives into a single unit in which files are stored in two copies, striping is the analogous concept of combining several drives into one unit with a larger capacity but without duplication. In technical terms, where mirroring is often spoken of as RAID-1, striping is often known as RAID-0. In very large setups, these two techniques may be combined to create RAID-10 disk arrays, but this would probably be pertinent only on larger desktop computers with many hard drive bays. | + | In these cases, would it not make more sense to use several drives to extend the total amount of space available, while retaining just a single copy of each file. In essence, we are trading some data security for available space. In the same way, the concept of mirroring deals with combining several hard drives into a single unit in which files are stored in two copies, striping is the analogous concept of combining several drives into one unit with a larger capacity but without duplication. In technical terms, where mirroring is often spoken of as RAID-1, striping is often known as RAID-0. In very large setups, these two techniques may be combined to create RAID-10 disk arrays, but this would probably be pertinent only on larger desktop computers with many hard drive bays.** |
+ | |||
+ | Pourquoi étendre un disque existant (striping) au lieu de le mettre en miroir ? | ||
+ | |||
+ | Pour certaines personnes, la mise en miroir des données peut même ne pas être nécessaire pour le flux de travail de l' | ||
+ | |||
+ | Dans ces cas, ne serait-il pas plus judicieux d' | ||
- | With some of the technologies used, striping may allow an existing disk drive to be extended – in some cases, even while it is mounted and working -- using a second drive. | + | **With some of the technologies used, striping may allow an existing disk drive to be extended – in some cases, even while it is mounted and working -- using a second drive. |
What about the file system? | What about the file system? | ||
- | To my knowledge, there are at least four different ways of setting up either mirroring or striping on a personal computer. The first includes the use of dedicated physical hardware (thus, logically called “hardware RAID”) to which the two disks are plugged in, and which manages the complete set and presents it as a single unit to the user. However, issues include difficulties in using such a system on a laptop, the fact that not all desktop motherboards include this feature, and the hassle of using proprietary software (that often is not available under Linux) to recuperate data in the event of a problem. Needless to say, I would not advise using this way of going about it in a domestic or small business setting. It should also be noted that such setups may or may not allow “hot-plugging” new drives and associating the new disks to existing arrays. Some reading may be necessary if looking into using such a system. | + | To my knowledge, there are at least four different ways of setting up either mirroring or striping on a personal computer. The first includes the use of dedicated physical hardware (thus, logically called “hardware RAID”) to which the two disks are plugged in, and which manages the complete set and presents it as a single unit to the user. However, issues include difficulties in using such a system on a laptop, the fact that not all desktop motherboards include this feature, and the hassle of using proprietary software (that often is not available under Linux) to recuperate data in the event of a problem. Needless to say, I would not advise using this way of going about it in a domestic or small business setting. It should also be noted that such setups may or may not allow “hot-plugging” new drives and associating the new disks to existing arrays. Some reading may be necessary if looking into using such a system.** |
- | The second way would be to set up a RAID array using the Linux kernel’s built-in md (multiple-disk) subsystem. Utilities such as mdadm give us access to the system, which is quite mature and works well. But the interface and commands are not always user-friendly, | + | Avec certaines des technologies utilisées, le striping peut permettre d' |
+ | |||
+ | Qu'en est-il du système de fichiers ? | ||
+ | |||
+ | À ma connaissance, | ||
+ | |||
+ | |||
+ | **The second way would be to set up a RAID array using the Linux kernel’s built-in md (multiple-disk) subsystem. Utilities such as mdadm give us access to the system, which is quite mature and works well. But the interface and commands are not always user-friendly, | ||
Mirroring or striping with BTRFS | Mirroring or striping with BTRFS | ||
- | This leaves us with two systems that are not based on managing individual disk drives, but whole file systems. In FCM#94, I wrote about the B-tree File System (BTRFS) and some of its applications. This file system has evolved even further since then, to the point that I consider it mature and use it on all my own computers (yes, I do eat my own dog food!). More to the point, Ubuntu’s installer, and that of most other modern Linux distributions, | + | This leaves us with two systems that are not based on managing individual disk drives, but whole file systems. In FCM#94, I wrote about the B-tree File System (BTRFS) and some of its applications. This file system has evolved even further since then, to the point that I consider it mature and use it on all my own computers (yes, I do eat my own dog food!). More to the point, Ubuntu’s installer, and that of most other modern Linux distributions, |
+ | |||
+ | La deuxième méthode consiste à configurer une matrice RAID en utilisant le sous-système md (multiple-disk) intégré au noyau Linux. Des utilitaires tels que mdadm nous donnent accès au système, qui est assez mûr et fonctionne bien. Mais l' | ||
+ | |||
+ | Miroir ou extension avec BTRFS | ||
+ | |||
+ | Il nous reste donc deux systèmes qui ne sont pas basés sur la gestion de disques individuels, | ||
- | In the following examples, I will be using two rotational hard drives connected via USB to demonstrate the principles. In all cases, a single partition of 100 GByte size will be created on each drive. | + | **In the following examples, I will be using two rotational hard drives connected via USB to demonstrate the principles. In all cases, a single partition of 100 GByte size will be created on each drive. |
Boot from the USB live image as usual, install the operating system using ubiquity on the first hard disk. When you get to the hard drive selection, create a partition. In my example, I will be using Kubuntu 20.04, and the disk to install to is /dev/sdc. Please do verify you are using the correct disk -- perhaps twice – better than just once! | Boot from the USB live image as usual, install the operating system using ubiquity on the first hard disk. When you get to the hard drive selection, create a partition. In my example, I will be using Kubuntu 20.04, and the disk to install to is /dev/sdc. Please do verify you are using the correct disk -- perhaps twice – better than just once! | ||
Ligne 48: | Ligne 98: | ||
We then add the second drive’s partition to the existing partition (containing our operating system). This can be done with the computer actually running the system. Supposing our main partition on the first drive with the operating system is /dev/sda1, and the new partition on the second drive is /dev/scd1, we would issue commands such as these: | We then add the second drive’s partition to the existing partition (containing our operating system). This can be done with the computer actually running the system. Supposing our main partition on the first drive with the operating system is /dev/sda1, and the new partition on the second drive is /dev/scd1, we would issue commands such as these: | ||
+ | |||
+ | $ sudo bash | ||
+ | |||
+ | # btrfs dev add /dev/sdc1 / -f | ||
+ | |||
+ | # btrfs fil show | ||
+ | Label: none uuid: aeb12e81-f5b1-4a48-b80d-d64624867456 | ||
+ | Total devices 2 FS bytes used 8.13GiB | ||
+ | devid 1 size 93.13GiB used 11.02GiB path /dev/sda1 | ||
+ | devid 2 size 97.66GiB used 0.00B path /dev/sdc1** | ||
+ | |||
+ | Dans les exemples suivants, je vais utiliser deux disques durs rotatifs connectés par USB pour démontrer les principes. Dans tous les cas, une seule partition de 100 GB sera créée sur chaque disque. | ||
+ | |||
+ | Démarrez à partir de l' | ||
+ | |||
+ | Démarrez votre nouveau système, comme d' | ||
+ | |||
+ | Nous ajoutons ensuite la partition du second disque à la partition existante (contenant notre système d' | ||
$ sudo bash | $ sudo bash | ||
Ligne 59: | Ligne 127: | ||
devid 2 size 97.66GiB used 0.00B path /dev/sdc1 | devid 2 size 97.66GiB used 0.00B path /dev/sdc1 | ||
- | At this point, the two separate partitions have been combined into a single BTRFS file system. With these two commands, we have a striped unit with a total capacity of about 200 GBytes. If striping (RAID-0) is desired, stop here. | + | |
+ | **At this point, the two separate partitions have been combined into a single BTRFS file system. With these two commands, we have a striped unit with a total capacity of about 200 GBytes. If striping (RAID-0) is desired, stop here. | ||
Mirroring (or RAID-1) has not yet been implemented, | Mirroring (or RAID-1) has not yet been implemented, | ||
Ligne 67: | Ligne 136: | ||
This converts both our data (-dconvert) and filesystem metadata (-mconvert) into RAID-1, effectively writing a second copy of each item on the other disk. Our two partitions should not report a similar amount of space being used. That’s it; we have a mirrored unit and our data will from this point on be effectively duplicated with one copy on each partition. | This converts both our data (-dconvert) and filesystem metadata (-mconvert) into RAID-1, effectively writing a second copy of each item on the other disk. Our two partitions should not report a similar amount of space being used. That’s it; we have a mirrored unit and our data will from this point on be effectively duplicated with one copy on each partition. | ||
- | Please note that both partitions are not quite the exact same size. With BTRFS, this is not an issue for striping. With mirroring, however, some space will be lost with unequal partitions. As a general rule of thumb, the final capacity of our mirrored drive will be about equal to that of the smallest of our two partitions. | + | Please note that both partitions are not quite the exact same size. With BTRFS, this is not an issue for striping. With mirroring, however, some space will be lost with unequal partitions. As a general rule of thumb, the final capacity of our mirrored drive will be about equal to that of the smallest of our two partitions.** |
- | Before continuing with our final option, let’s consider recovery from a failed disk. With BTRFS, if we are booting from a mirrored filesystem, the boot process will not continue at the point at which the Linux kernel mounts the drive. To get past that, we need to boot from another medium – an Ubuntu live image on a USB thumb drive is fine -- and fix the bad BTRFS volume. We have two choices, both of which include removing the offending bad drive. On the one hand, if we have a spare physical drive to plug in, we can physically replace the bad unit and then run the “btrfs replace” command to restore the array to operation. On the other, if we have no physical spares available at the time, we can re-balance the BTRFS volume to single-disk mode, and remove the bad drive using the “btrfs balance” and “btrfs remove” commands. At a later date – better sooner than later -- a second, working disk may be partitioned and added to the system as described above. | + | À ce stade, les deux partitions séparées ont été combinées en un seul système de fichiers BTRFS. Avec ces deux commandes, nous avons une unité stripée avec une capacité totale d' |
+ | |||
+ | La mise en miroir (ou RAID-1) n'a pas encore été implémentée, | ||
+ | |||
+ | # btrfs balance start -dconvert=raid1 -mconvert=raid1 / | ||
+ | |||
+ | Cela convertit à la fois nos données (-dconvert) et les métadonnées du système de fichiers (-mconvert) en RAID-1, en écrivant effectivement une deuxième copie de chaque élément sur l' | ||
+ | |||
+ | Veuillez noter que les deux partitions n'ont pas exactement la même taille. Avec BTRFS, ce n'est pas un problème pour le striping. Avec la mise en miroir, cependant, un peu d' | ||
+ | |||
+ | |||
+ | **Before continuing with our final option, let’s consider recovery from a failed disk. With BTRFS, if we are booting from a mirrored filesystem, the boot process will not continue at the point at which the Linux kernel mounts the drive. To get past that, we need to boot from another medium – an Ubuntu live image on a USB thumb drive is fine -- and fix the bad BTRFS volume. We have two choices, both of which include removing the offending bad drive. On the one hand, if we have a spare physical drive to plug in, we can physically replace the bad unit and then run the “btrfs replace” command to restore the array to operation. On the other, if we have no physical spares available at the time, we can re-balance the BTRFS volume to single-disk mode, and remove the bad drive using the “btrfs balance” and “btrfs remove” commands. At a later date – better sooner than later -- a second, working disk may be partitioned and added to the system as described above. | ||
Mirroring and striping with ZFS | Mirroring and striping with ZFS | ||
- | Although ZFS (at this time, the acronym is given no precise meaning) is an old player in the server market with Sun Microsystems behind it, it took some time for this technology to come to Linux. OpenZFS is gaining traction in the Linux kernel and Ubuntu distribution worlds, though perhaps not as swiftly as some server administrators would like. | + | Although ZFS (at this time, the acronym is given no precise meaning) is an old player in the server market with Sun Microsystems behind it, it took some time for this technology to come to Linux. OpenZFS is gaining traction in the Linux kernel and Ubuntu distribution worlds, though perhaps not as swiftly as some server administrators would like. ** |
+ | |||
+ | Avant de poursuivre avec notre dernière option, examinons la récupération d'un disque défaillant. Avec BTRFS, si nous démarrons à partir d'un système de fichiers miroir, le processus de démarrage ne se poursuivra pas au moment où le noyau Linux monte le disque. Pour contourner ce problème, nous devons démarrer à partir d'un autre support - une image Live d' | ||
+ | |||
+ | Miroir et extension avec ZFS | ||
+ | |||
+ | Bien que ZFS (pour l' | ||
+ | |||
+ | |||
+ | **ZFS file systems share many characteristics with BTRFS, though some details are different. For example, with ZFS, a virtual device (vdev) may be a single write unit, or may implement mirroring or other forms of data duplication -- but it cannot change from one scheme to another as can BTRFS. Thus, when a vdev is created, one must specify from the onset if we want mirroring to happen, or not. A second feature is that ZFS units are usually mounted automatically by the ZFS subsystem, without needing any entries in the system configuration. In brief, file /etc/fstab may contain zero entries on a system that uses only ZFS units. Finally, in current versions of Ubuntu, ZFS drivers are not part of the basic install, but need to be added through various apt packages. | ||
+ | |||
+ | For interested readers, there is a nice writeup by Jim Salter on ArsTechnica. Though server-focussed, | ||
+ | |||
+ | Les systèmes de fichiers ZFS partagent de nombreuses caractéristiques avec BTRFS, bien que certains détails soient différents. Par exemple, avec ZFS, un périphérique virtuel (vdev) peut être une unité d' | ||
- | ZFS file systems share many characteristics with BTRFS, though some details are different. For example, with ZFS, a virtual device | + | Pour les lecteurs intéressés, il y a un bon article de Jim Salter sur ArsTechnica. Bien que centré sur les serveurs, il devrait vous donner une bonne présentation initiale pour comprendre pleinement les forces spécifiques |
- | For interested readers, there is a nice writeup by Jim Salter on ArsTechnica. Though server-focussed, | ||
- | On Debian-based distributions such as Ubuntu, things have come on to the point that there has been talk of offering to install recent versions on a root ZFS file system, at least as a beta feature. As a matter of personal preference, for the time being I am steering away from this option, at least until ZFS tools mature slightly (they are now shown to be at version 0.8.4) and are included out-of-the-box on all Ubuntu and derivatives’ live boot media. The reasoning here is that, while ZFS does have some very nifty features, the path to recovering data from a crashed boot volume – and please note we are speaking only of a boot volume -- is a tad complex for regular users. For this reason, in the following example I will be working with: | + | **On Debian-based distributions such as Ubuntu, things have come on to the point that there has been talk of offering to install recent versions on a root ZFS file system, at least as a beta feature. As a matter of personal preference, for the time being I am steering away from this option, at least until ZFS tools mature slightly (they are now shown to be at version 0.8.4) and are included out-of-the-box on all Ubuntu and derivatives’ live boot media. The reasoning here is that, while ZFS does have some very nifty features, the path to recovering data from a crashed boot volume – and please note we are speaking only of a boot volume -- is a tad complex for regular users. For this reason, in the following example I will be working with: |
• A root file system on /dev/sda1 (first partition of our main hard drive), with an existing Kubuntu 20.04 installation. This is actually the very same BTRFS unit used previously. Which actual filesystem is used is not important, any familiar options such as ext4 or btrfs should work well. | • A root file system on /dev/sda1 (first partition of our main hard drive), with an existing Kubuntu 20.04 installation. This is actually the very same BTRFS unit used previously. Which actual filesystem is used is not important, any familiar options such as ext4 or btrfs should work well. | ||
Ligne 86: | Ligne 178: | ||
/ | / | ||
- | • Two partitions, /dev/sda2 and /dev/sdc1. These will be combined in a ZFS pool to hold user data. | + | • Two partitions, /dev/sda2 and /dev/sdc1. These will be combined in a ZFS pool to hold user data.** |
- | The purpose of the root file system is just to hold our operating system, and to do so separately from user data. This means it can be formatted easily without affecting user data, or even replaced by a partition on another disk (or even a live media) at a pinch. The ZFS pool will be mounted on /home, to hold user data. | + | Sur les distributions basées sur Debian telles qu' |
+ | ••Un système de fichiers racine sur /dev/sda1 (première partition de notre disque dur principal), avec une installation Kubuntu 20.04 existante. Il s'agit en fait de la même unité BTRFS utilisée précédemment. Le système de fichiers utilisé n'est pas important, toutes les options familières telles que ext4 ou btrfs devraient fonctionner. | ||
+ | |||
+ | # df -h | ||
+ | Filesystem | ||
+ | / | ||
+ | |||
+ | ••Deux partitions, /dev/sda2 et /dev/sdc1. Elles seront combinées dans un pool ZFS pour contenir les données de l' | ||
+ | |||
+ | |||
+ | **The purpose of the root file system is just to hold our operating system, and to do so separately from user data. This means it can be formatted easily without affecting user data, or even replaced by a partition on another disk (or even a live media) at a pinch. The ZFS pool will be mounted on /home, to hold user data. | ||
Before beginning, let us install the required packages. BTRFS is integrated into current Linux kernels, but ZFS is not. So, we will need: | Before beginning, let us install the required packages. BTRFS is integrated into current Linux kernels, but ZFS is not. So, we will need: | ||
Ligne 98: | Ligne 200: | ||
Now, let us create a zpool that uses striping to combine the two partitions. It is traditional to call this zpool the tank, but feel free to choose your own name. | Now, let us create a zpool that uses striping to combine the two partitions. It is traditional to call this zpool the tank, but feel free to choose your own name. | ||
+ | |||
+ | # zpool create tank /dev/sda2 /dev/sdc1 -f | ||
+ | # df -h | ||
+ | Filesystem | ||
+ | / | ||
+ | tank 756G 128K 756G 1% /tank** | ||
+ | |||
+ | Le but du système de fichiers racine est simplement de contenir notre système d' | ||
+ | |||
+ | Avant de commencer, installons les paquets nécessaires. BTRFS est intégré dans les noyaux Linux actuels, mais pas ZFS. Nous aurons donc besoin de : | ||
+ | |||
+ | # apt update ; apt install zfs-dkms zfsutils | ||
+ | |||
+ | Comme précédemment, | ||
+ | |||
+ | Maintenant, créons un zpool qui utilise le striping pour combiner les deux partitions. Il est traditionnel d' | ||
# zpool create tank /dev/sda2 /dev/sdc1 -f | # zpool create tank /dev/sda2 /dev/sdc1 -f | ||
Ligne 105: | Ligne 223: | ||
tank 756G 128K 756G 1% /tank | tank 756G 128K 756G 1% /tank | ||
- | Each partition is of about 400 GBytes, but in combination they yield 756 GBytes. Note that the new zpool has automatically been mounted as / | + | |
+ | **Each partition is of about 400 GBytes, but in combination they yield 756 GBytes. Note that the new zpool has automatically been mounted as / | ||
Within the zpool, we will now create a filesystem (vdev) called home, which may expand to use up all the space available. | Within the zpool, we will now create a filesystem (vdev) called home, which may expand to use up all the space available. | ||
Ligne 119: | Ligne 238: | ||
We now need to transfer over the contents of our /home directory, since this is where we will be mounting our new filesystem. | We now need to transfer over the contents of our /home directory, since this is where we will be mounting our new filesystem. | ||
+ | |||
+ | # mv /home/* / | ||
+ | |||
+ | Chaque partition est d' | ||
+ | |||
+ | Dans le zpool, nous allons maintenant créer un système de fichiers (vdev) appelé home, qui peut s' | ||
+ | |||
+ | # zfs create tank/home | ||
+ | # df -h | ||
+ | Filesystem | ||
+ | / | ||
+ | tank 756G 128K 756G 1% /tank | ||
+ | tank/ | ||
+ | |||
+ | Le nouveau système de fichiers a également été monté automatiquement, | ||
+ | |||
+ | Nous devons maintenant transférer le contenu de notre répertoire /home, puisque c'est là que nous allons monter notre nouveau système de fichiers. | ||
# mv /home/* /tank/home/ | # mv /home/* /tank/home/ | ||
- | We actually need to move the files (and not merely make a copy) since the /home directory will need to be empty. Otherwise, the ZFS filesystem will refuse to mount on a non-empty directory. | + | |
+ | **We actually need to move the files (and not merely make a copy) since the /home directory will need to be empty. Otherwise, the ZFS filesystem will refuse to mount on a non-empty directory. | ||
Finally, we program the new filesystem to mount automatically on /home. | Finally, we program the new filesystem to mount automatically on /home. | ||
Ligne 141: | Ligne 278: | ||
tank 756G | tank 756G | ||
- | As can be seen, the new ZFS vdev is now mounted in its place on /home, with the two striped volumes giving us the expected combined space. | + | As can be seen, the new ZFS vdev is now mounted in its place on /home, with the two striped volumes giving us the expected combined space.** |
- | With ZFS, to create a mirrored zpool instead of striped, the appropriate syntax would simply have been: | + | Nous devons en fait déplacer les fichiers (et pas seulement en faire une copie) puisque le répertoire /home devra être vide. Sinon, le système de fichiers ZFS refusera de monter sur un répertoire non vide. |
+ | |||
+ | Enfin, nous programmons le nouveau système de fichiers pour qu'il se monte automatiquement sur /home. | ||
+ | |||
+ | # zfs set mountpoint=/ | ||
+ | |||
+ | Maintenant, redémarrez. Le système devrait se réveiller avec tank/home monté sur /home : | ||
+ | |||
+ | $ mount | ||
+ | /dev/sda1 on / type ext4 (rw, | ||
+ | tank/home on /home type zfs (rw, | ||
+ | tank on /tank type zfs (rw, | ||
+ | |||
+ | $ df -h | ||
+ | Filesystem | ||
+ | / | ||
+ | tank/ | ||
+ | tank 756G | ||
+ | |||
+ | Comme on peut le voir, le nouveau vdev ZFS est maintenant monté à sa place sur /home, avec les deux volumes stripés nous donnant l' | ||
+ | |||
+ | |||
+ | **With ZFS, to create a mirrored zpool instead of striped, the appropriate syntax would simply have been: | ||
# zpool create tank mirror /dev/sda2 /dev/sdc1 -f | # zpool create tank mirror /dev/sda2 /dev/sdc1 -f | ||
Ligne 153: | Ligne 312: | ||
A decade ago, mirroring and striping were options that would have seemed exotic, or quite frankly difficult to use, to many users. Even today, systems administrators can have cause to rely on dedicated hardware to set up such arrays. However, the spread of BTRFS and ZFS gives regular users – albeit those with a rather technical bent -- the option of creating mirrored or striped units using only software that is available with the Linux kernel. | A decade ago, mirroring and striping were options that would have seemed exotic, or quite frankly difficult to use, to many users. Even today, systems administrators can have cause to rely on dedicated hardware to set up such arrays. However, the spread of BTRFS and ZFS gives regular users – albeit those with a rather technical bent -- the option of creating mirrored or striped units using only software that is available with the Linux kernel. | ||
- | Both technical solutions have their proponents, with some very good reasons on either side -- and a tendency to be quite vocal about it at times. Since our purpose is not to provoke an all-out flame war, let us merely suggest that BTRFS has perhaps a tad more flexibility as regards converting striped units into mirrored and vice-versa, can more easily extend an existing unit – even on the fly in a working system -- and is also slightly better integrated into standard kernels. Setting up a mirrored root partition with BTRFS is also marginally easier than with ZFS. On the other hand, the strong points of ZFS include more flexibility setting up and mounting both zpools and vdevs in various configurations, | + | Both technical solutions have their proponents, with some very good reasons on either side -- and a tendency to be quite vocal about it at times. Since our purpose is not to provoke an all-out flame war, let us merely suggest that BTRFS has perhaps a tad more flexibility as regards converting striped units into mirrored and vice-versa, can more easily extend an existing unit – even on the fly in a working system -- and is also slightly better integrated into standard kernels. Setting up a mirrored root partition with BTRFS is also marginally easier than with ZFS. On the other hand, the strong points of ZFS include more flexibility setting up and mounting both zpools and vdevs in various configurations, |
+ | |||
+ | Avec ZFS, pour créer un zpool en mirroring plutôt qu'en striping, la syntaxe appropriée aurait simplement été : | ||
+ | |||
+ | # zpool create tank mirror /dev/sda2 /dev/sdc1 -f | ||
+ | |||
+ | Veuillez noter le mot clé « mirror ». Les autres commandes seraient les mêmes dans les deux cas, en particulier la création du système de fichiers ZFS (vdev) dans le zpool. | ||
+ | |||
+ | Quelques réflexions finales | ||
+ | |||
+ | Il y a dix ans, le mirroring et le striping étaient des options qui auraient semblé exotiques, ou franchement difficiles à utiliser, à de nombreux utilisateurs. Même aujourd' | ||
+ | |||
+ | Les deux solutions techniques ont leurs partisans, avec de très bonnes raisons de part et d' | ||
+ | |||
+ | |||
+ | **Additionally, | ||
+ | |||
+ | All this opens interesting perspectives for the interested student of systems management -- or for the advanced user. However, we would suggest to begin by testing out either or both systems described here on a spare computer, not using them as your main system (i.e. on data you depend upon) until you feel thoroughly at home with their syntax and quirks.** | ||
- | Additionally, both ZFS and BTRFS feature snapshots, which facilitates rolling back system changes in the event of a disaster, which is perhaps better integrated into the GRUB bootloader under OpenSuSE. | + | De plus, ZFS et BTRFS proposent tous deux des instantanés, ce qui facilite le retour en arrière des modifications du système en cas de catastrophe, ce qui est peut-être mieux intégré dans le chargeur de démarrage |
- | All this opens interesting | + | Tout cela ouvre des perspectives |
issue189/disques_durs.1675020687.txt.gz · Dernière modification : 2023/01/29 20:31 de d52fr