Discussion :

[star]

[Oracle10gR2-AIX5.3] Baie de stokage Externe en RAID

Je consulte actuellement pour un serveur IBM System p5 550 (ou Bull Escala PL450) avec une baie de stockage externe.
Pour le stockage j'ai le choix entre un EMC Clariion CX3 20 ou un IBM DS4700 72.
Là ou je m'interroge c'est sur le niveau de RAID à mettre en oeuvre sur la baie pour une utilisation optimale avec Oracle Database 10g R2.
J'hésite entre RAID 10 ou RAID 5.
En fait, par ma connaissance très théorique du RAID j'ai cru comprendre que pour une BD Oracle le RAID 10 est celui recommandé par Oracle même.
Cependant, un de mes fournisseurs me conseille de faire du RAID 5 car pour lui (il s'agit d'un constructeur tout de même !) le gain de performance du RAID 10 obtenu avec la baie de stockage n'en vaut pas la chandelle, et qu'il serait plus logique de tirer partie en premier de la capacité de stockage du RAID 5 avec le même nombre de disques (8 disques FC 4Gb 15Krpm + 1 hotspare).
Ce que je sais de manière théorique c'est qu'il faut faire attention à la taille du stripe (bande) et qu'il faut aussi tenir compte du temps de reconstruction du RAID pour faire le choix.
Bon !
En fait, j'aimerai connaitre vos avis à ce niveau, car le choix est pour moi important pour la suite. si cela pouvait me faire gagner du temps, j'en serais satisfait. Autrement, j'aurais plus qu'à tout casser et tout refaire s'il faut.
Mais comme on dit souvent : Le temps c'est de l'argent et en perdre c'est vraiment dommage !
Merci d'avance pour vos précieux conseils !
.

[orafrance]

J'ai jamais rencontré de problème avec le RAID 5 et il faut bien admettre que l'économie réalisée est non négligeable. Le seul incident que j'ai eu c'est lors de l'activation du mirroring alors que la perf en écriture n'étaient déjà pas au top

[orafrance]

pour info : Configuration optimale du stockage

[star]

Merci Fred_D pour le lien "Configuration optimale du stockage" qui est d'une grande aide en effet.
Cependant, vos retours d'expérience me seraient encore plus profitable.
La théorie est une chose et la pratique en est une autre !
Je laisse mon post actif pour le moment, avec l'espoir d'être non seulement lu mais aussi conseillé par vous tous.
.

[star]

J'ai trouvé sur le site d'IBM un RedBook sur l'IBM System Storage DS4700 70 -> "DS4000 Series, Storage Manager and Copy Services" qui présente le DS4700, version Draft.
http://www.redbooks.ibm.com/redpiece.../sg247010.html
On y parle de RAID et ce qui est intérêssant, ceux sont les références sur l'utilisation de la baie avec une base de données (recherche avec mot clé database).
Exemple :

The write caching parameter enables the storage subsystem to cache write data instead of writing it directly to the disks. This can improve performance significantly, especially for environments with random writes such as databases.

.

[Jaouad]

Pour Oracle : je conseille en régle génèrale du 1+0 mais jamais de Raid 5 , a cause du calcul du bit de parité lors de l'écriture

[star]

Dans le même ordre d'idée, j'irai encore plus loin !
Il serait aussi beaucoup plus performant de distribuer la charge E/S en mettant en oeuvre le load balancing sur les chemins multiples entre le serveur et la baie de stockage, et entre les 2 contrôleurs de la baie. Il s'agirait de répartir les unités logiques de manière équilibrée sur les 2 contrôleurs et d'exploiter les chemins multiples entre le serveur et chaque contrôleur. Les requêtes d'E/S seraient réparties sur les différents chemins entre le serveur et chacun des 2 contrôleurs.



RedBook sur l'IBM System Storage DS4700 -> "DS4000 Series, Storage Manager and Copy Services"

[Page 102]
It is important to match up host or fabric connections to the DS4700 by attaching one connection to each controller. In doing so, you take advantage of the DS4700’s ability to fail over and distribute the workload among the two controllers.

[Page 129]
Balancing the workload allows you to increase bandwidth to fulfill the most demanding requirements.

[Page 177]
Round-robin (load distribution or load balancing) is used when the RDAC driver discovers that there are multiple data paths from the host to an individual controller. In such a configuration, it is assumed that no penalty is incurred for path switches that do not result in a controller ownership change, thereby enabling the multi-path driver to exploit redundant I/O path bandwidth by distributing (in a round-robin fashion) I/O requests across paths to an individual controller.

[Page 198]
To balance the workload between the controllers, you can change the preferred ownership of a logical drive to the other controller, and normally the Storage Subsystem is balanced better regarding the workload.

[Page 198]
If a particular controller has considerably more I/O activity, consider moving logical drives to the other controller in the Storage Subsystem.

[Page 199]
You may notice a disparity in the total I/Os (workload) of controllers. For example, the workload of one controller is heavy or is increasing over time, while that of the other controller is lighter or more stable. In this case, consider changing the controller ownership of one or more logical drives to the controller with the lighter workload.
Tip: Here are some guidelines for LUN assignment and storage partitioning:
Assign LUNs across all controllers to balance controller utilization.
If you have highly used LUNs, where possible, move them away from other LUNs and put them on their own separate array. This will reduce disk contention for that array.

[Page 269]
The preferred controller handles the logical drive normally if both controllers and I/O paths are online. You can distribute your logical drives between both controllers to provide better load balancing between them. The default is to alternate the logical drives on the two controllers.

[Page 198]
Each logical drive has a preferred controller of ownership. This controller normally handles all I/O requests for this particular logical drive. In other words, each logical drive is owned by one and only one controller. The alternate controller only takes over and handles the I/O requests in the case of a failure along the I/O path, for example, a defect host bus adapter or switch. When defining logical drives, the system normally alternates ownership between the two controllers.

RedBook sur l'IBM System Storage DS4700 -> "DS4000 Best Practices and Performance Tuning Guide"

[Page 143]
However, the transaction logs of a database application have a much high write ratio, and as such perform better in a different RAID array. This reason also adds to the need to place the logs on a separate logical drive which for best performance should be located on a different array that is defined to better support the heavy write need.
Best Practice: Database tablespaces, journals and logs should never be co-located on the same logical drive or RAID array.

[Page 143]
As mentioned earlier, in a database environment we actually have two separate environments with the tablespaces, and the journals and logs. As the tablespaces are normally high reads and low writes, and the journals and logs are high writes with low reads. This environment is best served by two different RAID types.

[Page 152]
Best Practice: The differences described above outline the major reasons for our recommendation to keep the journals and logs on different arrays than the tablespaces for database applications.

[Page 153]
For large size databases consider using the host volume management software to build the database volume to be used for the application. Build the volume across sets of logical drives laid out per the RAID type discussion above. In using multiple arrays you will also be able to increase the controllers which are involved in handling the load therefore getting full use of the storage servers resources.

[Page 54]
However, when assigning logical volumes to the systems, it is very important to remember that the DS4000 Storage Server uses a preferred controller ownership approach for communicating with LUNs. This means that every LUN is owned by only one controller. It is, therefore, important at the system level to make sure that traffic is correctly balanced among controllers.

Petit Bémol sur le load balancing ou load distribution (bien dommage d'ailleurs !!!)

[Page 40]
In a single server environment, AIX is the other OS that allows load sharing (also called load balancing). The best practice is not to use load balancing in AIX as it can have performance issues and cause disk thrashing.
Best Practice: Do not enable load balancing for AIX.

[Page 76]
Note: Do not enable load balancing for AIX.

Si cela peut servir à tous !
Merci
.

[star]

Avant, d'en revenir au RAID avec les bases Oracle, j'aimerai discuter du cashing avec Oracle et vous poser une question la-dessus.


RedBook sur l'IBM System Storage DS4700 -> "DS4000 Series, Storage Manager and Copy Services"

[Page 202]
The write caching parameter enables the storage subsystem to cache write data instead of writing it directly to the disks. This can improve performance significantly, especially for environments with random writes such as databases.

[Page 182]
- Use write caching on RAID-1 arrays. Because a RAID-1 write will not complete until both writes have been done (two disks), performance of writes can be improved through the use of a write cache. When using a write cache, be sure it is battery-backed up.

[Page 183]
Use write caching on RAID-5 arrays, because RAID-5 writes will not be completed until at least two reads and two writes have occurred. The response time of writes will be improved through the use of write cache (be sure it is battery-backed up).

[Page 184]
Use RAID-10 when performance is critical. Use write caching on RAID-10. Because RAID-10 write will not be completed until both writes have been done, the performance of the writes can be improved through the use of a write cache (be sure it is battery-backed
up).

[Page 199]
Cache memory is an area of temporary volatile storage (RAM) on the controller that has a faster access time than the drive media. This cache memory is shared for read and write operations. Efficient use of the RAID controller cache is essential for good performance of the DS4000 Storage Server.

[Page 200]
Here is how a read I/O request is handled in this model. A host issues a read I/O request that is sent over a path (such as a Fibre Channel) to the disk system. The request is received by a disk system host adapter. The host adapter checks whether the requested data is already in cache, in which case it is immediately sent back to the host. If the data is not in cache, the
request is forwarded to a disk adapter that reads the data from the appropriate disk and copies the data into cache. The host adapter sends the data from cache to the requesting host.

[Page 200]
Most (hardware) RAID controllers have some form of read and/or write caching. You should obviously plan to take advantage of these caching capabilities, as they enhance the effective I/O capacity of the disk subsystem. The principle of these controller-based caching mechanisms is to gather smaller and potentially nonsequential I/O requests coming in from
the host server (for example, SQL Server) and try to batch them with other I/O requests; consequently, the I/O requests are sent as larger (32 KB to 128 KB), and possibly sequential, requests to the hard disk drives. The RAID controller cache arranges incoming I/O requests by making the best use of the hard disk's underlying I/O processing ability. This increases the disk I/O throughput.

[Page 201]
Read caching allows read operations from the host to be stored in controller cache memory. If a host requests data that is not in the cache, the controller reads the needed data blocks from the disk and places them in the cache. Until he cache is flushed, any other requests for this data are fulfilled with the cache data instead of initiating another read operation to the disk.

[Page 203]
Write caching can increase the performance of write operations. The data is not written straight to the disk drives; it is only written to the cache. From an application perspective, this is much faster than waiting for the disk write operation to complete. Therefore, you can expect a significant gain in application writing performance. It is the responsibility of the cache controller to eventually flush the unwritten cache entries to the disk drives.

RedBook sur l'IBM System Storage DS4700 -> "DS4000 Best Practices and Performance Tuning Guide"

[Page 150]
On the DS4000 Storage Server the cache blocksize is a variable value that can be set to 4K or 16K. The main goal with setting this value is to not waste space. This is a storage server wide parameter, and when set, it is the value to be used by all cache operations. For example, if the IO of greatest interest is that from your database operations during the day rather than your weekly backups, you would want to tune this value to handle the high transactions best. Knowing that the higher transactions will have smaller IO size, using the 4K setting is generally best for transaction intense environments.
Best Practice: Set the cache blocksize to 4K for the DS4000 system normally for transaction intense environments.

[Page 150]
Tip: Throughput operations though impacted by smaller cache blocksize can still perform reasonable if all other efforts have been accounted for. Transaction based operations are normally the higher concern, and therefore should be the focus for setting the server wide values if applicable.

[Page 156]
For write IO in a transaction based environment you can enable write cache, and write cache mirroring for cache protection. This allows the write IOs to be acknowledged even before they are written to disks as the data is in cache, and backed up by the second mirror in the other controller's cache. This improves write performance dramatically; this sequence is
actually a set of two options doing both write caching, and mirroring to the second cache. If you are performing a process that can handle loss of data, and can be restarted, you may chose to disable the mirroring, and see very high write performance.

Que dire du qualificatif "dramatically" du dernier paragraphe !

If you are performing a process that can handle loss of data, and can be restarted, you may chose to disable the mirroring, and see very high write performance.

Je me demande de quel manière une BD Oracle pourrait fonctionner dans ce mode sans corruption des données écrites ?
Je me demande s'il ne suffirait pas d'isoler les redo logs en ligne sur un contrôleur, groupe RAID propre en mode write througth ?
Vos suggestions sont les bienvenue !
Merci
.

[star]

Pour en finir avec le RAID et les bases Oracle :


RedBook sur l'IBM System Storage DS4700 -> "DS4000 Series, Storage Manager and Copy Services"

[Page 181]
RAID-0: For performance, but generally not recommended
RAID-0 is also known as data striping. It is well-suited for program libraries requiring rapid
loading of large tables, or more generally, applications requiring fast access to read-only
data or fast writing. RAID-0 is only designed to increase performance. There is no
redundancy, so any disk failures require reloading from backups. Select RAID level 0 for
applications that would benefit from the increased performance capabilities of this RAID
level. Never use this level for critical applications that require high availability.

[Page 181]
RAID-1: For availability/good read response time
RAID-1 is also known as disk mirroring. It is most suited to applications that require high
data availability, good read response times, and where cost is a secondary issue. The
response time for writes can be somewhat slower than for a single disk, depending on the
write policy. The writes can either be executed in parallel for speed or serially for safety.
Select RAID level 1 for applications with a high percentage of read operations and where
the cost is not the major concern.
Because the data is mirrored, the capacity of the logical drive when assigned RAID level 1
is 50 percent of the array capacity.

[Page 182]
Some recommendations when using RAID-1 include:
– Use RAID-1 for the disks that contain your operating system. It is a good choice,
because the operating system can usually fit on one disk.
– Use RAID-1 for transaction logs. Typically, the database server transaction log can fit
on one disk drive. In addition, the transaction log performs mostly sequential writes.
Only rollback operations cause reads from the transaction logs. Therefore, we can
achieve a high rate of performance by isolating the transaction log on its own RAID-1
array.

[Page 183]
RAID-5: High availability and fewer writes than reads
RAID level 5 stripes data and parity across all drives in the array. RAID level 5 offers both
data protection and increased throughput. When you assign RAID-5 to an array, the
capacity of the array is reduced by the capacity of one drive (for data-parity storage).
RAID-5 gives you higher capacity than RAID-1, but RAID level 1 offers better performance.
RAID-5 is best used in environments requiring high availability and fewer writes than
reads.
RAID-5 is good for multi-user environments, such as database or file system storage,
where typical I/O size is small, and there is a high proportion of read activity. Applications
with a low read percentage (write-intensive) do not perform as well on RAID-5 logical
drives because of the way a controller writes data and redundancy data to the drives in a
RAID-5 array. If there is a low percentage of read activity relative to write activity, consider
changing the RAID level of an array for faster performance.

[Page 184]
RAID-10: Higher performance than RAID-1
RAID-10, also known in as RAID 0+1, implements block interleave data striping and
mirroring. In RAID-10, data is striped across multiple disk drives, and then those drives are
mirrored to another set of drives.
The performance of RAID-10 is approximately the same as RAID-0 for sequential I/Os.
RAID-10 provides an enhanced feature for disk mirroring that stripes data and copies the
data across all the drives of the array. The first stripe is the data stripe; the second stripe is
the mirror (copy) of the first data stripe, but it is shifted over one drive. Because the data is
mirrored, the capacity of the logical drive is 50 percent of the physical capacity of the hard
disk drives in the array.
The recommendations for using RAID-10 are:
Use RAID-10 whenever the array experiences more than 10 percent writes. RAID-5 does
not perform well as RAID-10 with a large number of writes.

[Page 184]
When comparing RAID-10 to RAID-5:
RAID-10 writes a single block through two writes. RAID-5 requires two reads (read original
data and parity) and two writes. Random writes are significantly faster on RAID-10.
RAID-10 rebuilds take less time than RAID-5 rebuilds. If a real disk fails, RAID-10 rebuilds
it by copying all the data on the mirrored disk to a spare. RAID-5 rebuilds a failed disk by
merging the contents of the surviving disks in an array and writing the result to a spare.
RAID-10 is the best fault-tolerant solution in terms of protection and performance, but it
comes at a cost. You must purchase twice the number of disks that are necessary with
RAID-0.

RedBook sur l'IBM System Storage DS4700 -> "DS4000 Best Practices and Performance Tuning Guide"

[Page 152]
Some people feel strongly about database journals and logs also belonging on RAID 1 as
well.


[Page 152]
So with these differences, you can either place the database journals and logs on a RAID 1 or
a RAID 5, depending on how sequential and well fit your data is. Testing of both types of
arrays for your best performance is recommended.
Best Practice: The differences described above outline the major reasons for our
recommendation to keep the journals and logs on different arrays than the tablespaces for
database applications.

Je termine là ce post bien qu'il manque pour être exhaustif le chapitre sur la taille des blocs Oracle, OS, cache et segment.
Merci !
.