> disk of 18TB (not need for SSD if you can parallel write)
Do note that you can put, like, at most?, 1TB of hot/warm data on this 18TB drive.
Imagine you do a query, and 100GB of the data to be searched are on 1 HDD. You will wait 500s-1000s just for this hard drive. Imagine a bit higher concurrency with searching on this HDD, like 3 or 5 queries.
You can't fill these drives full with hot or warm data.
> To store 100 petabytes of data by purchasing disks yourself, you would need approximately 5556 18TB hard drives totaling $1,666,800.
You want to have 1000x more drives and only fill 1/1000 of them. Now you can do a parallel read!
Do note that you can put, like, at most?, 1TB of hot/warm data on this 18TB drive.
Imagine you do a query, and 100GB of the data to be searched are on 1 HDD. You will wait 500s-1000s just for this hard drive. Imagine a bit higher concurrency with searching on this HDD, like 3 or 5 queries.
You can't fill these drives full with hot or warm data.
> To store 100 petabytes of data by purchasing disks yourself, you would need approximately 5556 18TB hard drives totaling $1,666,800.
You want to have 1000x more drives and only fill 1/1000 of them. Now you can do a parallel read!
> You would need 3 times that for redundancy
With erasure coding you need less, like 1.4x-2x.