Ceph and RADOS file storage architecture

Paper: Ceph: A scalable, high-performance distributed file system
Authors: Sage A. Weil

Liked:
• This file storage architecture presents a trade off between performance and consistency. For applications which frequently does small read and write on data, strict consistency and synchronous I/O can impose high latency penalty which can have adverse effect on performance.
• Use of CRUSH simplifies problem of “data location and data distribution.” It also helps in keeping load of system balanced.
• This architecture is scalable as elimination of allocation list makes metadata simple. Also I/O operations are performed separately.

Disliked:
• Theoretically performance of system will go down as load on system will increase. As number of replicas increases in high load scenarios, more consistency overhead will occur. Due to this large internal resources will also be consumed.
• Failure conditions are assumed to be a norm rather than exception which is against Lampson’s hint of “Treating exception and worst case scenarios differently”.
• The separation of data in Metadata and data can cause two-round trip for a data access. Here one request fetches the capability ticket to file object and subsequent trip can lead to data read/ write. This may cause overhead though caching of capability ticket can reduce that a little. Also no method is defined to recover MDS though failure is rare event.
• CRUSH seems to be a deterministic function though it is called pseudo random. Paper has not clearly mentioned how CRUSH function takes care of dynamic replication? How new replicated objects will be accessed by client. It seems all read will be directed to primary replica only. In this how architecture is leveraging replication? Scalability is limited by choice of degree of coherence.

details will be provided later.. right now i dont have any time