HyDFS : Distributed File System

HyDFS : Distributed File System
GoPythonDistributed Systems
Developed a distributed file system as part of the coursework for CS 425 at the University of Illinois at Urbana-Champaign (UIUC). The project involves creating a system where files are stored across multiple networked computers, allowing them to work together as if they were a single, cohesive storage unit.

Tasks Accomplished

• Designed and implemented a scalable Distributed File System which is a hybrid of Cassandra and HDFS
• Efficient handling of replicas by implementing consistent hashing for data distribution and replication across nodes, ensuring fault tolerance, load balancing, and high availability in the system.
• Implemented SWIM Membership Protocol to achieve efficient failure detection and membership management
• Developed distributed grep command to efficiently lookup logs from various machines from one client