Introduction
What Is Diesel
Diesel is a highly available distributed Object File Store with the ability to scale across datacenters. The intention is for Diesel to implement S3 API meaning you can use any S3 client library against it.
Why Diesel
The inevitable question that comes up is are we just creating something that already exists. To answer this let’s look at some common questions
Are there open source products or libraries that store files as objects that can scale and are distributed? Yes CEPH Gluster MinIO SeaweedFS.
Are they feasible for deployment and use within Docker for Motiv’s MicroServices? Yes though Gluster and CEPH are not in JT’s opinion (Too hard and difficult to support and maintain) MinIO SeaweedFS
For MinIO and SeaweedFS do they support Large Files? Seaweed can but it is kind of an afterthought with their product. It requires the client to manage the chunks. Because of this it falls off for us here at Motiv. In addition to this the scalability of the files across datacenters happens as the file system itself is replicated. Would like to see a more slick solution https://github.com/chrislusf/seaweedfs/wiki/Large-File-Handling MinIO does support large files properly
For MinIO does it scale across datacenters? Global Deploy No. It can replicate for DR but it doesn’t scale datacenters https://github.com/minio/minio/issues/5738
https://docs.min.io/docs/distributed-minio-quickstart-guide.html
Last updated