hdfs

Star

Here are 1,025 public repositories matching this topic...

Language: All

Filter by language

All 1,025 Java 250 Python 191 Jupyter Notebook 108 Shell 97 Scala 78 Go 24 JavaScript 23 HTML 16 C++ 15 Dockerfile 12

Sort: Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

seaweedfs

seaweedfs / seaweedfs

Star 25.7k

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, xDC replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding. Enterprise version is at seaweedfs.com.

kubernetes distributed-systems fuse replication cloud-drive s3 posix s3-storage hdfs distributed-storage distributed-file-system erasure-coding object-storage blob-storage seaweedfs hadoop-hdfs tiered-file-system

Updated Sep 2, 2025
Go

heibaiying / BigData-Notes

Star 16.6k

大数据入门指南 ⭐

phoenix scala kafka big-data spark yarn hive hadoop storm bigdata hbase zookeeper hdfs mapreduce flume azkaban sqoop

Updated Jan 5, 2024
Java

ceph

ceph / ceph

Star 15.4k

Ceph is a distributed object, block, and file storage platform

kubernetes fuse replication storage high-performance smb s3 cloud-storage nfs posix object-store hdfs block-storage distributed-storage iscsi software-defined-storage distributed-file-system erasure-coding highly-available nvme-over-fabrics

Updated Sep 1, 2025
C++

juicefs

juicedata / juicefs

Star 12.1k

JuiceFS is a distributed POSIX file system built on top of Redis and S3.

go redis golang distributed-systems storage filesystem bigdata s3 posix hdfs cloud-native object-storage

Updated Sep 1, 2025
Go

wangzhiwubigdata / God-Of-BigData

Star 10.3k

专注大数据学习面试,大数据成神之路开启。Flink/Spark/Hadoop/Hbase/Hive...

kafka spark hive hadoop bigdata hbase zookeeper hdfs flume flink azkaban

Updated Aug 7, 2023

piskvorky / smart_open

Sponsor

Star 3.4k

Utils for streaming large files (S3, HDFS, gzip, bz2...)

python streaming s3 file hdfs hacktoberfest webhdfs boto gzip-stream bz2 streaming-data

Updated Sep 1, 2025
Python

TileDB-Inc / TileDB

Star 2k

The Universal Storage Engine

data-science storage-engine s3 sparse-data scientific-computing s3-storage arrays hdfs data-analysis dataframes tiledb dense-data sparse-arrays

Updated Aug 27, 2025
C++

collabH / bigdata-growth

Star 1.7k

大数据知识仓库涉及到数据仓库建模、实时计算、大数据、数据中台、系统设计、Java、算法等。

kafka spark hive hadoop bigdata kudu hbase olap hdfs mapreduce flink debezium bigdatalearning hudi

Updated Aug 20, 2025
Shell

water8394 / BigData-Interview

Star 1.6k

🎯 🌟[大数据面试题]分享自己在网络上收集的大数据相关的面试题以及自己的答案总结.目前包含Hadoop/Hive/Spark/Flink/Hbase/Kafka/Zookeeper框架的面试题知识总结

kafka spark yarn hadoop bigdata interview hbase hdfs mapreduce interview-questions flink

Updated Aug 30, 2021

colinmarc / hdfs

Star 1.4k

A native go client for HDFS

go commandline hdfs

Updated Jan 22, 2025
Go

Addax

wgzhao / Addax

Star 1.3k

A fast and versatile ETL tool that can transfer data between RDBMS and NoSQL seamlessly

mysql database influxdb hive hadoop etl clickhouse excel kudu impala postgresql oracle hdfs sqlserver trino

Updated Sep 2, 2025
Java

spotify / snakebite

Star 857

A pure python HDFS client

python hdfs python-hdfs-client

Updated Apr 19, 2022
Python

HariSekhon / DevOps-Python-tools

Star 806

80+ DevOps & Data CLI Tools - AWS, GCP, GCF Python Cloud Functions, Log Anonymizer, Spark, Hadoop, HBase, Hive, Impala, Linux, Docker, Spark Data Converters & Validators (Avro/Parquet/JSON/CSV/INI/XML/YAML), Travis CI, AWS CloudFormation, Elasticsearch, Solr etc.

python linux docker aws elasticsearch devops json cloudformation spark hadoop avro travis-ci solr gcp hbase pyspark hdfs parquet dockerhub gcf

Updated Apr 25, 2025
Python

sunnyandgood / BigData

Star 682

💎🔥大数据学习笔记

mysql linux shell hive hadoop hbase zookeeper hdfs mapreduce flume sqoop

Updated May 13, 2019
Java

Stratio / sparta

Star 528

Real Time Analytics and Data Pipelines based on Spark Streaming

workflow lambda streaming real-time scala kafka spark analytics spark-streaming olap hdfs sparksql triggers streaming-data stratio sparta stratio-sparta

Updated Oct 24, 2019
Scala

lensesio / kafka-connect-ui

Star 515

Deprecated - See Lenses.io Community Edition

redis mqtt elasticsearch twitter kafka cassandra influxdb jms s3 documentdb hdfs kafka-connect

Updated May 7, 2025
JavaScript

dromara / CloudEon

Sponsor

Star 476

CloudEon uses Kubernetes to install and deploy open-source big data components, enabling the containerized operation of an open-source big data platform. This allows you to reduce your focus on underlying resource management and maintenance.

kubernetes yarn hadoop bigdata hdfs cloudnative doris