Filesystems#

Interface#

FileInfo(path, FileType type=FileType.Unknown)

FileSystem entry info.

FileSelector(base_dir, ...)

File and directory selector.

FileSystem()

Abstract file system API.

Filesystem Implementations#

LocalFileSystem([use_mmap])

A FileSystem implementation accessing files on the local machine.

S3FileSystem([access_key, secret_key, ...])

S3-backed FileSystem implementation

GcsFileSystem(bool anonymous=False, *[, ...])

Google Cloud Storage (GCS) backed FileSystem implementation

HadoopFileSystem(str host, int port=8020, ...)

HDFS backed FileSystem implementation

SubTreeFileSystem(base_path, FileSystem base_fs)

Delegates to another implementation after prepending a fixed base path.

AzureFileSystem(account_name[, account_key, ...])

Azure Blob Storage backed FileSystem implementation

To define filesystems with behavior implemented in Python:

PyFileSystem(handler)

A FileSystem with behavior implemented in Python.

FileSystemHandler()

An abstract class exposing methods to implement PyFileSystem's behavior.

FSSpecHandler(fs)

Handler for fsspec-based Python filesystems.

Utilities#

copy_files(source, destination[, ...])

Copy files between FileSystems.

initialize_s3(...)

Initialize S3 support

finalize_s3()

resolve_s3_region(bucket)

Resolve the S3 region of a bucket.

S3LogLevel(value[, names, module, qualname, ...])