FILE

Description​

The File table-valued-function (tvf) is a wrapper around table functions like S3, HDFS, and LOCAL, providing a unified interface to access file contents on different storage systems.

This function is supported since version 3.1.0.

Syntax​

FILE(
 {StorageProperties},
 {FileFormatProperties}
)

• {StorageProperties}

  The StorageProperties section is used to fill in connection and authentication information related to the storage system. For details, please refer to the [Supported Storage Systems] section.

• {FileFormatProperties}

  The FileFormatProperties section is used to fill in properties related to file formats, such as CSV delimiters. For details, please refer to the [Supported File Formats] section.

Supported Storage Systems​

• hdfs

• aws s3

• google cloud storage

• Alibaba Cloud OSS

• Tencent Cloud COS

• Huawei Cloud OBS

• MINIO

Supported File Formats​

• Parquet

• ORC

• Text/CSV/JSON

Examples​

Accessing S3 Storage​

select*fromfile(
"fs.s3.support"="true",
"uri"="s3://bucket/file.csv",
"s3.access_key"="ak",
"s3.secret_key"="sk",
"s3.endpoint"="endpoint",
"s3.region"="region",
"format"="csv"
);

Accessing HDFS Storage​

select*fromfile(
"fs.hdfs.support"="true",
"uri"="hdfs://path/to/file.csv",
"fs.defaultFS"="hdfs://localhost:9000",
"hadoop.username"="doris",
"format"="csv"
);

Accessing Local Storage​

select*fromfile(
"fs.local.support"="true",
"file_path"="student.csv",
"backend_id"="10003",
"format"="csv"
);

Using desc function to View Table Structure​

descfunctionfile(
"fs.s3.support"="true",
"uri"="s3://bucket/file.csv",
"s3.access_key"="ak",
"s3.secret_key"="sk",
"s3.endpoint"="endpoint",
"s3.region"="region",
"format"="csv"
);