Create an external stage with your own S3 bucket and credentials where your TSV files are stored.
CREATE STAGE tsv_query_stage
URL = 's3://load/tsv/'
CONNECTION = (
ACCESS_KEY_ID = '<your-access-key-id>'
SECRET_ACCESS_KEY = '<your-secret-access-key>'
);
Step 2. Create Custom TSV File Format
CREATE FILE FORMAT tsv_query_format
TYPE = TSV,
RECORD_DELIMITER = '\n',
FIELD_DELIMITER = ',',
COMPRESSION = AUTO;
SELECT $1, $2, $3
FROM @tsv_query_stage
(
FILE_FORMAT => 'tsv_query_format',
PATTERN => '.*[.]tsv'
);
If the TSV files is compressed with gzip, we can use the following query:
SELECT $1, $2, $3
FROM @tsv_query_stage
(
FILE_FORMAT => 'tsv_query_format',
PATTERN => '.*[.]tsv[.]gz'
);
Query with Metadata
Query TSV files directly from a stage, including metadata columns like METADATA$FILENAME
and METADATA$FILE_ROW_NUMBER
:
SELECT
METADATA$FILENAME,
METADATA$FILE_ROW_NUMBER,
$1, $2, $3
FROM @tsv_query_stage
(
FILE_FORMAT => 'tsv_query_format',
PATTERN => '.*[.]tsv'
);
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4