Skip to main content

S3 Tutorial

This functionality requires an experimental s3 plugin. Read More

Frictionless supports reading data from an S3 cloud source. You can read files in any format that is available in your bucket.

CLI
pip install frictionless[s3]

Reading Data#

You can read from this source using Package/Resource, for example:

Python
from pprint import pprint
from frictionless import Resource
resource = Resource(path='s3://bucket/table.csv')
pprint(resource.read_rows())

For reading from a private bucket you need to setup AWS creadentials as it's described in the Boto3 documentation.

Writing Data#

A similiar approach can be used for writing:

Python
from frictionless import Resource
resource = Resource(path='data/table.csv')
resource.write('s3://bucket/table.csv')

Configuring Data#

There is a Control to configure how Frictionless read files in this storage. For example:

Python
from frictionless import Resource
from frictionless.plugins.s3 import S3Control
resource = Resource(data=[['id', 'name'], [1, 'english'], [2, 'german']])
resource.write('table.new.csv', control=controls.S3Control(endpoint_url='<url>'))

References:

Last updated on by roll