Frictionless supports reading data from a AWS cloud source. You can read files in any format that is available in your S3 bucket.
pip install frictionless[aws] pip install 'frictionless[aws]' # for zsh shell
You can read from this source using
Package/Resource, for example:
from pprint import pprint from frictionless import Resource resource = Resource(path='s3://bucket/table.csv') pprint(resource.read_rows())
For reading from a private bucket you need to setup AWS creadentials as it's described in the Boto3 documentation.
A similiar approach can be used for writing:
from frictionless import Resource resource = Resource(path='data/table.csv') resource.write('s3://bucket/table.csv')
There is a
Control to configure how Frictionless read files in this storage. For example:
from frictionless import Resource from frictionless.plugins.s3 import S3Control resource = Resource(data=[['id', 'name'], [1, 'english'], [2, 'german']]) resource.write('table.new.csv', control=controls.S3Control(endpoint_url='<url>'))
Aws control representation
(*, name: Optional[str] = None, title: Optional[str] = None, description: Optional[str] = None, s3_endpoint_url: str = https://s3.amazonaws.com) -> None