Frictionless supports reading data from a AWS cloud source. You can read files in any format that is available in your S3 bucket.
pip install frictionless[aws]
pip install 'frictionless[aws]' # for zsh shell
You can read from this source using Package/Resource
, for example:
from pprint import pprint
from frictionless import Resource
resource = Resource(path='s3://bucket/table.csv')
pprint(resource.read_rows())
For reading from a private bucket you need to setup AWS creadentials as it's described in the Boto3 documentation.
A similiar approach can be used for writing:
from frictionless import Resource
resource = Resource(path='data/table.csv')
resource.write('s3://bucket/table.csv')
There is a Control
to configure how Frictionless read files in this storage. For example:
from frictionless import Resource
from frictionless.plugins.s3 import S3Control
resource = Resource(data=[['id', 'name'], [1, 'english'], [2, 'german']])
resource.write('table.new.csv', control=controls.S3Control(endpoint_url='<url>'))
Aws control representation
(*, name: Optional[str] = None, title: Optional[str] = None, description: Optional[str] = None, s3_endpoint_url: str = https://s3.amazonaws.com) -> None
str