Edit page in Livemark
(2024-12-13 12:49)

AWS Schemes

Frictionless supports reading data from a AWS cloud source. You can read files in any format that is available in your S3 bucket.

pip install frictionless[aws]
pip install 'frictionless[aws]' # for zsh shell

Reading Data

You can read from this source using Package/Resource, for example:

from pprint import pprint
from frictionless import Resource

resource = Resource(path='s3://bucket/table.csv')
pprint(resource.read_rows())

For reading from a private bucket you need to setup AWS creadentials as it's described in the Boto3 documentation.

Writing Data

A similiar approach can be used for writing:

from frictionless import Resource

resource = Resource(path='data/table.csv')
resource.write('s3://bucket/table.csv')

Configuration

There is a Control to configure how Frictionless read files in this storage. For example:

from frictionless import Resource
from frictionless.plugins.s3 import S3Control

resource = Resource(data=[['id', 'name'], [1, 'english'], [2, 'german']])
resource.write('table.new.csv', control=controls.S3Control(endpoint_url='<url>'))

Reference

schemes.AwsControl (class)

schemes.AwsControl (class)

Aws control representation

Signature

(*, name: Optional[str] = None, title: Optional[str] = None, description: Optional[str] = None, s3_endpoint_url: str = https://s3.amazonaws.com) -> None

Parameters

  • name (Optional[str])
  • title (Optional[str])
  • description (Optional[str])
  • s3_endpoint_url (str)

schemes.awsControl.s3_endpoint_url (property)

Signature

str