Edit page in Livemark
(2022-09-19 18:33)

Data Actions

Describe

Describe is a high-level function (action) to infer a metadata from a data source.

Example

from frictionless import describe

resource = describe('table.csv')
print(resource)
{'name': 'table',
 'type': 'table',
 'path': 'table.csv',
 'scheme': 'file',
 'format': 'csv',
 'encoding': 'utf-8',
 'mediatype': 'text/csv',
 'schema': {'fields': [{'name': 'id', 'type': 'integer'},
                       {'name': 'name', 'type': 'string'}]}}

Reference

describe (function)

describe (function)

Describe the data source

Signature

(source: Union[Any, NoneType] = None, *, type: Union[str, NoneType] = None, stats: bool = False, **options)

Parameters
  • source (typing.Union[typing.Any, NoneType]): data source
  • type (typing.Union[str, NoneType]): source type - `schema`, `resource` or `package` (default: infer)
  • stats ()
  • options

Extract

Extract is a high-level function (action) to read tabular data from a data source. The output is encoded in 'utf-8' scheme.

Example

from pprint import pprint
from frictionless import extract

rows = extract('table.csv')
pprint(rows)
[{'id': 1, 'name': 'english'}, {'id': 2, 'name': '中国人'}]

Reference

extract (function)

extract (function)

Extract resource rows

Signature

(source: Optional[Any] = None, *, type: Optional[str] = None, dialect: Optional[Union[Dialect, str]] = None, schema: Optional[Union[Schema, str]] = None, limit_rows: Optional[int] = None, process: Optional[IProcessFunction] = None, filter: Optional[IFilterFunction] = None, stream: bool = False, resource_name: Optional[str] = None, **options)

Parameters
  • source (Optional[Any]): data source
  • type (Optional[str]): source type - package of resource (default: infer)
  • dialect (Optional[Union[Dialect, str]])
  • schema (Optional[Union[Schema, str]])
  • limit_rows (Optional[int])
  • process (Optional[IProcessFunction])
  • filter (Optional[IFilterFunction])
  • stream (bool)
  • resource_name (Optional[str])
  • options

Validate

Validate is a high-level function (action) to validate data from a data source.

Example

from frictionless import validate

report = validate('table.csv')
print(report.valid)
True

Reference

validate (function)

validate (function)

Validate resource

Signature

(source: Union[Any, NoneType] = None, *, type: Union[str, NoneType] = None, dialect: Union[frictionless.dialect.dialect.Dialect, str, NoneType] = None, schema: Union[frictionless.schema.schema.Schema, str, NoneType] = None, stats: Union[frictionless.stats.Stats, NoneType] = None, checklist: Union[frictionless.checklist.checklist.Checklist, str, NoneType] = None, checks: List[frictionless.checklist.check.Check] = [], pick_errors: List[str] = [], skip_errors: List[str] = [], resource_name: Union[str, NoneType] = None, limit_errors: int = 1000, limit_rows: Union[int, NoneType] = None, parallel: bool = False, **options)

Parameters
  • source (typing.Union[typing.Any, NoneType]): a data source
  • type (typing.Union[str, NoneType]): source type - inquiry, package, resource, schema or table
  • dialect (typing.Union[frictionless.dialect.dialect.Dialect, str, NoneType])
  • schema (typing.Union[frictionless.schema.schema.Schema, str, NoneType])
  • stats (typing.Union[frictionless.stats.Stats, NoneType])
  • checklist (typing.Union[frictionless.checklist.checklist.Checklist, str, NoneType])
  • checks (typing.List[frictionless.checklist.check.Check])
  • pick_errors (typing.List[str])
  • skip_errors (typing.List[str])
  • resource_name (typing.Union[str, NoneType])
  • limit_errors ()
  • limit_rows (typing.Union[int, NoneType])
  • parallel ()
  • options

Transform

Transform is a high-level function (action) to transform tabular data from a data source.

Example

from frictionless import transform, steps

resource = transform('table.csv', steps=[steps.cell_set(field_name='name', value='new')])
print(resource.read_rows())
[{'id': 1, 'name': 'new'}, {'id': 2, 'name': 'new'}]

Reference

transform (function)

transform (function)

Transform resource

Signature

(source: Union[Any, NoneType] = None, *, type: Union[str, NoneType] = None, pipeline: Union[frictionless.pipeline.pipeline.Pipeline, str, NoneType] = None, steps: Union[List[frictionless.pipeline.step.Step], NoneType] = None, **options)

Parameters
  • source (typing.Union[typing.Any, NoneType]): data source
  • type (typing.Union[str, NoneType]): source type - package, resource or pipeline (default: infer)
  • pipeline (typing.Union[frictionless.pipeline.pipeline.Pipeline, str, NoneType])
  • steps (typing.Union[typing.List[frictionless.pipeline.step.Step], NoneType]): transform steps
  • options
It's a beta version of Frictionless Framework (v5). Read Frictionless Framework (v4) docs for a version that is currently installed by default by pip.