Edit page in Livemark
(2022-09-19 18:33)

Field Steps

The Field steps are responsible for managing a Table Schema's fields. You can add or remove them along with more complex operations like unpacking.

Add Field

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_add(name="note", value="eu", descriptor={"type": "string"}),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'id', 'type': 'integer'},
            {'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'},
            {'name': 'note', 'type': 'string'}]}
+----+-----------+------------+------+
| id | name      | population | note |
+====+===========+============+======+
|  1 | 'germany' |         83 | 'eu' |
+----+-----------+------------+------+
|  2 | 'france'  |         66 | 'eu' |
+----+-----------+------------+------+
|  3 | 'spain'   |         47 | 'eu' |
+----+-----------+------------+------+

Reference

steps.field_add (class)

steps.field_add (class)

Add field

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, value: Optional[Any] = None, formula: Optional[Any] = None, function: Optional[Any] = None, position: Optional[int] = None, descriptor: Optional[IDescriptor] = None, incremental: bool = False) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • value (Optional[Any])
  • formula (Optional[Any])
  • function (Optional[Any])
  • position (Optional[int])
  • descriptor (Optional[IDescriptor])
  • incremental (bool)

steps.field_add.name (property)

NOTE: add docs

Signature

str

steps.field_add.value (property)

NOTE: add docs

Signature

Optional[Any]

steps.field_add.formula (property)

NOTE: add docs

Signature

Optional[Any]

steps.field_add.function (property)

NOTE: add docs

Signature

Optional[Any]

steps.field_add.position (property)

NOTE: add docs

Signature

Optional[int]

steps.field_add.descriptor (property)

NOTE: add docs

Signature

Optional[IDescriptor]

steps.field_add.incremental (property)

NOTE: add docs

Signature

bool

Filter Fields

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_filter(names=["id", "name"]),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'id', 'type': 'integer'},
            {'name': 'name', 'type': 'string'}]}
+----+-----------+
| id | name      |
+====+===========+
|  1 | 'germany' |
+----+-----------+
|  2 | 'france'  |
+----+-----------+
|  3 | 'spain'   |
+----+-----------+

Reference

steps.field_filter (class)

steps.field_filter (class)

Filter fields

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, names: List[str]) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • names (List[str])

steps.field_filter.names (property)

NOTE: add docs

Signature

List[str]

Merge Fields

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
     source,
     steps=[
     	 # seperator argument can be used to set delimeter. Default value is '-'
    	 # preserve argument keeps the original fields
         steps.field_merge(name="details", from_names=["name", "population"], preserve=True)
     ],
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'id', 'type': 'integer'},
            {'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'},
            {'name': 'details', 'type': 'string'}]}
+----+-----------+------------+--------------+
| id | name      | population | details      |
+====+===========+============+==============+
|  1 | 'germany' |         83 | 'germany-83' |
+----+-----------+------------+--------------+
|  2 | 'france'  |         66 | 'france-66'  |
+----+-----------+------------+--------------+
|  3 | 'spain'   |         47 | 'spain-47'   |
+----+-----------+------------+--------------+

Reference

steps.field_merge (class)

steps.field_merge (class)

Merge fields This step can be added using the `steps` parameter for the `transform` function.

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, from_names: List[str], separator: str = -, preserve: bool = False) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • from_names (List[str])
  • separator (str)
  • preserve (bool)

steps.field_merge.name (property)

NOTE: add docs

Signature

str

steps.field_merge.from_names (property)

NOTE: add docs

Signature

List[str]

steps.field_merge.separator (property)

NOTE: add docs

Signature

str

steps.field_merge.preserve (property)

NOTE: add docs

Signature

bool

Move Field

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_move(name="id", position=3),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'},
            {'name': 'id', 'type': 'integer'}]}
+-----------+------------+----+
| name      | population | id |
+===========+============+====+
| 'germany' |         83 |  1 |
+-----------+------------+----+
| 'france'  |         66 |  2 |
+-----------+------------+----+
| 'spain'   |         47 |  3 |
+-----------+------------+----+

Reference

steps.field_move (class)

steps.field_move (class)

Move field

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, position: int) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • position (int)

steps.field_move.name (property)

NOTE: add docs

Signature

str

steps.field_move.position (property)

NOTE: add docs

Signature

int

Pack Fields

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
    	# field_type returns packed fields as JSON Object. Default value for field_type is 'array'
    	# preserve argument keeps the original fields
        steps.field_pack(name="details", from_names=["name", "population"], as_object=True, preserve=True)
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'id', 'type': 'integer'},
            {'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'},
            {'name': 'details', 'type': 'object'}]}
+----+-----------+------------+-----------------------------------------+
| id | name      | population | details                                 |
+====+===========+============+=========================================+
|  1 | 'germany' |         83 | {'name': 'germany', 'population': '83'} |
+----+-----------+------------+-----------------------------------------+
|  2 | 'france'  |         66 | {'name': 'france', 'population': '66'}  |
+----+-----------+------------+-----------------------------------------+
|  3 | 'spain'   |         47 | {'name': 'spain', 'population': '47'}   |
+----+-----------+------------+-----------------------------------------+

Reference

steps.field_pack (class)

steps.field_pack (class)

Pack fields

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, from_names: List[str], as_object: bool = False, preserve: bool = False) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • from_names (List[str])
  • as_object (bool)
  • preserve (bool)

steps.field_pack.name (property)

NOTE: add docs

Signature

str

steps.field_pack.from_names (property)

NOTE: add docs

Signature

List[str]

steps.field_pack.as_object (property)

NOTE: add docs

Signature

bool

steps.field_pack.preserve (property)

NOTE: add docs

Signature

bool

Remove Field

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_remove(names=["id"]),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'}]}
+-----------+------------+
| name      | population |
+===========+============+
| 'germany' |         83 |
+-----------+------------+
| 'france'  |         66 |
+-----------+------------+
| 'spain'   |         47 |
+-----------+------------+

Reference

steps.field_remove (class)

steps.field_remove (class)

Remove field

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, names: List[str]) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • names (List[str])

steps.field_remove.names (property)

NOTE: add docs

Signature

List[str]

Split Field

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_split(name="name", to_names=["name1", "name2"], pattern="a"),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'id', 'type': 'integer'},
            {'name': 'population', 'type': 'integer'},
            {'name': 'name1', 'type': 'string'},
            {'name': 'name2', 'type': 'string'}]}
+----+------------+--------+-------+
| id | population | name1  | name2 |
+====+============+========+=======+
|  1 |         83 | 'germ' | 'ny'  |
+----+------------+--------+-------+
|  2 |         66 | 'fr'   | 'nce' |
+----+------------+--------+-------+
|  3 |         47 | 'sp'   | 'in'  |
+----+------------+--------+-------+

Reference

steps.field_split (class)

steps.field_split (class)

Split field

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, to_names: List[str], pattern: str, preserve: bool = False) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • to_names (List[str])
  • pattern (str)
  • preserve (bool)

steps.field_split.name (property)

NOTE: add docs

Signature

str

steps.field_split.to_names (property)

NOTE: add docs

Signature

List[str]

steps.field_split.pattern (property)

NOTE: add docs

Signature

str

steps.field_split.preserve (property)

NOTE: add docs

Signature

bool

Unpack Field

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_update(name="id", value=[1, 1], descriptor={"type": "string"}),
        steps.field_unpack(name="id", to_names=["id2", "id3"]),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'},
            {'name': 'id2', 'type': 'any'},
            {'name': 'id3', 'type': 'any'}]}
+-----------+------------+-----+-----+
| name      | population | id2 | id3 |
+===========+============+=====+=====+
| 'germany' |         83 |   1 |   1 |
+-----------+------------+-----+-----+
| 'france'  |         66 |   1 |   1 |
+-----------+------------+-----+-----+
| 'spain'   |         47 |   1 |   1 |
+-----------+------------+-----+-----+

Reference

steps.field_unpack (class)

steps.field_unpack (class)

Unpack field

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, to_names: List[str], preserve: bool = False) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • to_names (List[str])
  • preserve (bool)

steps.field_unpack.name (property)

NOTE: add docs

Signature

str

steps.field_unpack.to_names (property)

NOTE: add docs

Signature

List[str]

steps.field_unpack.preserve (property)

NOTE: add docs

Signature

bool

Update Field

Example

from pprint import pprint
from frictionless import Package, Resource, transform, steps

source = Resource(path="transform.csv")
target = transform(
    source,
    steps=[
        steps.field_update(name="id", value=str, descriptor={"type": "string"}),
    ]
)
print(target.schema)
print(target.to_view())
{'fields': [{'name': 'id', 'type': 'string'},
            {'name': 'name', 'type': 'string'},
            {'name': 'population', 'type': 'integer'}]}
+------+-----------+------------+
| id   | name      | population |
+======+===========+============+
| None | 'germany' |         83 |
+------+-----------+------------+
| None | 'france'  |         66 |
+------+-----------+------------+
| None | 'spain'   |         47 |
+------+-----------+------------+

Reference

steps.field_update (class)

steps.field_update (class)

Update field

Signature

(*, title: Optional[str] = None, description: Optional[str] = None, name: str, value: Optional[Any] = None, formula: Optional[Any] = None, function: Optional[Any] = None, descriptor: Optional[IDescriptor] = None) -> None

Parameters
  • title (Optional[str])
  • description (Optional[str])
  • name (str)
  • value (Optional[Any])
  • formula (Optional[Any])
  • function (Optional[Any])
  • descriptor (Optional[IDescriptor])

steps.field_update.name (property)

NOTE: add docs

Signature

str

steps.field_update.value (property)

NOTE: add docs

Signature

Optional[Any]

steps.field_update.formula (property)

NOTE: add docs

Signature

Optional[Any]

steps.field_update.function (property)

NOTE: add docs

Signature

Optional[Any]

steps.field_update.descriptor (property)

NOTE: add docs

Signature

Optional[IDescriptor]

It's a beta version of Frictionless Framework (v5). Read Frictionless Framework (v4) docs for a version that is currently installed by default by pip.