Edit page in Livemark
(2024-01-29 13:37)

Checklist Class

Creating Checklist

Checklist is a set of validation checks and a few addition settings. Let's create a checklist:

from frictionless import Checklist, checks

checklist = Checklist(checks=[checks.row_constraint(formula='id > 1')])
print(checklist)
{'checks': [{'type': 'row-constraint', 'formula': 'id > 1'}]}

Validation Checks

The Check concept is a part of the Validation API. You can create a custom Check to be used as part of resource or package validation.

from frictionless import Check, errors

class duplicate_row(Check):
    code = "duplicate-row"
    Errors = [errors.DuplicateRowError]

    def __init__(self, descriptor=None):
        super().__init__(descriptor)
        self.__memory = {}

    def validate_row(self, row):
        text = ",".join(map(str, row.values()))
        hash = hashlib.sha256(text.encode("utf-8")).hexdigest()
        match = self.__memory.get(hash)
        if match:
            note = 'the same as row at position "%s"' % match
            yield errors.DuplicateRowError.from_row(row, note=note)
        self.__memory[hash] = row.row_position

    # Metadata

    metadata_profile = {  # type: ignore
        "type": "object",
        "properties": {},
    }

It's usual to create a custom Error along side with a Custom Check.

Reference

Checklist (class)

Check (class)

Checklist (class)

Checklist representation. A class that combines multiple checks to be applied while validating a resource or package.

Signature

(*, name: Optional[str] = None, title: Optional[str] = None, description: Optional[str] = None, checks: List[Check] = NOTHING, pick_errors: List[str] = NOTHING, skip_errors: List[str] = NOTHING) -> None

Parameters

  • name (Optional[str])
  • title (Optional[str])
  • description (Optional[str])
  • checks (List[Check])
  • pick_errors (List[str])
  • skip_errors (List[str])

checklist.name (property)

A short name(preferably human-readable) for the Checklist. This MUST be lower-case and contain only alphanumeric characters along with "-" or "_".

Signature

Optional[str]

checklist.type (property)

Type of the object

Signature

ClassVar[Union[str, None]]

checklist.title (property)

A human-readable title for the Checklist.

Signature

Optional[str]

checklist.description (property)

A detailed description for the Checklist.

Signature

Optional[str]

checklist.checks (property)

List of checks to be applied during validation such as "deviated-cell", "required-value" etc.

Signature

List[Check]

checklist.pick_errors (property)

Specify the errors names to be picked while validation such as "sha256-count", "byte-count". Errors other than specified will be ignored.

Signature

List[str]

checklist.skip_errors (property)

Specify the errors names to be skipped while validation such as "sha256-count", "byte-count". Other errors will be included.

Signature

List[str]

checklist.add_check (method)

Add new check to the schema

Signature

(check: Check) -> None

Parameters

  • check (Check)

checklist.clear_checks (method)

Remove all the checks

Signature

() -> None

checklist.get_check (method)

Get check by type

Signature

(type: str) -> Check

Parameters

  • type (str)

checklist.has_check (method)

Check if a check is present

Signature

(type: str) -> bool

Parameters

  • type (str)

checklist.remove_check (method)

Remove check by type

Signature

(type: str) -> Check

Parameters

  • type (str)

checklist.set_check (method)

Set check by type

Signature

(check: Check) -> Optional[Check]

Parameters

  • check (Check)

Check (class)

Check representation. A base class for all the checks. To add a new custom check, it has to be derived from this class.

Signature

(*, name: Optional[str] = None, title: Optional[str] = None, description: Optional[str] = None) -> None

Parameters

  • name (Optional[str])
  • title (Optional[str])
  • description (Optional[str])

check.name (property)

A short url-usable (and preferably human-readable) name. This MUST be lower-case and contain only alphanumeric characters along with “.”, “_” or “-” characters.

Signature

Optional[str]

check.type (property)

A short name(preferably human-readable) for the Check. This MUST be lower-case and contain only alphanumeric characters along with "-" or "_".

Signature

ClassVar[str]

check.title (property)

A human-readable title for the Check.

Signature

Optional[str]

check.description (property)

A detailed description for the Check.

Signature

Optional[str]

check.Errors (property)

List of errors that are being used in the Check.

Signature

ClassVar[List[Type[Error]]]

check.resource (property)

Signature

Resource

check.connect (method)

Connect to the given resource

Signature

(resource: Resource)

Parameters

  • resource (Resource): data resource

check.validate_end (method)

Called to validate the resource before closing

Signature

() -> Iterable[Error]

check.validate_row (method)

Called to validate the given row (on every row)

Signature

(row: Row) -> Iterable[Error]

Parameters

  • row (Row): table row

check.validate_start (method)

Called to validate the resource after opening

Signature

() -> Iterable[Error]