Skip to main content

Design Guide

This guides provides a high-level overview of the Frictionless Framework architecture. It will be useful for plugin authors and advanced users.

Reading Flow#

Frictionless uses modular approach for its architecture. During reading a data source goes through various subsystems which are selected depending on the data characteristics:

Reading

For example for a local CSV file it will use:

  • LocalLoader
  • CsvParser

System Object#

The most important undelaying object in the Frictionless Framework is system. It's an singletone object avaialble as frictionless.system. This object can be used to instantiate different kind of lower-level as though Check, Step, or Type. Here is a quick example of using the system object:

Python
from frictionless import Resource, system
resource = Resource(path='https://example.com/data/table.csv')
check = system.create_check({'code': 'duplicate-row'})
control = system.create_control(resource, descriptor={'httpTimeout': 1000})
dialect = system.create_dialect(resource, descriptor={'delimiter': ';'})
loader = system.create_loader(resource)
parser = system.create_parser(resource)
server = system.create_server('api')
step = system.create_step({'code': 'table-validate'})
storage = system.create_storage('sql', 'postgresql://database')
type = system.create_type(resource.get_field('id'))

As an extension author you might use the system object in various cases. For example, take a look at this MultipartLoader excerpts:

Python
def read_line_stream(self):
for number, path in enumerate(self.__path, start=1):
with system.create_loader(Resource(path=path)) as loader:
for line_number, line in enumerate(loader.byte_stream, start=1):
if not self.__headless and number > 1 and line_number == 1:
continue
yield line

It's important to understand that creating low-level objects in general is more corect using the system object than just classes because it will include all the available plugins in the process.

Last updated on by Shashi Gharti