The most important undelaying object in the Frictionless Framework is system. It's an singleton object avaialble as frictionless.system.
Using the system object a user can alter the execution context. It uses a Python context manager so it can be used in anyway that it's possible in Python, for example, it can be nested or combined.
If data or metadata comes from a trusted origin, it's possible to disable safety checks for paths:
with system.use_context(trusted=True):
extract('/path/to/file/is/absolute.csv')
To raise warning or errors on data problems, it's possible to use onerror context value. It's default to ignore and can be set to warn or error:
with system.use_context(onerror='error'):
extract('table-with-error-will-raise-an-exeption.csv')
By default, the framework uses coming v2 version of the standards for outputing metadata. It's possible to alter this behaviour:
with system.use_context(standards='v1'):
describe('metadata-will-be-in-v1.csv')
It's possible to provide a custom requests.Session:
session = requests.Session()
with system.use_context(http_session=session):
with Resource(BASEURL % "data/table.csv") as resource:
assert resource.header == ["id", "name"]
This object can be used to instantiate different kind of lower-level as though Check, Step, or Field. Here is a quick example of using the system object:
from frictionless import Resource, system
# Create
adapter = system.create_adapter(source, control=control)
loader = system.create_loader(resource)
parser = system.create_parser(resource)
# Detect
system.detect_resource(resource)
field_candidates = system.detect_field_candidates()
# Select
Check = system.selectCheck('type')
Control = system.selectControl('type')
Error = system.selectError('type')
Field = system.selectField('type')
Step = system.selectStep('type')
As an extension author you might use the system object in various cases. For example, take a look at this MultipartLoader excerpts:
def read_line_stream(self):
for number, path in enumerate(self.__path, start=1):
resource = Resource(path=path)
resource.infer(sample=False)
with system.create_loader(resource) as loader:
for line_number, line in enumerate(loader.byte_stream, start=1):
if not self.__headless and number > 1 and line_number == 1:
continue
yield line
It's important to understand that creating low-level objects in general is more corect using the system object than just classes because it will include all the available plugins in the process.
The Plugin API almost fully follows the system object's API. So as a plugin author you need to hook into the same methods. For example, let's take a look at a builtin Csv Plugin:
class CsvPlugin(Plugin):
"""Plugin for CSV"""
# Hooks
def create_parser(self, resource: Resource):
if resource.format in ["csv", "tsv"]:
return CsvParser(resource)
def detect_resource(self, resource: Resource):
if resource.format in ["csv", "tsv"]:
resource.type = "table"
resource.mediatype = f"text/{resource.format}"
def select_Control(self, type: str):
if type == "csv":
return CsvControl
Loader representation
(resource: Resource)
Specifies if the resource is remote.
bool
types.IBuffer
Resource byte stream The stream is available after opening the loader
types.IByteStream
Whether the loader is closed
bool
Resource
Resource text stream The stream is available after opening the loader
types.ITextStream
Close the loader as "filelike.close" does
() -> None
Open the loader as "io.open" does
Read bytes stream
() -> types.IByteStream
Detect metadta using sample
(buffer: bytes)
Buffer byte stream
(byte_stream: types.IByteStream)
Create bytes stream
() -> types.IByteStream
Decompress byte stream
(byte_stream: types.IByteStream) -> types.IByteStream
Process byte stream
(byte_stream: types.IByteStream) -> ByteStreamWithStatsHandling
Read text stream
Write from a temporary file
(path: str) -> Any
Create byte stream for writing
(path: str) -> types.IByteStream
Store byte stream
(byte_stream: types.IByteStream) -> Any
Parser representation
(resource: Resource)
Specifies if parser requires the loader to load the data.
ClassVar[bool]
Data types supported by the parser.
ClassVar[List[str]]
types.ICellStream
Whether the parser is closed
bool
Loader
Resource
types.ISample
Close the parser as "filelike.close" does
() -> None
Open the parser as "io.open" does
Read list stream
() -> types.ICellStream
Create list stream from loader
() -> types.ICellStream
Wrap list stream into error handler
(cell_stream: types.ICellStream) -> CellStreamWithErrorHandling
Create and open loader
() -> Optional[Loader]
Write row stream from the source resource
(source: TableResource) -> Any
Plugin representation It's an interface for writing Frictionless plugins. You can implement one or more methods to hook into Frictionless system.
Create adapter
(source: Any, *, control: Optional[Control] = None, basepath: Optional[str] = None, packagify: bool = False) -> Optional[Adapter]
Create loader
(resource: Resource) -> Optional[Loader]
Create parser
(resource: Resource) -> Optional[Parser]
Detect field candidates
(candidates: List[dict[str, Any]]) -> None
Hook into resource detection
(resource: Resource) -> None
System representation This class provides an ability to make system Frictionless calls. It's available as `frictionless.system` singletone.
A flag that indicates if resource, path or package is trusted.
ClassVar[List[str]]
A flag that indicates if resource, path or package is trusted.
bool
Type of action to take on Error such as "warn", "raise" or "ignore".
types.IOnerror
Setting this value user can use feature of the specific version. The default value is v2.
types.IStandards
Return a HTTP session This method will return a new session or the session from `system.use_http_session` context manager
Create adapter
(source: Any, *, control: Optional[Control] = None, basepath: Optional[str] = None, packagify: bool = False) -> Optional[Adapter]
Create loader
(resource: Resource) -> Loader
Create parser
(resource: Resource) -> Parser
Deregister a plugin
(name: str)
Create candidates
() -> List[dict[str, Any]]
Hook into resource detection
(resource: Resource) -> None
Register a plugin
(name: str, plugin: Plugin)