Edit page in Livemark
(2024-01-29 13:37)

Contributing

We welcome contributions from anyone! Please read the following guidelines, and feel free to reach out to us if you have questions. Thanks for your interest in helping make Frictionless awesome!

Introduction

We use Github as a code and issues hosting platform. To report a bug or propose a new feature, please open an issue. For pull requests, we would ask you initially create an issue and then create a pull requests linked to this issue.

Prerequisites

To start working on the project:

Install Python headers if they are missing:

sudo apt-get install libpython3.10-dev

Enviroment

For development orchestration we use Hatch for Python (defined in pyproject.toml). We use make to run high-level commands (defined in Makefile)

pip3 install hatch

Before starting with the project we recommend configuring hatch. The following line will ensure that all the virtual environments will be stored in the .python directory in the project root:

hatch config set 'dirs.env.virtual' '.python'

Now you can setup you IDE to use a proper Python path:

.python/frictionless/bin/python

Enter the virtual environment before starting the work. It will ensure that all the development dependencies are installed into a virtual environment:

hatch shell

Using Docker

Use the following command to build the container:

make docker

This should take care of setting up everything. If the container is built without errors, you can then run commands like make inside the container to accomplish various tasks (see the next section for details).

To make things easier, we can create an alias:

alias "frictionless-dev=docker run --rm -v $PWD:/home/frictionless -it frictionless-dev"

Then, for example, to run the tests, we can use:

frictionless-dev make test

Development

Codebase

Frictionless is a Python3.8+ framework, and it uses some common Python tools for the development process (we recommend enabling support of these tools in your IDE):

You also need git to work on the project, and make is recommended.

Documentation

To contribute to the documentation, please find an article in the docs folder and update its contents. We write our documentation using Livemark. Livemark provides an ability to provide examples without providing an output as it's generated automatically.

It's possible to run this documentation portal locally:

livemark start

Running tests offline

VCR library records the response from HTTP requests locally as cassette in its first run. All subsequent calls are run using recorded metadata from previous HTTP request, so it speeds up the testing process. To record a unit test(as cassette), we mark it with a decorator:

@pytest.mark.vcr
def test_connect_with_server():
	pass

Cassettee will be recorded as "test_connect_with_server.yaml". A new call is made when params change. To skip sensitive data, we can use filters:

@pytest.fixture(scope="module")
def vcr_config():
    return {"filter_headers": ["authorization"]}

Regenerating cassettes for CKAN

CKAN_APIKEY=***************************

Regenerating cassettes for Zenodo

Read

ZENODO_ACCESS_TOKEN=***************************

Write

ZENODO_SANDBOX_ACCESS_TOKEN=***************************
base_url='base_url="https://sandbox.zenodo.org/api/'

Regenerating cassettes for Github

GITHUB_NAME=FD
[email protected]
GITHUB_ACCESS_TOKEN=***************************

Releasing

To release a new version: