We welcome contributions from anyone! Please read the following guidelines, and feel free to reach out to us if you have questions. Thanks for your interest in helping make Frictionless awesome!
We use Github as a code and issues hosting platform. To report a bug or propose a new feature, please open an issue. For pull requests, we would ask you initially create an issue and then create a pull requests linked to this issue.
To contribute to the documentation, please find an article in the docs
folder and update its contents. We write our documentation using Livemark. Livemark provides an ability to provide examples without providing an output as it's generated automatically.
It's possible to run this documentation portal locally:
livemark start
Frictionless is a Python3.8+ framework, and it uses some common Python tools for the development process:
pyright
black
pylama
pytest
You also need git
to work on the project, and make
is recommended.
After cloning the repository, you can set up the development environment either by creating a virtual environment or a docker container.
Create a virtual environment and install the dependencies by following this code:
this will install a
git commit
hook running the tests
python3.8 -m venv .python
source .python/bin/activate
pip install wheel
make install
alias "frictionless=python -m frictionless"
Note: You may need to run sudo apt-get install postgresql libpq-dev
on a Debian-based system, because the python Postgres module depends on some postgres CLI tools.
Use the following command to build the container:
make docker-setup
This should take care of setting up everything. If the container is
built without errors, you can then run commands like make
inside the
container to accomplish various tasks (see the next section for details).
To make things easier, we can create an alias:
alias "frictionless-dev=docker run --rm -v $PWD:/home/frictionless -it frictionless-dev"
Then, for example, to run the tests, we can use:
frictionless-dev make test
Then you can run various make commands:
make docs
- build the docsmake format
- format source codemake install
- install the dependencies (we did before)make lint
- lint the projectmake release
- release a new versionmake test
- run the testsmake test-ci
- run the tests (including integration)We also recommend running underlying commands like pytest
or pylama
to speed up the development process, though this is optional.
To release a new version:
master
branchfrictionless/__version__
following the SemVer standardCHANGELOG.md
if it's not a patch release (major or micro)make release
which create a release commit and tag and push it to GithubVCR library records the response from HTTP requests locally as cassette in its first run. All subsequent calls are run using recorded metadata from previous HTTP request, so it speeds up the testing process. To record a unit test(as cassette), we mark it with a decorator:
@pytest.mark.vcr
def test_connect_with_server():
pass
Cassettee will be recorded as "test_connect_with_server.yaml". A new call is made when params change. To skip sensitive data, we can use filters:
@pytest.fixture(scope="module")
def vcr_config():
return {"filter_headers": ["authorization"]}
CKAN_APIKEY=***************************
Read
ZENODO_ACCESS_TOKEN=***************************
Write
ZENODO_SANDBOX_ACCESS_TOKEN=***************************
base_url='base_url="https://sandbox.zenodo.org/api/'
GITHUB_NAME=FD
GITHUB_EMAIL=frictionle[email protected]
GITHUB_ACCESS_TOKEN=***************************