Responsibly

Toolkit for Auditing and Mitigating Bias and Fairness of Machine Learning Systems 🔎🤖🧰

Responsibly is developed for practitioners and researchers in mind, but also for learners. Therefore, it is compatible with data science and machine learning tools of trade in Python, such as Numpy, Pandas, and especially scikit-learn.

The primary goal is to be one-shop-stop for auditing bias and fairness of machine learning systems, and the secondary one is to mitigate bias and adjust fairness through algorithmic interventions. Besides, there is a particular focus on NLP models.

Responsibly consists of three sub-packages:

1. responsibly.dataset

   Collection of common benchmark datasets from fairness research.

2. responsibly.fairness

   Demographic fairness in binary classification, including metrics and algorithmic interventions.

3. responsibly.we

   Metrics and debiasing methods for bias (such as gender and race) in word embedding.

For fairness, Responsibly’s functionality is aligned with the book Fairness and Machine Learning - Limitations and Opportunities by Solon Barocas, Moritz Hardt and Arvind Narayanan.

If you would like to ask for a feature or report a bug, please open a new issue or write us in Gitter.

Requirements

• Python 3.6+

Installation

Install responsibly with pip:

$ pip install responsibly

or directly from the source code:

$ git clone https://github.com/ResponsiblyAI/responsibly.git
$ cd responsibly
$ python setup.py install

Citation

If you have used Responsibly in a scientific publication, we would appreciate citations to the following:

@Misc{,
  author = {Shlomi Hod},
  title =  {{Responsibly}: Toolkit for Auditing and Mitigating Bias and Fairness of Machine Learning Systems},
  year =   {2018--},
  url =    "http://docs.responsibly.ai/",
  note =   {[Online; accessed <today>]}
}

Contents

• Datasets
  • Usage
  • General Dataset
  • Available Datasets
  • FICO Dataset
• Classification Fairness
  • Metrics
  • Algorithmic Interventions
• Word Embedding Bias
  • Bolukbasi Bias Measure and Debiasing
  • WEAT
  • Utilities
  • Word Embedding Benchmarks
• Demos
  • Demo - Fairness Analysis of COMPAS by ProPublica
  • Demo - Fairness Analysis of FICO
  • Demo - Bias in Word Embedding
• For Contributors
  • Setup
  • Development Tasks
  • Continuous Integration
  • Release Tasks
• Revision History
  • 0.1.3 (2021/04/02)
  • 0.1.2 (2020/09/15)
  • 0.1.1 (2019/08/04)
  • 0.1.0 (2019/07/31)
  • 0.0.5 (2019/06/14)
  • 0.0.4 (2019/06/03)
  • 0.0.3 (2019/04/10)
  • 0.0.2 (2018/09/01)
  • 0.0.1 (2018/08/17)
• License

Indices and tables

• Index

• Module Index

• Search Page