Inclusion by design: exploring gender responsive designs in digital welfare
Designing gender responsive ADMS

SHARE

Inclusion by design: exploring gender responsive designs in digital welfare is the second of a two-report series. The first report demonstrated how Western welfare systems have long reflected the gendered disadvantages present in society and shone a light on examples of discriminatory automated decision-making systems (ADMS) managing digital welfare systems in several Western Countries. It concluded by highlighting gender equality principles for the digital welfare state. This report builds on those principles. There is still time to address and rectify the gendered harms welfare systems can inflict.

The implementation of automation has been mostly a two-sided conversation between policymakers and technologists, an approach that does not take into consideration the implications of digitising old ways of working and inherited social structures. We cannot assume that technology is a neutral actor when it is history that has built it. Instead, it is systems designed to acknowledge the historic biases that preceded them and understand the complex lives of those who seek their services that lead to a more equitable future.

To design digital welfare systems in this way requires an understanding of the interconnected nature of the social categorisations that the system users will belong to. Relating to gender this could mean many things, including the increased probability of a women being unemployed while also more likely to be engaged in in unpaid labour positions such as caring for children or other dependent family members. If public sectors are going to automate the decision-making processes that manage access to welfare systems, these ADMS must include different frameworks to address the intersectionality of women users.

In this regard, this report maps a way forward in which design can play a fundamental role in envisioning digital welfare services contributing to a more gender-equitable future. Gender responsive design means compiling and using gender-relevant datasets and statistics, incorporating gender analysis and gender impact assessments, and recognising that co-design and human oversight are fundamental to avoiding the automation of errors and inequalities.
When public servants design a service, they have to think they one day might be the very person who needs to access that service. It requires a level of empathy to understand that some public services have become a painful life event for the user. If the state cannot go through the exercise to understand the level of trauma and sensitivity, and furthermore does not apply that empathy to design, it is giving a bad service.
Daniel Abadie - former Digital Secretary of Argentina

Gender responsive designs

The first design concept proposes a data governance model supported by a human-centric experience framework that empowers vulnerable women while preserving their privacy. DefaultsMatter is a data institution that works independently from the government to help women in vulnerable situations access benefits. Its greater goal is to contribute to a better national data infrastructure and improve policies around benefits that disproportionately affect women. The concept uses a welfare payment programme similar to Universal Credit as a scenario and aims to address the programme ADMS’ lack of flexibility and diversity by nourishing the system with human relationships and trust to build a fairer ecosystem.

Download the report to learn more about how DefaultsMatter seeks to achieve this goal and scenarios highlighting how it would work in practice, including the scenario displayed below.

Who are we missing? is a design process available to public agencies in charge of deploying digital welfare services. The process facilitates a less biased decision-making process using a combination of inclusive and diverse teams and gender-responsive algorithms. The concept explores the challenge of how to create a more humane, diverse, and collaborative approach to the design of ADMS in the public sector.

The exploration spans before, during and after the building of an ADMS:

  • Before: Could collaboration between key stakeholders inside and outside of public institutions improve upon the lack of diversity in teams?
  • During: What if we could include in the workstream and datasets those groups usually excluded?
  • After: What if we could ensure a higher standard of fairness regarding groups who may need social protection?

Download the report to learn more about the organisation-wide practices designed to create a culture that organically encourages teams to seek diversity and to use cases of them in action.

The third design concept is a digital service that proposes an improvement to the interactions between humans and machines, incentivising ‘human oversight’ for automated decisions by giving caseworkers a more holistic understanding of benefit claimants. The approach suggests a better collaborative process, using the unique skills and experience of humans alongside the analytical and statistical analysis of machines.

Human says yes! leverages two key elements:

  • Artificial & collective intelligence for better decisions: pairing artificial intelligence of the ADMS with the collective intelligence and experiences of caseworkers, so decisions are built and improved with both inputs.
  • Immediate & long-term feedback loops: by recording the decisions of caseworkers and considering those as collective intelligence, the solution helps build a fairer feedback loop in the current case. It also creates the basis for reviewing the ADMS design or the application of rules and policies in the long term.

The report uses two scenarios to illustrate how this concept would work in practice. Both practices include the development of interfaces that help both caseworkers and claimants understand and consider the real-life context that their cases exist in and even other cases that are similar to their own. This not only makes the decision-making decisions more transparent, but also provides a platform for contesting ADMS decisions.

Download the report to learn more.