Description: The report represents a state-of-the-art comprehensive understanding on domestic violence in the partner countries, providing information on related stakeholders and services, notably crime reporting technologies and support services, investigation and prevention policies and measures.

Description: The report includes (1) the evaluation protocol, (2) a list of indicators for the evaluation of the short-term and mid-term learning outcomes and impact of the training and (3) questionnaires and other tools related to the serious game for police officers’ training.

Description: A serious game module, which, through behavioural exercises relying on role play and mock cases, aims to accurately depict the interaction a police officer or judicial system agent could have with victims of domestic violence and thus train the latter into better communicating with victims and more effectively approaching and handling their cases.

Description: A report, which includes the (1) guidelines for a safe, ethical and legal deployment of the chatbot; (2) the protocol to be followed in case of failure/problem with the chatbot; (3) a list of indicators for the evaluation; (4) rule for dialogue supervision organisation, continuous chatbot training and dialogue improvement.

Executive Summary:

This report consists of deliverable 3.1 of the ISEDA project: Deployment and Evaluation Protocol. The deliverable is intended to identify best methodologies and practices in order to support the safe, ethical and legal development and deployment of a chatbot for victim-survivors of domestic abuse. The chatbot will be used by victim-survivors in order to access information about domestic abuse, and also potentially as a place to store evidence that will be permissible at court.

The deliverable was informed via a literature search within major journals and databases for research relating to artificial intelligence, mobile apps and conversational agents in relation to domestic abuse, and a discussion with consortium members from member states. The authors also drew upon one existing app and one chatbot which support survivors of domestic abuse within a Scottish (FollowItApp) and Latin America and Caribbean (SARA) context. The literature and existing technological tools were assessed in order to establish what currently works well in relation to supporting victim-survivors via apps/chatbots, what the negatives are, useful features and a discussion of evaluation.

Many positives were identified in relation to supporting victim-survivors via technology. Overall, these points related to victim-survivors feeling that technology is less-judgemental than people, being able to be more open when using technology and technology being more accessible and convenient that human contact. Positives for young people of using digital technology were particularly highlighted. There were, however, negatives to using digital technology. There was suggestion that machines cannot accurately emulate human characteristics, for example, they cannot read body language or maintain eye contact. Victim-survivors also have concerns over the safety and security of using digital technology, such as whether their data is safe, and whether perpetrators can track them. Though accessibility of digital technology was highlighted as a positive, there were some concerns that it may not be accessible to all, such as if a person does not own or have access to a smartphone, and for some neuro-diverse survivors. These potential pitfalls should be considered and addressed as far as possible within this project.

Many features which could be useful when developing a new chatbot were highlighted within the literature and through research on FollowItApp and SARA. Key examples include the ability to establish rapport with victim-survivors in order to form a trusting relationship, ensuring high levels of safety and security surrounding the chatbot, ensuring the chatbot is inclusive and easy to use for survivors, and being able to provide a speedy, accurate, tailored service to victim-survivors. The importance of evaluation being built into the app was also raised.

In relation to evaluation, the report outlines indicators that have currently been used to assess apps, such as the Mobile Application Rating Scale, and the adapted version. Though these tools have been assessed to be unsuitable for the ISEDA project due to its focus on violence prevention and response, there are elements of their evaluation questions which are deemed useful to inform this project. Ongoing evaluation of the chatbot is key in order to ensure it remains up-to-date, accurate, trauma-informed and is technologically sound.

Guidelines for the safe, ethical and legal deployment of the chatbot have been developed and outlined in this report. Primary considerations are that the chatbot must be 1) efficient and effective, meeting its aims and objectives, 2) safe and ethical, taking into account data protection, potential abuse of the chatbot by perpetrators, and a trauma-informed approach and 3) legalities surrounding the chatbot, such as ensuring that it is clear who owns the IP address and data, who funds and updates the platform, where it is hosted and the need for solicitors to be involved. There should remain an awareness that legalities may differ between participating countries.

To support the safe, ethical and legal deployment of the chatbot, we have proposed protocols that should be developed and referred to throughout the life of the chatbot. The protocols relate to crisis management, complaints, subject access requests, data ownership and access, disclosures of other crimes, perpetrator ‘hacking’, information updates, handling sensitive data, language used in the chatbot and an evaluation protocol. This is not an exhaustive list. It is important that all protocols maintain a trauma-informed focus and are updated and developed throughout the project.

The protocol to be followed in case of failure of the chatbot was outlined specifically. It highlights the need to be aware how to respond to the person who is using the chatbot at the time of a failure/glitch, what happens to their data and at what point a human will become involved. There needs to be consideration of who will respond speedily in the case of glitches and failure in order to ensure victim-survivors remain confident using the app.

The chatbot needs to be evaluated in the pilot stage of this project to ensure that it is fit-for-purpose. To do so, groups of victim-survivors, professionals from third-sector women’s organisations and police will be asked to participate in supporting this. Evaluation questions for each group have been developed to assess the technological elements of the chatbot, how well it meets the needs of domestic abuse survivors and the extent to which victim-survivors feel safe when using/as a result of using the chatbot. The findings from these questions will be acted upon to develop the chatbot further.

Finally, we were asked to outline the rule for dialogue supervision organisation, continuous chatbot training and dialogue improvement. The proposals for this section are based primarily on views of participants at the consortium meeting. Suggestions include establishing a specialist IT team, ensuring the chatbot is updated continuously and defining the parameters of this, and making sure that development is informed by the voices of NGOs and victim-survivors. The questions of ensuring that funding is available for this was also raised as a key point.

We also highlighted further suggestions to be considered during the implementation stage of the chatbot and beyond. For example, considering the different beneficiaries of that chatbot, and what each of their needs are, how the chatbot is moderated and resourced and linking in victim-survivors with humans at the ‘right’ point.

Overall, this report sets out considerations that are important to action during chatbot development. They focus primarily on ensuring that victim-survivors of domestic abuse remain safe when using the chatbot, that they are receiving a service that is tailored to their needs, and one that is speedy and accurate. In order that this is the case, we have outlined protocols to be developed, and proposed key questions to be discussed by stakeholders when developing the tool. Further collaboration with members of the consortium and victim-survivors moving forward is key to ensuring the development of a successful chatbot.

Description: Report containing qualitative and quantitative data, including indicators directly provided by the chatbot, in accordance with the impact indicators, on the impact of the chatbot.

Description: A report containing data about the behaviour and attitudinal change of perpetrators, and the impact of this behaviour on the victim-survivors longitudinally and in a comparable way among the different sites.

Description: Developed materials throughout the awareness raising campaigns and education programs implemented in the ISEDA project, as well as information on their impact and how and what key stakeholders were engaged, including guidelines for adaptation and evaluation, available in the 7 languages of the consortium (BG, GR, ES, CY, EN, FI, IT).

Description: Report on the potential of the replicability of ISEDA technological and social solutions, notably the chatbot, data platform, the training and the Impact Toolkit of perpetrator programmes.

Description: A policy brief for European common approaches to fight domestic violence, including how the data platform can help support policy making.