Specific sets of impact metrics on the same theme are combined into a single indicator. Impact scores – out of 42 - are calculated based on a set of rules which takes these combinations into account. A higher score means the project is carrying out more activities related to the theme of the indicator, and is, therefore, more likely to have a higher positive impact in this area.
For example, the first indicator described below is that of “activeness”. Three impact metrics are combined to formulate this indicator. A high score means that the project is carrying out more activities related to “activeness”.
It is possible that the project may receive recommendations for indicators the project is not really interested in, or focused on. For example, the project may not have considered its economic productivity, and may not be interested in improving it. That’s OK. No project is obligated to take action on any of the recommendations MICS proposes.
To measure the impact of citizen science on society, we initially considered two indicators: activeness and involvement.
Definition: Activeness is the level of cognitive engagement; where “active” indicates that the participant requires full cognitive engagement during participation, and “passive” suggests there is no engagement beyond setup.
Questions:
Why we use it:
Activeness is included in the ECSA characteristics of citizen science and explained in the explanation notes of this document.
“Participants need to be aware of the contribution and participate actively and intentionally, as this is necessary for cases where the information that participants produced is not directly used by the project, but only as a secondary use of data (e.g. reusing images that people share on a social networking site). We recommend transparency about roles and expectations…. it is highly recommended to be open and transparent about choices that were made about the roles of participants. The project owner has responsibility for communicating that the participants are contributing to research.”
Activeness is further discussed in the associated peer-reviewed publication, “Contours of citizen science: a vignette study”.
All inidcator scores are calcualted by summing the weights of the answer options selected. Weights for the activeness indicator are given below:
Question number | Question text | Answers | Weights |
Society 4 | How much responsibility is offered to the participants? (with options depending on interests, availability and knowledge). | Not much | 0 |
Something in the middle | 8 | ||
A lot | 16 | ||
I don't know | 4 | ||
Society 8 | Are the participants satisfied with the process of participation in the project? | Yes, and it has been measured | 12 |
Yes, but it has not been measured | 10 | ||
No | 0 | ||
I don't know | 4 | ||
Society 12 | Are participants aware they are contributing to a research project? | Yes | 14 |
No | 0 | ||
I don't know | 4 |
Recommendations | Lower score | Higher score |
The activeness of participants within a project is an important aspect of citizen science. Efforts should be made to make participants aware they are contributing to a research project through clear communication channels, and to offer them the opportunity to be responsible for their activities. If the project has not measured their degree of satisfaction in the process, it might want to consider to consider investigating this further using this paper as a starting point. | 0 | 12 |
The activeness of participants within a project is an important aspect of citizen science. Activeness depends on participants being aware that they are contributing to a project, having a lot of responsibility in the project, and being satisfied with the process of participation. This project should ensure that all aspects of activeness have been considered. | 13 | 33 |
The activeness of participants within a project is an important aspect of citizen science, and this project has made great efforts to ensure participants are aware they are contributing to a research project, have responsibility in the project, and are satisfied with the process of participation. Great job! | 34 | 42 |
Definition: Involvement is the degree of participation in different stages of a process.
Questions:
Why we use it:
Involvement is included in the ECSA characteristics of citizen science.
“Research involving citizen science can take many forms, and the roles of the participants can include, for example: identifying a research question, collecting or analysing data to support or refute a hypothesis; monitoring environmental or health conditions for management or policy outcomes; and creation of generic data within a domain to support a wide range of research questions.”
Involvement is further discussed in the associated peer-reviewed publication, “Contours of citizen science: a vignette study” as well as Kieslinger’s paper, “ The Challenge of Evaluation: An Open Framework for Evaluating Citizen Science Activities”.
To measure the impact of citizen science on governance, we initially considered two indicators: policy and the sustainable development goals.
Definition: Policy is a deliberate system of guidelines to guide decisions and achieve rational outcomes. A policy is a statement of intent and is implemented as a procedure or protocol.
Questions:
Why we use it:
Policy is addressed in the ECSA characteristics of citizen science where it is covered under “Links to decision-making”.
“Citizen science projects may include an intervention into the current state of affairs, such as local decision making. This might happen in activities that fall under banners such as participatory action research, community science, or addressing environmental injustice.”
In the explanation notes of this document, it is also noted that “Citizen science can be used in cases where the participants are concerned with an issue and want to actively change the situation, be it concern over public health, medical support to a group of patients, or addressing a pollution issue.”
Definition: The Sustainable Development Goals (SDGs) or Global Goals are a collection of 17 interlinked global goals designed to be a "blueprint to achieve a better and more sustainable future for all". The SDGs were set up in 2015 by the United Nations General Assembly and are intended to be achieved by 2030.
Questions:
Why we use it:
Aside from the SDGs becoming the latest buzzword in citizen science (see Fritz et al., 2019, for example) and MICS’ commitment to considering the SDGs in our project Grant Agreement, the United Nations’ 2030 Agenda for Sustainable Development is an ambitious plan for “people, planet and prosperity”, aimed at achieving a sustainable future for all; what’s not to like about it?!
To measure the impact of citizen science on economy, we initially considered two indicators: economic productivity and financial sustainability.
Definition: Economic productivity measures output per unit of input, such as labour, capital, or any other resource. It is often calculated for the economy as a ratio of gross domestic product (GDP) to hours worked.
Questions:
Why we use it:
Economic productivity relates to SDG 8.2 Diversify, innovate and upgrade for economic productivity: Achieve higher levels of economic productivity through diversification, technological upgrading and innovation, including through a focus on high-value added and labour-intensive sectors
Definition: Financial sustainability is the assessment that a project will have sufficient funds to meet all its resource and financial obligations, whether the fund continues or not.
Questions:
Why we use it:
Citizen science is frequently hailed as being an inexpensive alternative to traditional science. Palmer et al. note, “with its relatively low cost centred on non-recurring investments, citizen science is inherently more scalable than traditional tools”. But financial sustainability is more than simply using cheap equipment - it’s also about planning ahead for the exploitation of outputs, for example - which is why this indicator is a little broader than Palmer’s original definition.
To measure the impact of citizen science on science and technology, we initially considered two indicators: scientific productivity and interdisciplinary science.
Definition: Scientific productivity refers to the productivity of scientists in their research performance. In other words, the term concerns how much output scientists produce within a certain time period, or compared to the inputs that are utilized for the research.
Questions:
Why we use it:
Scientific productivity is included in the ECSA characteristics of citizen science under the heading of data and knowledge generation.
“Citizen science, scientific, academic, and policy-oriented research can include different forms of data and knowledge generation, including novel data generation, creation of new analyses, or production of new knowledge in written and other forms. The knowledge produced in such projects should aspire to disciplinary standards, such as appropriate data quality and quality assurance, the peer review of project publications and materials, or policy-relevant evidence that is fit for decision-making.”
Kieslinger et al. also note the importance of citizen scientists participating in publications, or having their engagement recognised.
Definition: Interdisciplinary science is the collaborative process of integrating knowledge/expertise from trained individuals of two or more disciplines.
Questions:
Why we use it:
There is evidence that interdisciplinarity is statistically significantly and positively associated with research impact (Okamura, 2019).
To measure the impact of citizen science on environment, we initially considered two indicators: environmental footprint and environmental awareness.
Definition: The environmental (or ecological) footprint measures how fast we consume resources and generate waste compared to how fast nature can absorb our waste and generate resources.
Questions:
Why we use it:
Environmental footprint relates to SDG 9.4 Upgrade all industries and infrastructures for sustainability: By 2030, upgrade infrastructure and retrofit industries to make them sustainable, with increased resource-use efficiency and greater adoption of clean and environmentally sound technologies and industrial processes, with all countries taking action in accordance with their respective capabilities.
It also relates to SDG 12.2 Sustainable management and use of natural resources and SDG 12.7 Promote sustainable public procurement practices.
Moreover, even if a citizen-science project does not aim to have a positive impact on the environment, there are still ways in which the environmental footprint can be addressed; thus, this indicator is relevant to all citizen-science projects.
Definition: Environmental awareness can be broadly defined as the attitude regarding environmental consequences of human behaviour (Ham et al., 2016)
Questions:
Why we use it:
Environmental awareness relates to SDG 12.8 Promote universal understanding of sustainable lifestyles and SDG 13.3 Build knowledge and capacity to meet climate change.
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 824711.
Enter your email address above to receive updates on the MICS project. Please see our privacy policy for more information.
MICS science is open science. All the information on this website is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.