MICS analysis of project impact in the Citizen Science Sector 

To comparatively assess the impact of the citizen-science element of MONOCLE, we utilised the MICS (Measuring Impact of Citizen Science) standardised impact assessment tool.  

The impact-assessment process on the MICS platform includes over 200 questions, each with a pre-defined set of answers that users choose from. These questions assess hundreds of indicators and are based on current impact-assessment methods and other frameworks, such as the ECSA characteristics of citizen science. The specific answers from each project are analysed through a series of artificial-intelligence (AI) algorithms, which result in impact scores across five domains – science and technology, society, governance, environment and economy – all of which are scored out of a maximum of 42 points. 

At the time of submission, the algorithms are still in a “learning” phase of development, with the machine-learning scores attributed to the project subject to change based on further data input. However, the rule-based scores are calculated based on a set of rules which combine specific sets of impact metrics on the same theme into a single indicator and are therefore already useful to consider.  

MONOCLE scored highest in the science and Technology domain, with a score of 31 for scientific productivity – a measure of how much output scientists produce within a certain time period, or compared to the inputs that are utilized for the research – and a score of 36 for interdisciplinary science: the collaborative process of integrating knowledge/expertise from trained individuals of two or more disciplines. This collaborative process has been shown to be statistically significantly and positively associated with research impact. These scores reflect the collaborative nature of the work conducted in WP2 and WP3 (development of sensors and integration into platforms such as citizen science), and the outputs of the project in terms of technology and solutions. 

MONOCLE also produced respectable scores in the Environment domain, with scores of 26 and 22 for environmental awareness and environmental footprint, respectively. These scores reflect the status of the project as ready (and tested) to engage with citizen science projects rather than achieved societal impact. Across other domains there is mixed feedback. In Economy, while economic productivity (output per unit of input) did not score highly, financial sustainability - the assessment that a project will have sufficient means to sustain its outputs - did (31), reflecting the exploitation efforts of WP8. In Governance, an understanding of, and reporting around the SDGs scored well (26), however Policy was much lower (7). This score would be improved upon publication and uptake of the drafted white paper in the short term, and it is worth noting that the assessment on the MICS platform can be updated at any time to reflect the evolution of the project, even beyond the end of the project. 

In Society, MONOCLE scored well (26) in activeness - the level of cognitive engagement of participants – but less well in involvement; the degree of participation in different stages of a project. This is a fair criticism of the ongoing citizen science element of MONOCLE: whilst participants had to fully cognitively engage with the activities, there were limited phases of the project in which they played a role – namely data collection, and with the sole purpose of testing new technology. These scores are not surprising considering the scope of MONOCLE is not to do citizen science, but to develop support for it, and so including citizen scientists in other phases of the project would not have been appropriate in this instance. 

In summary, MONOCLE had a positive impact in aspects of all impact domains, with particularly positive impact on science and technology as a whole, as well as financial sustainability and participant activeness within the economy and society domains, respectively. 

According to the MICS project, Cos4Cloud scored highest in the Environment domain; with scores of 40 for environmental awareness – the attitude regarding environmental consequences of human behaviour – and 32 for environmental footprint. The MICS platform notes that Cos4Cloud goes to great lengths to promote environmental awareness and educate participants on environmental challenges (through the sustained efforts of WP8 Communication, Outreach and Stakeholder Engagement); and to measure improvements in participants' environmental attitudes, behaviour and knowledge (as part of activities in WP5 COS4CLOUD services in practice, and WP6 Networking, Training and Capacity Building).

Cos4Cloud also scored highly in the Science and Technology domain, with 37 for scientific productivity and 33 for interdisciplinary science. This reflects the high number of peer-reviewed publications produced by the project, and the highly collaborative nature of the Cos4Cloud team; with technological developers working alongside social scientists and educators.

The project scored well in both the Economy and Society domains. Cos4Cloud scored an impressive 42 for economic productivity, indicating an explicit improvement in economic productivity through “diversification, technological upgrading and innovation”; specifically, the creation of thirteen new services for citizen observatories. The project scores 26 for financial sustainability, due to a combination of positive aspects – the use of an exploitation plan, for example – and less positive economic aspects, such as the need for recurring investments in the technology developed. Cos4Cloud scored 34 and 30 for activeness – the level of cognitive engagement in participating citizen scientists – and involvement – the degree of participation in different stages of a process – respectively. This reflects the highly collaborative co-design and testing process that formed part of the development of the thirteen services, and their incorporation into citizen observatories.

Cos4Cloud scored lower in the Governance domain, with a score of 16 for the Sustainable Development Goal (SDG) indicator and 6 for the policy indicator. In terms of the currently low score for policy, it is likely that with further dissemination of the two policy briefs produced in the final month of the project, that Cos4Cloud’s impact on policy will increase over time.

In summary, the citizen science element of Cos4Cloud has made a positive impact across multiple MICS domains, with particularly high scores in scientific and economic productivity, reflecting the thirteen innovative services developed within the project. The potential for policy briefs and further publications beyond the lifetime of the project suggests that Cos4Cloud could have a more impactful legacy than at present.