CoAct Database Methodology
The CoAct Database is built through systematic coding of publicly available information on cooperative climate initiatives. Cooperative climate initiatives are identified through following launches as orchestration events, including annual Conferences of the Parties (COP) and regional Climate Weeks. Additional initiatives are added based on snowballing approaches. An initiative is included in the CoAct Database when it meets all of the following criteria: (1) involves two or more actors; (2) involves one or more non-state and/or subnational actor(s); (3) addresses climate change through mitigation and/or adaptation; and (4) operates across borders, with implementation in and/or partners from two or more countries.
Data is then collected through systematic coding following a detailed codebook using standardised initiative-specific coding forms. Sources of information include initiatives’ websites and subdomains, reports and other documentation, newsletters, news items, and social media channels.
Our methodology includes several steps to support reliability and validity. First, each coder has followed an extensive training process prior to coding. The training consists of an overview of the general workflow, reading the codebook, a walkthrough of coding an example initiative, and practice coding of some initiatives followed by a discussion about discrepancies in the coded data. Second, the codebook supports coding throughout the full process and is regularly updated to improve reliability. Third, we have weekly meetings to discuss issues, doubts, and questions. Each coder is also supported by a mentor from the core team to have a dedicated point of contact to discuss questions if they arise outside of meetings. Fourth, data collection forms are updated in each coding cycle for enhancing reliability. Finally, an intercoder reliability check is performed regularly by randomly selecting a sample of initiatives that is then double coded. Intercoder reliability indicators are calculated for each data category to identify weaknesses in the reliability. Action is taken to improve the reliability.
Aggregated indices
Based on the collected data, four key aggregated indices are calculated to provide insights into initiatives' output effectiveness, accountability, inclusiveness and capacity.
Output effectiveness
Output effectiveness is calculated using the Function-Output-Fit (FOF) score. This indicator measures if the outputs that the initiatives produce contribute to fulfilling the functions they aim to perform. Functions can include knowledge dissemination, setting standards, providing funding, or implementing on-the-ground. Outputs, such as publication of reports or organisation of events, can contribute to some of these functions. The FOF score represents how much of an initiatives' functions are supported by corresponding outputs. FOF scores fall within a range of 0 to 1, with 0 representing no corresponding outputs and 1 representing that the initiative works on all their functions by producing outputs. FOF scores can be interpreted as the likelihood that an initiative will meet their social and environmental goals. A ‘1’ or a ‘full function-output-fit’ does not ensure desired changes or problem-solving, rather output effectiveness measured by FOF should be interpreted as a precondition for achieving other forms of effectiveness, including environmental impacts. Conversely, the absence of any fitting outputs precludes any attributable and desired effects. The table below gives an approximate interpretation of FOF scores.
Accountability
An accountability index is calculated by assessing which accountability mechanisms and indicators they present. This indicators can, for example, include monitoring arrangements and the publications of annual reports, the presence of an organisational governance structure including executive and advisory bodies, whether the initiative has accountability mechanisms for their members in place, and if they have set quantified targets. Each indicator is normalised to fall between 0 and 1, and an average over all categories represent the accountability index.
Inclusiveness
The inclusiveness of an initiative is calculated by assessing how well actor types and geographic regions of leaders, executive and advisory body members, and countries of implementation is represented in the participants and involved actors. For instance, the index combines indicators such as the share of actor types in the participants of the initiative are represented in the leaders, executive, or advisory bodies, or if countries of implementation are representative of the participant or leader locations. Additionally, this indicator takes the institutional openness and the presence of non-profit and non-governmental organisations into account, Similarly to the accountability index, an average of all normalised (between 0 and 1) indicators represent the inclusiveness index.
Capacity
Finally, an index representing the capacity of an initiative is calculated using various data categories representing diverse aspects of capacity. For instance, one indicator shows if the initiative has a dedicated staff or a secretariat, while another shows the presence of funders for the initiative. The capacity index is calculated as the average of all normalised capacity indicator.
More information
For more information, download a detailed methodology on our indicators here: