1. Introduction
Poles apart from its humble beginnings in the early 2000s, impact investing has since grown to gather the interest of investors globally who aim to generate positive social and environmental impact along with a financial return. As the world grapples with a myriad of challenges ranging from climate change to income inequality, the need for innovative and sustainable solutions has become more pressing than ever. With global impact investments exceeding $1 Trillion for the first time in 2022, the inclination of an ever-increasing number of investors to make choices that matter on issues that affect us all is as clear as day. The major roadblock, however, facing eager investors remains the lack of high-quality impact data to prove the effectiveness of the impact investment interventions.
The Signature Impact Framework (SIF) aims to address this issue by providing a methodology for conducting a rigorous, actionable, and cost-effective assessment of the impact of a given business. The SIF process involves three stages, allowing ex ante impact underwriting, interim impact reporting, and ex post impact additionality assessment. SIF is developed under the premise that better impact measurement allows more transparency and efficiency in impact investment decision-making. By applying the framework in the field of financial literacy, this article provides a glimpse of how SIF can be used in practice with the hope that the adoption of measurement systems such as SIF would increase confidence in impact investment opportunities, which in turn would mobilize large-scale investments from institutions that have been hesitant to embrace sustainable investments thus far.
2. Need
In 2022, the impact investing industry reached a new record in assets under management of USD 1.164 trillion (Dean Hand et al., 2022). Yet, financial literacy- an important prerequisite that should govern all financial decisions- continues to be a major challenge across the globe, but especially in the UAE. According to a Standard & Poor survey from 2015, only 38% of adults in the UAE are financially literate (Cleofe Maceda, 2015). The issue is specifically worrisome among young people, aged 18-24 years who represent roughly 10% of the overall population. According to the (Emirates Foundation for Youth Development, 2012), studies from 2011-2012 have shown that 70% of participants have no prior financial planning and 70 – 80% of debts were spent on acquiring luxuries rather than basic items. The foundation predicts that a youth debt crisis can threaten the UAE society and economy. More recently, a report by VISA highlighted that 43% of youth respondents (16-24-year-olds) believed they were not readily trained well to manage their own money. Additionally, 53% of respondents said that their current schooling did not prepare them for financial management (Adul Rawuf, 2022).
To enable an overview of possible interventions in the field of financial literacy and potentially impactful interventions, a proprietary database comprising over 250 FinLit studies was created. As of now, 100 of the studies on different financial literacy interventions have been reviewed and annotated, detailing information such as intervention type, a vector of beneficiaries, research methods, and financial literacy indicators. It was developed and refined based on a systematic review study conducted by (Miller et al., 2014). Currently, the database comprises studies from 35 nations from 1979 to 2022 which are systematically analyzed according to the authors, title, year of publication, a summary of the research, country of study, sample size, the overall impact of the intervention, subgroup impact, whether the study was a randomized controlled trial (RCT), the intensity of the intervention in terms of hours of exposure, whether there was a teachable moment, the subgroup targeted by the intervention, the type of intervention (e.g., individual counseling or classroom-based seminars), the outlet where the research was published, and the Sustainable Development Goals (SDGs) targeted by the intervention.
Despite the prevalence of these financial interventions and evidence that they impact beneficiaries’ financial literacy either objectively or subjectively in 73 out of 100 cases, the rigor of the evaluation process is highly heterogeneous. For instance, an RCT study provides a higher level of evidence compared to a study that only employed interviews and anecdotes to evaluate impacts. Additionally, studies with longitudinal elements will give investors a better understanding of whether the intervention will create long-term impacts. Therefore, besides outlining the metrics used to measure the impact of interventions on financial literacy, the database also provides information on the methodology used to evaluate impacts. Studies with a high level of evidence include those with these measurement methodologies:
1. The presence of a Randomized Controlled Trial (RCT) or quasi-experiment;
2. Comparison of pre-post intervention results through baseline & follow-up measurements; and
3. Longitudinal effects contextualized through long-term follow-ups.
The first two components allow researchers to draw difference-in-difference estimates to account for both cross-sectional and time-variant characteristics that may be relevant when measuring the relationship between intervention and impact. The third feature allows illustration of whether the interventions obtain a long-term, persistent effect on financial literacy for the participants of an intervention. Out of 100 studies, only 41 employed at least one of the above evaluation methods. Specifically, 33 studies employed RCTs, 8 studies employed difference-in-difference estimates, and 10 articles studied the longitudinal effects contextualized through long-term follow-ups.
This compendium of financial information aims to palliate the financial quandaries created by the lack of high-quality impact data to prove the effectiveness of a financial intervention. Since lack of data affects all stages of the investment process: before a transaction, when investors need information on proven interventions to identify valid targets; during the holding period, when the investors need to monitor and assess the portfolio’s social and environmental performance; and at the exit, when investors need conclusive evidence about the impact generated along with financial returns during the lifetime of the project, the need for a transparent and robust framework for evaluating the impact of an intervention is felt more deeply than ever before.
This is exactly where the Signature Impact Framework (SIF) pitches in. Developed to provide companies and investors with a blueprint to conduct a rigorous, actionable, and cost- effective assessment of the impact of a given business, the SIF allows investors to make informed decisions, reduces their due diligence costs, and most importantly helps them create constructive impact.
The key principle underlying SIF is that while the overall effects of an enterprise on society are extremely difficult if not impossible to measure, companies can demonstrate if and how they are intentionally delivering positive change along a clearly defined set of goals enshrined in their mission. We call this “Signature Impact”, which denotes the precise and distinctive contribution that a business can make to societal welfare.
3. Methodology
The SIF process involves three stages: ex ante impact underwriting, interim impact reporting, and ex post impact additionality assessment. By using the SIF, companies and investors can compare the impact of different interventions and identify best practices to adopt. The benefits of using the SIF include providing high-quality impact data to prove the effectiveness of interventions, allowing for better decision-making in impact investing, and promoting positive social and environmental change while generating financial returns.
Stage 1:
Signature Impact Underwriting (SIU)
The first stage of the SIF, termed the Signature Impact Underwriting (SIU), consists of underwriting the expected signature impact of a specific intervention. For this purpose, the theory of change (ToC) of the proposed activities is validated against the data retrieved on financial literacy interventions. The ToC (also referred to as Logic Model or Results Chain) is a hypothesis about why a desired effect is expected to materialize in a particular context. To do so, it causally links the planned activities to a desired impact.
Before laying down the ToC of a given intervention, SIU foresees a preliminary qualitative analysis of the relevance of its intervention in the potentially affected area (SDGs, region, country, community, stakeholder group). Reference is made to the extent of the problem (i.e., how many people are affected) and urgency (i.e., what the consequences of it are if it is not addressed). The analysis is currently made on a case-by- case basis based on the researcher’s individual judgment but going forward standardized brackets should be used for specific problems to allow for a quantitative assessment. The preliminary analysis is complemented by an ex ante assessment of the organization’s experience and capacity to implement the proposed intervention successfully. For example, a proven track record in similar projects contributes to obtaining better scores at the underwriting stage.
The subsequent stage of the SIU process is the identification of the ToC where the specific attributes of a proposed intervention are identified. The detailed metadata classification reported in the FinLit database allows a precise benchmarking of the intervention against the available historical evidence of achieved impacts in financial literacy programs. The expected impact is measured in terms of the success rate (namely the share of documented impactful projects over the total number of projects) by country, type of intervention and type of beneficiary. The three individual pillars will then be aggregated to generate a score measuring the joint probability of impact conditional on the country, type of intervention, and beneficiary.
Stage 2:
Signature Impact Reporting (SIR)
The second step, the Signature Impact Reporting, identifies which metrics or KPIs should be used when assessing a specific intervention and what the data shows in terms of measurement approaches used. With regards to the identifications of these impact indicators in observing specific interventions- with characteristics indicated in the SIU stage- the SIF Proprietary Database can be used to locate studies with their interested financial behavior to identify the metrics to measure the outcome of saving behavior. Indicators across 8 areas are identified, and the most frequently observed measurement metrics used in the 100 analyzed literature are consolidated.
In a similar vein to SIU, the interim reporting stage will refer to extant evidence about metrics more frequently used in benchmarked interventions. The categorization reported in the database will allow us to identify the sub-group of comparable studies and the suitable KPIs that have been more frequently used. The final choice will be made by considering the overlapping set of KPIs in similar studies, or their combination if different sets have been used. By reporting the targeted and achieved outcomes during the investment period, relative impact performance against the set objectives can be calculated and used to monitor the execution of the project.
Stage 3:
Signature Impact Additionality Assessment
In an ideal setting for impact measurement, interventions are assessed by running a randomized controlled experiment with at least two groups: a group of randomly selected beneficiaries receiving the intervention (the treatment group), and a control group that did not. By comparing the observed changes before and after the treatment within these two groups, these experiments called Randomized Control Trials (RCTs) allow us to quantify the impact an intervention achieves. However, RCTs are very costly, time- intensive and logistically challenging. For the ex post assessment of impact, we thus recommend quasi-experimental methods that do not involve randomization but still use counterfactuals.
One widely used approach is the difference-in-difference (Diff-in-Diff) methodology, where a comparison is made with a similar population, namely one that is not offered the intervention but is receiving “treatment as usual”. Both groups receive pre- and post-assessments, and the difference between those assessments is used to determine the impact of the new intervention. A Diff-in-Diff test is aimed at estimating the causal effects of a given intervention on one or more impact KPIs identified in the SIP stage and will ultimately allow us to assess impact additionality.
The concept of additionality has been initially developed in development finance to identify projects and investments that would have not occurred absent the specific intervention of institutions such as the World Bank, the European Investment Bank, and other Development Finance Institutions. Indeed, one of the key objectives of these multilateral financial institutions is to tackle market failures and provide capital often at concessionary terms to unbankable projects and avoid the crowding out of private investments. In the impact investing field, impact additionality refers to whether the target outcomes would have occurred anyway, without the investment underlying the intervention. For this reason, the quasi-experimental test that we envisage represents a powerful tool for the Signature Impact Additionality Assessment (SIAA). Our analysis recommends the Diff-in-Diff approach at the SIAA stage in the context of financial literacy. This is because the SIAA stage allows the investor to measure with a high level of confidence whether the intervention achieved its goal and the extent of impact achieved. The investor could then use this data in combination with financial returns to assess the economic and social performance of the investee and identify possibly risk-return-impact trade-offs of its investment strategy.
Under some circumstances, SIAA will deliver impact measures that can be monetized. Investors could then use the actual investment amount to compute the impact multiple of money of social return on investment. SIAA does not always deliver a monetary estimate of impact. However, in combination with the SIR analysis, it could provide a consistent set of estimated KPIs that would allow a meaningful comparison of impact achieved across projects in the same vertical.
4. Conclusion and the Way Forward
While the evaluation of impacts in impact investing is a complex process that requires careful consideration of various factors, a data-driven, rigorous approach such as the Signature Impact Framework (SIF) can help to assuage many of the risks associated with any financial decision, particularly so in the impact investment space. The SIF has been piloted in the field of financial literacy, helping extend the knowledge and understanding of financial concepts and risks, and the skills, motivation, and confidence to apply these in order to make effective decisions across a range of financial contexts. The SIF has been used to assess the impact of interventions aimed at improving financial literacy in the United Arab Emirates (UAE). By using the SIF, researchers were able to identify best practices for improving financial literacy and measure the effectiveness of different interventions. This information can be used to inform future interventions and improve financial well-being for individuals and society. By following the three-stage process proposed by SIF, investors can predict and quantify the potential impacts of their portfolio, track the progress of their start-ups, and evaluate the impacts achieved over a predefined period. To further enhance the impact evaluation process, investors can use a proprietary database that is specific to a particular industry or vertical. This can help benchmark different companies within that industry and conduct metadata and statistical analyses to evaluate the impact of investments in that industry. The results of these analyses can provide investors with a broader understanding of the impact of their investments and enable them to make more informed decisions about where to allocate their resources in the future.
However, there are still challenges that must be addressed, such as the lack of high-quality data in most MEASA markets. To tackle this issue, investors should encourage investee firms to collect data on agreed-upon key performance indicators (KPIs), obtain third-party verification, and commit to conducting Social Impact and Additionality Assessment (SIAA) with appropriate control groups. Moreover, data comparability can also pose a challenge, as each startup and firm has its own economic and social agendas and unique methods to deliver services and impacts. Standardization may not capture the nuances of these differences, leading to inaccurate assessments, even within the same industry.
Despite its limits, the proposed framework has distinctive features such as the extensive benchmarking to existing reported evidence and the rigorous attempt to identify the genuine additionality of interventions. By extending the SIF beyond the financial literacy sector to other industries, such as healthcare and environment protection, SIF can be streamlined and back tested, facilitating the quality of impact investment globally and contributing to positive social and environmental change.