Control Groups - Revision history

Wiki Gardener at 16:02, 6 October 2014

2014-10-06T16:02:58Z

Ranisha at 12:16, 14 August 2014

2014-08-14T12:16:17Z

Wiki Gardener at 10:23, 8 August 2014

2014-08-08T10:23:52Z

Ranisha at 14:49, 22 July 2014

2014-07-22T14:49:49Z

Ranisha at 14:24, 22 July 2014

2014-07-22T14:24:15Z

Wiki Gardener at 13:09, 4 June 2014

2014-06-04T13:09:04Z

Katharina Wiedemann at 10:00, 17 May 2012

2012-05-17T10:00:42Z

Katharina Wiedemann at 09:59, 17 May 2012

2012-05-17T09:59:01Z

Katharina Wiedemann at 09:58, 17 May 2012

2012-05-17T09:58:25Z

Katharina Wiedemann at 09:56, 17 May 2012

2012-05-17T09:56:54Z

@@ Line 2: / Line 2: @@
 = Overview<br/> =
-The term '''control group''' is used when the evaluation employs an [[Control Groups#Attribution Problem|experimental design]] and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias<ref name="Word Bank - http://bit.ly/1oLVNIX">Word Bank - http://bit.ly/1oLVNIX</ref>.<br/>
+The term '''control group''' is used when the evaluation employs an [[Control Groups#Attribution Problem|experimental design]] and the term comparison group is associated with a [[Quasi-Experimental_or_Non-Experimental_Designs|quasi-experimental design]]. In [[Quasi-Experimental_or_Non-Experimental_Designs|non-experimental design]], program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of [[Selection_Bias|selection bias]]<ref name="Word Bank - http://bit.ly/1oLVNIX">Word Bank - http://bit.ly/1oLVNIX</ref>.<br/>
-Control groups are included in evaluation designs in order to estimate the impact of a development measure.
+Control groups are included in evaluation designs in order to estimate the [[Portal:Impacts|impact]] of a development measure.
 <br/>
@@ Line 47: / Line 47: @@
 <br/>
-== Importance of Counterfactual Analysis<br/> ==
+= Importance of Counterfactual Analysis<br/> =
 Proper analysis of the attribution problem is to compare the situation ‘with’ an intervention to what would have happened in the absence of an intervention, the ‘without’ situation (the ''counterfactual''). Such comparison of the situation ''“with and without” ''the intervention is challenging since it is not possible to observe how the situation would have been without the intervention, and has to be constructed by the evaluator.
@@ Line 71: / Line 71: @@
 <span><span>[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span></span>
 [[Category:Questionnaires_/_Interviews]]

@@ Line 2: / Line 2: @@
 = Overview<br/> =
-The term '''control group''' is used when the evaluation employs an [[Control Groups#Attribution Problem|experimental design]] and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias<ref name="Word Bank - http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:384339~pagePK:148956~piPK:216618~theSitePK:384329,00.html">Word Bank - http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:384339~pagePK:148956~piPK:216618~theSitePK:384329,00.html</ref>.<br/>
+The term '''control group''' is used when the evaluation employs an [[Control Groups#Attribution Problem|experimental design]] and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias<ref name="Word Bank - http://bit.ly/1oLVNIX">Word Bank - http://bit.ly/1oLVNIX</ref>.<br/>
 Control groups are included in evaluation designs in order to estimate the impact of a development measure.
@@ Line 56: / Line 56: @@
 *[[Portal:Impacts|Impact Portal on energypedia]]<br/>
-*[[Impact_Monitoring_Guidelines|Impact Monitoring Guidelines]]<br/>
+*[[Impact Monitoring Guidelines|Impact Monitoring Guidelines]]<br/>
 *[[Guidelines for Sampling/ Selection Criterias|Guidelines for Sampling / Selection Criterias]]<br/>
 *[http://siteresources.worldbank.org/EXTOED/Resources/nonie_guidance.pdf Impact Evaluations and Development. Nonie Guidance on Impact Evaluation], [http://siteresources.worldbank.org/EXTOED/Resources/nonie_guidance.pdf Leeuw, F. & Vaessen, J. (2009)]<br/>
 *[https://energypedia.info/Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html W]<span><span>[http://bit.ly/1ryaTIW orld Bank –Impact Evaluation (2008][http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html )]</span></span><br/>
+<br/>
 = References<br/> =
@@ Line 69: / Line 69: @@
 <br/>
-<span><span>[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span> </span>
+<span><span>[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span></span>
 [[Category:Impacts]]
 [[Category:Questionnaires_/_Interviews]]

@@ Line 2: / Line 2: @@
 = Overview<br/> =
-The term '''control group''' is used when the evaluation employs an [[Control Groups#Attribution Problem|experimental design]] and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias (see World Bank 2008).
+The term '''control group''' is used when the evaluation employs an [[Control Groups#Attribution Problem|experimental design]] and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias<ref name="Word Bank - http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:384339~pagePK:148956~piPK:216618~theSitePK:384329,00.html">Word Bank - http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:384339~pagePK:148956~piPK:216618~theSitePK:384329,00.html</ref>.<br/>
 Control groups are included in evaluation designs in order to estimate the impact of a development measure.
@@ Line 8: / Line 8: @@
 <br/>
-The main challenge in determining impacts results from the fact that it is impossible to observe, how people undergoing an intervention would have acted if this intervention had not occurred. Only a well designed form of this so-called allows for robust statements about the impact that is actually attributable to the intervention.<ref name="Leeuw & Vaessen: 2009">Leeuw & Vaessen: 2009</ref>
+The main challenge in determining impacts results from the fact that it is impossible to observe, how people undergoing an intervention would have acted if this intervention had not occurred. Only a well designed form of this so-called allows for robust statements about the impact that is actually attributable to the intervention.<ref name="Leeuw, F. & Vaessen, J. (2009): Impact Evaluations and Development. Nonie Guidance on Impact Evaluation - http://siteresources.worldbank.org/EXTOED/Resources/nonie_guidance.pdf">Leeuw, F. & Vaessen, J. (2009): Impact Evaluations and Development. Nonie Guidance on Impact Evaluation - http://siteresources.worldbank.org/EXTOED/Resources/nonie_guidance.pdf</ref><br/>
-In general, evaluation is concerned with measuring the effects of an intervention (or ‘treatment’) such as a training course or electrification. The determination of the ''true'' effects of electrification on, for example, income would require comparing the ''outcome variable'' income after the household has received the treatment to the situation of not having received it. The fundamental problem here is that households can never be observed in the situation of not-having received electrification if they are in fact electrified. To solve this, the unobservable situation of not being electrified has to be replaced in the evaluation design by a group of non-electrified households, the ''control group''. The control group is hence included in order to simulate how the '''outcome variable '''would have developed if the intervention would not have taken place.
+In general, evaluation is concerned with measuring the effects of an intervention (or ‘treatment’) such as a training course or electrification. The determination of the true effects of electrification on, for example, income would require comparing the outcome variable income after the household has received the treatment to the situation of not having received it. The fundamental problem here is that households can never be observed in the situation of not-having received electrification if they are in fact electrified.<br/>
 <u>In order to effectively fulfill this task, this control group should be composed of individuals that</u>
@@ Line 19: / Line 23: @@
 <br/>
-Often the non-connected households in a target region of an electrification project are taken as a control group. This group, though, is generally not able to simulate the connected household’s behavior in the hypothetical non-electrified scenario. The reason is that the two groups tend to show differences in education, income, motivation and other household characteristics. For example, richer households are more inclined to get connected. Including the non-connected households as control group in an electrification evaluation would therefore be deficient in the sense that the estimated effect of the intervention would be biased by these differences. A second obvious comparison group is the beneficiary group itself at the point of time before the intervention – when these households have not yet been electrified. The problem here is that it can rarely ruled out that other environmental factors such as the general economic development have changed. As a consequence, it cannot be separated which part of the change in outcomes can be attributed to the electrification intervention and which part to the environmental change.
+Often the non-connected households in a target region of an electrification project are taken as a control group. This group, though, is generally not able to simulate the connected household’s behavior in the hypothetical non-electrified scenario. The reason is that the two groups tend to show differences in education, income, motivation and other household characteristics. For example, richer households are more inclined to get connected. Including the non-connected households as control group in an electrification evaluation would therefore be deficient in the sense that the estimated effect of the intervention would be biased by these differences.<br/>
 The most promising approach to select a control group is to include a '''comparable region '''into the study design that has not been and will not be electrified. This allows comparing the changes in both regions before and after electrification, excluding both biases from changing environmental factors and differences between individual households. The selection of such regions, though, requires some methodological skills and field experiences. In some cases it might also not be possible to find comparable regions that will not be electrified or treated by any other development intervention.
@@ Line 27: / Line 33: @@
 = Options for Choice of Control Region<br/> =
-The obvious counterfactual situation for the stove and electrification projects would be Option A – a region where these interventions have not been taken place before. However, Option B provides as well for a viable counterfactual. While Option A investigates in how far the project sites outperform the control regions, Option B explores in how far the project sites catch up in relation to the control regions. The crucial point is that between the before and the after survey, no such intervention takes places. At this point, one disadvantage of Option A becomes evident: It is often not for sure that during that time period the control region actually remains non-electrified or without improved stoves interventions respectively. It has already happened that unforeseen activities by other donors or partner country institutions in the course of the project period rendered the implementation of this approach impossible. On the other hand, Option A has the advantage that it can serve as an “ex-ante impact assessment”. This implies that the information gathered in the control regions helps to anticipate the beneficiary’s behaviour, potential difficulties and opportunities to be expected in the wake of the EnDev project before project implementation. These findings can then potentially be fed back into the project design.
+<u>The obvious counterfactual situation for the stove and electrification projects would be:</u><br/>
 <br/>
@@ Line 45: / Line 55: @@
 = Further Information =
-*[[Portal:Impacts|Portal:Impacts]]
+*[[Portal:Impacts|Impact Portal on energypedia]]<br/>
-*[[Guidelines_for_Sampling/_Selection_Criterias|Guidelines for Sampling/ Selection Criterias]]<br/>
+*[[Impact_Monitoring_Guidelines|Impact Monitoring Guidelines]]<br/>
-*EnDev2 Evaluation & Monitoring Modules / RWI (2009).
+= References<br/> =
-*Leeuw, F. & Vaessen, J. (2009): Impact Evaluations and Development. Nonie Guidance on Impact Evaluation. Draft Version for Discussion at hte Cairo conference March-April, 2009.
-*[[Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html|W]]<span><span>[http://bit.ly/1ryaTIW orld Bank –Impact Evaluation (2008][http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html )].</span></span>
-*Impact Evaluations and Development: NONIE Guidance on Impact Evaluation 2009: URL:[http://www.worldbank.org/ieg/nonie/guidance.html http://www.worldbank.org][http://www.worldbank.org/ieg/nonie/guidance.html /ieg/nonie/guidance.html] 02/11/2009.
-<references /><br/>
+<span><span><references /></span></span><br/>
 <br/>
-<span style="font-weight: normal; font-size: 10pt; font-family: 'arial','sans-serif'; mso-bidi-font-weight: bold">[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span>
+<span><span>[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span> </span>
 [[Category:Impacts]]

@@ Line 45: / Line 45: @@
 = Further Information =
+*[[Portal:Impacts|Portal:Impacts]]
 = References =

@@ Line 51: / Line 51: @@
 *EnDev2 Evaluation & Monitoring Modules / RWI (2009).
 *Leeuw, F. & Vaessen, J. (2009): Impact Evaluations and Development. Nonie Guidance on Impact Evaluation. Draft Version for Discussion at hte Cairo conference March-April, 2009.
-*[[Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html|W]]<span><span>[[Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html|orld Bank –Impact Evaluation (2008]][http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html )].</span></span>
+*[[Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html|W]]<span><span>[http://bit.ly/1ryaTIW orld Bank –Impact Evaluation (2008][http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html )].</span></span>
 *Impact Evaluations and Development: NONIE Guidance on Impact Evaluation 2009: URL:[http://www.worldbank.org/ieg/nonie/guidance.html http://www.worldbank.org][http://www.worldbank.org/ieg/nonie/guidance.html /ieg/nonie/guidance.html] 02/11/2009.
 <references /><br/>
 <span style="font-weight: normal; font-size: 10pt; font-family: 'arial','sans-serif'; mso-bidi-font-weight: bold">[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span>
-<span style="font-weight: normal;  font-size: 10pt;  font-family: 'arial','sans-serif';  mso-bidi-font-weight: bold">[[Impact Monitoring Guidelines|--> Back to Impact Monitoring Guidelines]]</span>
 [[Category:Impacts]]

← Older revision		Revision as of 10:00, 17 May 2012
Line 41:		Line 41:

	<span style="font-size: 10pt"><font face="Arial">Proper analysis of the attribution problem is to compare the situation ‘with’ an intervention to what would have happened in the absence of an intervention, the ‘without’ situation (the ''counterfactual''). Such comparison of the situation ''“with and without” ''the intervention is challenging since it is not possible to observe how the situation would have been without the intervention, and has to be constructed by the evaluator.</font></span>		<span style="font-size: 10pt"><font face="Arial">Proper analysis of the attribution problem is to compare the situation ‘with’ an intervention to what would have happened in the absence of an intervention, the ‘without’ situation (the ''counterfactual''). Such comparison of the situation ''“with and without” ''the intervention is challenging since it is not possible to observe how the situation would have been without the intervention, and has to be constructed by the evaluator.</font></span>
−

@@ Line 3: / Line 3: @@
 = Introduction<br/> =
-The term '''control group''' is used when the evaluation employs an [[Control_Groups#Attribution_Problem|e]]'''[[Experimental design|xperimental design]]''' and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias (see World Bank 2008).
+The term '''control group''' is used when the evaluation employs an [[Control_Groups#Attribution_Problem|experimental design]] and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias (see World Bank 2008).
 Control groups are included in evaluation designs in order to estimate the impact of a development measure.
@@ Line 24: / Line 24: @@
 The most promising approach to select a control group is to include a '''comparable region '''into the study design that has not been and will not be electrified. This allows comparing the changes in both regions before and after electrification, excluding both biases from changing environmental factors and differences between individual households. The selection of such regions, though, requires some methodological skills and field experiences. In some cases it might also not be possible to find comparable regions that will not be electrified or treated by any other development intervention.

@@ Line 3: / Line 3: @@
 = Introduction<br/> =
-The term '''control group''' is used when the evaluation employs an [[Control_Groups#Attribution_Problem|'''experimental design''' ]]and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias (see World Bank 2008).
+The term '''control group''' is used when the evaluation employs an [[Control_Groups#Attribution_Problem|e]]'''[[Experimental design|xperimental design]]''' and the term comparison group is associated with a quasi-experimental design. In non-experimental design, program participants are compared to non-participants by controlling statistically for differences between participants and non-participants. These three evaluation designs vary in feasibility, cost, the degree of clarity and validity of results, and the degree of selection bias (see World Bank 2008).
 Control groups are included in evaluation designs in order to estimate the impact of a development measure.
@@ Line 33: / Line 33: @@
-= Attribution Problem<ref>Impact Evaluations and Development: NONIE Guidance on Impact Evaluation 2009: URL:http://www.worldbank.org/ieg/nonie/guidance.html 02/11/2009.</ref><br/> =
+= Attribution Problem<br/> =
 To what extent can results of interest be attributed to an intervention? <span style="font-size: 10pt"><font face="Arial">The attribution problem is often referred to as the central problem in impact evaluation. The central question is to what extent can changes in outcomes of interest be ''attributed ''to a particular intervention? Attribution refers both to isolating and measuring accurately the particular contribution of an intervention and ensuring that causality runs from the intervention to the outcome.</font></span>
@@ Line 53: / Line 53: @@
 [[Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html|<span>''<span><font color="#002bb8">W</font></span>''</span>]]<span>''<span>[[Web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242~menuPK:415130~pagePK:148956~piPK:216618~theSitePK:384329,00.html|<font color="#002bb8">orld Bank –Impact Evaluation (2008</font>]][http://web.worldbank.org/WBSITE/EXTERNAL/TOPICS/EXTPOVERTY/EXTISPMA/0,,contentMDK:20188242%7EmenuPK:415130%7EpagePK:148956%7EpiPK:216618%7EtheSitePK:384329,00.html <font color="#002bb8">)</font>].</span>''</span>
-'''''<span style="font-weight: normal;  font-size: 10pt;  font-family: 'arial','sans-serif';  mso-bidi-font-weight: bold"></span>'''<span style="font-weight: normal;  font-size: 10pt;  font-family: 'arial','sans-serif';  mso-bidi-font-weight: bold"></span>''
+'''''<span style="font-weight: normal;  font-size: 10pt;  font-family: 'arial','sans-serif';  mso-bidi-font-weight: bold">Impact Evaluations and Development: NONIE Guidance on Impact Evaluation 2009: URL:</span>'''<span style="font-weight: normal;  font-size: 10pt;  font-family: 'arial','sans-serif';  mso-bidi-font-weight: bold">[http://www.worldbank.org/ieg/nonie/guidance.html http://www.worldbank.org/ieg/nonie/guidance.html] 02/11/2009.</span>''