Overcoming Challenges in Social Intervention Evaluation

There may be pressures, both from those involved and by the Administration, to get unbiased information quickly. This can cause problems such as calling for information at inopportune moments, conditioning the evaluator to provide information in a particular direction, or falsifying reality. This leads to a distorted view, where information is received from the appraiser in a partisan, arbitrary, and self-interested manner, solely to get information to places or persons of interest. All this can lead to an assessment where the assessor bureaucratically accepts the values of the authorities and provides information that will help them carry out their policy objectives. This also includes what McDonald called self-critical evaluation, in which evaluators provide external validation of policy in exchange for the acceptance of their recommendations.

If external evaluation is requested by those involved in social intervention, the same approaches made to the administration are valid for those involved in the program. They will make a decisive choice if they choose the team conducting the evaluation. Evaluators chosen may be related to the philosophy of the program and may well get a report conforming to their views. They may also withhold information if the assessment goes through unwanted channels, so the use of the reports should be democratic to facilitate reflection, self-criticism, and consistent decision-making.

Challenges Faced by Evaluators

The issues that face the evaluator (qualitative perspective) before, during, and after a qualitative assessment process are, according to some authors (Woods, Scriven, and SABIRON Sierra):

• The evaluator runs the risk of being used by one of the parties involved, especially if there are internal problems. With information, the assessor will become a powerful person to whom some involved may request information.
• Some involved may provide biased information or information conditioned by feelings towards the evaluator (hostility). The evaluator will have to be discreet, friendly, and intelligent.

Planning and Ethical Considerations

• The evaluator's work planning should avoid rigidity and secrecy by making clear what is meant and how. In the existing hierarchy in public intervention (vertical), the initial negotiations will remember how, when, and to whom reports will be delivered.
• Note-taking or recording will be discreet, as little as possible, not harming the spontaneity necessary for development assessment. This should be neither long nor short to avoid a superficial approach to reality.
• It is desirable that the evaluation team meet regularly to solve problems as they arise. Ethics requires data confidentiality.

Anguera analyzes the problem from the perspective of quality and links these problems to the evaluator:

1. Subjectivity, attributing their own prejudices to the group.
2. Possible feelings or lack of critical capacity.
3. Loss of group spontaneity.
4. Possible standardization by absorption of influence in the life of the group.
5. Habitual deficiencies.
6. The replicas just exist.
7. Bias may be present.

Difficulties in Internal or Self-Assessment

Internal evaluation is performed by those involved in the design and implementation of social intervention. The following problems may arise:

• Resistance of some professionals to be evaluated.
• The individualistic nature of professionals, their lack of professional motivation.
• The lack of time and lack of technical support and credibility.
• Delay of time-critical issues, concealment of substantive problems.
• Impatience for results.

Improving Objectivity in Self-Evaluation

In improving the quality of public interventions, we may find difficulties both at individual and organizational nature, to be objective in self-evaluation. Lack of motivation, poor preparation of equipment can allow for the lack of objectivity and not knowing how to work as a group are other difficulties. Another obstacle could be the tendency of institutions to ignore human issues in the initiation and organization of work.

Other barriers that must be broken, according to Simons, include the ranking that clearly makes it impossible to analyze processes at work and tend to multiply the lines of resistance, territoriality, and privacy for the professional to master their space without allowing observers to alter the normal activity it undertakes.