GUIDELINES FOR EVALUATION OF
CONTINUING EDUCATION PROGRAMMES


The impact of literacy and adult education programmes clearly shows that in the future, skill development and increased economic opportunity must form one of the major thrust areas. The challenge before the National Literacy Mission is to create a continuing education system where the effervescence of the mass upsurge of the literacy campaigns can be channeled into structuring a continuous and life-long learning process.

In the post literacy and continuing education stages, greater emphasis is being placed on skill development and acquisition of new learning. For those who have acquired basic literacy skills, we need to link these skills more intricately with their lives. This can only become a reality when they learn not only to practice these skills in their day-to-day lives, but also clearly understand that these skills will be of vital importance to them in order to improve the quality of their existence.

The continuing education scheme is postulated on the principles of:

  • Treating basic literacy, post literacy and continuing education as one sustained, coherent learning process.
  • Establishing a responsive and alternative structure for life-long learning.
  • Responding to the needs of all sections of society.
  • Learning not to be seen as a function of alphabets, but as all modes of human capacity building.
  • Addressing the socio-economic situations of the community to provide infrastructure for larger development initiatives.

Thus, the scheme of continuing education, taken up in a district after it has completed the total literacy and post literacy phases, makes the learners aware of the power and significance of education. They realise that education is the agency for improving their lives and they tend to find ways to use their literacy skills in their everyday life to make it more meaningful and rewarding. The continuing education scheme is, therefore, multi-faceted and enjoys suprem flexibility to allow grassroot community participation and managerial initiative.

Establishment of continuing Education Centres (CECs) and Nodal Continuing Education Centres (NCESs) is the principal mode of implementing continuing education programmes. The centres follow an area-specific, community-based approach. The scheme envisages one CEC for each village to serve a population of about 1,500-2,000 people. 10 such centres form a cluster. A separate Nodal Continuing Education Centre will supervise, monitor and coordinate the activities of the cluster.

The centres are run by facilitators or preraks, who are drawn from the community itself. The continuing education centre serves as:

  • Library and reading room
  • Teaching-learning centre for continuing education programmes
  • Vocational training centre
  • External centre for facilities of other development departments
  • Discussion forum for sharing ideas and solving problems A composite information window for the community
  • Cultural centre
  • Sports and recreation centre

The CECs, including the nodal one, are set up in active consultation with the user community and its programme is designed to meet their demands. The stress on imparting literacy skills to non-literates is sustained. Teaching of primers, identification of target groups, environment-building activities, and other items of work which were connected with illiteracy eradication continues unabated.

Wide acceptance and local sustainability is achieved by involving NGOs, voluntary agencies, social workers, panchayati raj institutions in the planning and implementation of the scheme of continuing education. Various development departments, cooperative societies, technical institutions and professional groups provide inputs needed by the programmed. State Resource Centers and Jan Shilshan Sansthans join hands by giving the necessary resource and training support.

Apart from establishing CECs, the scheme also undertakes the following programmes: Equivalency programme designed as an alternative education programmed equivalent to existing formal, or vocational education.

Income-generating programme where the participants acquire or upgrade their vocational skills and take up income-generating activities.

Quality of life improvement programme which aims to equip learners and the community with essential knowledge, attitude, values and skills to raise their standard of living.

Individual interest promotion programme to provide opportunities for learners to participate and learn about their individually chosen social, cultural, spiritual, health, physical and artistic interests.

The project for a continuing education programmed is formulated by the Zilla Saksharta Samiti taking the district as a unit. The proposal is submitted for approval to the State Literacy Mission Authority of the concerned State or to the National Literacy Mission in case the district falls under a union territory.

While the central government does provide financial assistance for initial establishment and running of CECs, in the long run all such CECs are expected to become self-sustaining. The scheme, therefore, envisages provision of financial assistance from the central government and the state governments. The central government provides 100 per cent financial assistance to a district for the first 3 years. Thereafter, the cost is to be shared on a 50:50 basis by the central and state governments for the next 2 years. From the 6th year onwards, States are expected to bear full costs.

Clearly, the effectiveness of a CEC and the scope of its activities will be significantly determined by the extent of support it enjoys from the community. The CECs must be perceived by the people as arising from their won initiatives to meet their explicit needs. To achieve this objective, the ZSS will be expected to devise all possible ways to enlist community support and mobilize financial and material resources from the community itself so that the CECs and their programmes become self-sufficient in the course of time.

The continuing education scheme is visualized as a mechanism for integrated human resource development. A well-organised and coordinated infrastructure for continuing education will advance the cause of literacy in India leading to the creation of a learning society.

Evaluation Systems under NLM

There is a growing demand for monitoring and evaluation based on critical performance indicators of development programmes all over the world. There is a broad based expectation that evaluation, information and analysis will increasingly shape improvements in programmes and projects as well as broaden learning at the policy level. Recognizing this, the National Literacy Mission has devised on elaborate system of evaluation covering all phases of the literacy programmed, namely, Total Literacy Campaign (TLC), post Literacy Programmed (PLP) and continuing Education Programmed (CEP).

Self Evaluation by Learners
Each of the three primers used for teaching the 3 R's top the learners during TLC contain three tests. The learners get a fair idea of their learning strengths and weaknesses when they attempt these tests. The volunteer teachers are expected to ensure that the learners take these tests and use them as learning tools.

Internal Evaluation
The records of the terminal tests in each primer T3, T6 and T9 are supposed to be maintained at the village/Panchayat level. This would be termed Internal Evaluation. The ZSS may develop test papers based on the Dave Committee recommendations and carry out internal evaluation at various stages of the Literacy Phase to get proper feed back of the learning strengths and weaknesses among the adult learners.

External Evaluation
Beside the self evaluation by the learners and internal evaluation of the campaign every campaign district is subject to 2 more evaluations, namely, concurrent evaluation (process evaluation) which is carried out by the agencies within the State and final evaluation (summative evaluation) which is carried out by agencies outside the State.

(a) Concurrent Evaluation
The concurrent evaluation focuses on studying the process of implementation, detecting bottlenecks, shortfalls and deficiencies and suggesting corrective measures. Concurrent evaluation is carried out in 2 stages.

(b) Final Evaluation
Final evaluation of a TLC is carried out by an empanelled agency outside the State. The ZSS initiates the process when, in its estimation, about 60% of the targeted learners have completed/almost completed primer III. The final evaluation mainly focuses on learning outcomes. The main purpose is to provide on objective and reliable assessment of the success rate vis--vis the target taking into consideration the Dave Committee recommendations.

Evaluation System for PLP
The evaluation of PLP aims at studying the extent to which PLP objectives, both at the individual and tat the community level have been achieved. It assesses the extent to which requisite activities have been under taken along with their effectiveness. It also provides valuable feedback to the ZSS in order to overcome shortcomings and bottlenecks that may be identified. Finally, it evaluates the preparedness of the ZSS for the CE Programmed. PLP evaluation is carried out in 2 stages-mid-term and final for districts, which have a 2-year duration. In the case of projects sanctioned for one year, the objectives of mid-term evaluation are merged and only one final evaluation is carried out.

Evaluation System for Continuing Education
As the name suggests, this is a continuing programmed with no fixed time period. However, in practical terms the CEP is being financed by the central Government for the first three years. Cost sharing between State and Central Government is on a 50:50 basis for 2 years thereafter. From the 6th year onwards, State Governments would be bearing full costs.

There is a lot of flexibility in the CEP to cater to local needs and, therefore, the proposal presented by the ZSS and approved by the SLMA for CE in a district should serve as the major reference material for evaluation apart from the CE document prepared by the NLM.

External evaluations by NLM and Appraisals by SLMA/SDAE are to be carried out as per the following timetable:

After 2 years		External Evaluation
After 3 years		Appraisal Report by SLMA/SDAE
After 4 years		External Evaluation
After 5 years		Appraisal Report by SLMA/SDAE
After 7 years		External Evaluation
Thereafter, SLMA/SDAEs and NLM will conduct evaluations in their own three year cycles.

Aspects to be covered in Evaluation of CEP

8 aspects are listed below which should be covered in the evaluation of a CEP. These cover all the functions specified in the policy document. Since these will be covered by the ZSS over a period of time there has to be a specific focus in various evaluations and appraisals to be carried out. The objective of evaluation of a CEP is to assess the actual performance in relation to what was proposed in the project submitted by the ZSS and approved by the SLMA/NLM in respect of each of these 8 aspects.

1. Establishment of Continuing Education Centers

  • Locational Appropriateness
  • Numerical Adequacy
  • Physical Structure
  • Furniture and Equipment
  • Books and Reading Material
  • Personnel and their Training
  • Any other items/assets
2. Core Functions of Nodal and CE Centers
  • Support Functions of Nodal Centers
  • Library
  • Reading Room
  • Information Center
  • Sports Center
  • Target specific Functional Programmes
3. Equivalency Programmes
  • Types of programmes taken up
  • Collaborative efforts by ZSS with State Resource Centers
  • Networking with the NOS/SOS/NGOs
  • Certification
  • Benefits accrued to the learners
4. Income Generating Programmes
  • Selection of vocations/trades/skills
  • Whether need-based
  • Method of selection
  • Delivery system: Feed back from beneficiaries
  • Networking and coordination mechanism with various development agencies
  • Benefits accrued to the learners
5. Quality of Life Improvement Programmes (QUIPs)
  • Enlisting the educational programmes taken up by ZSS under various quality of life indicators
    (Family Welfare, Health, Nutrition, Drinking Water, Sanitation, Population Education, Environment, Social Services, Communal Harmony, Transportation etc.)
  • Delivery system of education programmes
  • Presentation methods
  • 6. Individual Interest Promotion Programmes (IIPS)
  • Methods of identification of interested individuals
  • Methods of knowledge individual interest areas
  • Resource information on identified interest areas
  • Efforts made to cater to the needs of the identified areas
  • Facilities/guidance provided and for how many networking and coordination of relevant agencies
  • Feedback from beneficiaries
  • 7. Performance of Mopping up Operation
    • Door to door Resurvey by ZSS
      (a) Methodology: What, How, When, By whom
      (b) Lists of learners under mopping up
    • What was mopping up target?
      (a) unenrolled non-literates
      (b) Drop-outs during TLC/PL
      (c) learners still to complete P-III & those who could not achieve NLM norms
      (d) New entrants
    • Efforts made to cover the above four categories
    • Arrangements for teaching learning
    • Availability of primers
    • Testing learners on sample basis
    • Estimation of district level achievement on the basis of above testing
    • How many reached NLM norms?
    • How many still to reach?
    8. Other Aspects
    • Mahila Mandals
    • Yuvak Mandals
    • Women's Thrift Groups
    • Volunteer Groups
    • Cultural Groups
    • Other fall outs
    Focus in various Evaluations and Appraisals The focus of the three external evaluations should be as follows:
    1st Evaluation : Administrative and organizational aspects
    2nd Evaluation : Programmed implementation and processes
    3rd Evaluation : Sustainability of CE along with programme implementation and processes
    The 1st and 2nd appraisals to be carried out by SLMA/SDAE should have foci similar to the 2nd and 3rd External Evaluations respectively.

    Statistical Profile It is suggested that a brief statistical profile may be prepared in the beginning by the agency. This may include:
    (a) Information on Literacy Programmed

      Output of PLP
    • Coverage numerical, M/F SC/ST
    • Time gap between PLP and CE
      • (b) Health/ Demographic Parameters
      • Birth Rate/Death Rate/Still Birth incidence
      • Infant Mortality Rate
      • Incidence of common diseases
      • Epidemic occurrence, if any
      • Sex Ratio
      (c) Education Data Mainly on Primary education, enrollment, drop out rates, etc.

      Important points to be borne in mind by the Evaluators
      Certain important points are listed here which should be borne in mind by the evaluators while planning the evaluation. These points will have a bearing on the methodology in terms of coverage of aspects, sampling, tools etc.

      1. CE Project proposal of ZSS: The proposal presented by ZSS and approved by SLMA/NLM shall serve as the major reference material for evaluation apart from the CE documents of NLM

      2. The districts should be encouraged to come out with their successes and failures as well as their special efforts for briefing the external agencies. The agencies should try to have as much dialogue as possible with ZSS functionaries.

      3. The assessment of programmed benefits to the learners/participants/beneficiaries should be an important part of the evaluation.

      4. The attitudinal and behavioral aspects can be evaluated but cognitive and educational delivery systems are to be focused on.

      5. Flow of funds and their utilization as well as the needs of the project should be studied (This should be an evaluation exercise and not an audit exercise).

      Tools for Evaluation
      The following is the broad list of tools required for evaluating the CEP programmes:

      1. Schedule to record information on CE from the ZSS. Statistical/dates/activities/special features/financial flows/ other items.

      2. Questionnaire for interviewing the beneficiaries/learners

      3. Interview guide for discussions with the functionaries at various levels.

      4. List of points for focus group discussion with the beneficiaries of various programmes (Note on focus group discussion enclosed at the end).

      5. Literacy test for learners (P-III) under mopping up operation. A separate chapter is devoted for each of the three external evaluations to be carried out by the external agencies. In these chapters more detailed material is presented. The time frame for completing the external evaluations is given below. Time Frame
      The following time frame is suggested for the three external evaluations:

      1st (after 2 years)		:	45-60 days
      2nd (after 4 years)		:	60-90 days
      3rd (at the end of 7 years)	:	60-90 days
      

      First External Evaluation:
      After Two Years*

      It is most likely that the ZSS spends the first one or two years of the continuing education programmed in administrative and organizational work such as locating the CE centers and Nodal CE centers, obtain the space for the centers, procuring the furniture and other material, purchase of books, selection of preraks and nodal preraks and training them etc. A question may be asked as to why external evaluation at the end of the first 2 years since actual CE activities may not have yet started. The answer is that these preparations are very important. The creation of physical and personnel infrastructure is crucial for running the CE Programmed successfully. Hence it is useful to have a third party assessment of this.

      Location Appropriateness and Numerical Adequacy It will be useful to study whether the CE centers and Nodal CE centers have been provided in the district as per the norms and guidelines given by the NLM, whether they are adequate in number and are located appropriately so that the participants can reach them conveniently. Though these aspects are to some extent scrutinized by the NLM while sanctioning the CE project, it is necessary to study these aspects in more detail by the external agency. If there is any lacuna, improvements could be suggested and ZSS could implement the suggestions.

      This is a macro exercise and the plan for the entire district should be studied in the light of the guidelines of CE scheme. Suggestions for improvement should be specific and should be reasoned out.

      On the other points listed below the evaluation will have to be carried out on sample basis.

      Sample Design and Size
      Fifty continuing education centers and 5 connected nodal CE centers should be covered in the sample to study the following points. The centers should be randomly chosen so as to represent the entire district. For this purpose, random selection from each block or region could be done and this could be in 2 stages - village and the center.

      A list of connected nodal centers of the above 50 CE centers should be prepared and 5 nodal centers among them should be selected as the sample by random method.

      The above sample should be used for studying physical structure., furniture and equipments, books and reading material, personnel and their training and any other items/assets specific to the district. Certain indications are suggested below for each of these items.

      Indicators/points for study of CE and Nodal CE physical structure

  • Was people's participation sought?
  • Does it have sufficient space for stocking and displaying books and other material, for 'Charcha Mandal' meetings, reading space, etc?
  • Are basic facilities available like light, water and toilet etc?
  • Is it secure for the women to come in?
  • Are the name board, charts, wall hangings, and display materials properly placed?
  • Furniture and Equipments
    The CE scheme booklet of NLM gives the list of furniture and equipment to be provided to the centers. It should be verified as to whether all the furniture and equipment are physically present at the centers and whether they are used for the centers. Adequacy of these may also be studied with a view to suggesting modifications in the CE scheme itself. Most general furniture equipment items are almirah, petromax, ground-table for putting newspapers, Roller board etc. Sports items and recreational material are also provided along with a bicycle.

    Books and Reading Material?
    There is a provision for purchase of books, newspapers and periodicals. It will be necessary to find out whether the grants have been utilized by the CE and Nodal CE centers, whether the books are possessed by the centers, whether the newspapers and periodical are being kept for reading at these centers and so on.

    It is not possible to undertake stock checking of the books but the evaluators should properly observe and enquire so that he/she can judge whether the books are in possession of the center or are in circulation among the readers. There is a need for qualitative assessment of the books and materials. The following points should be covered while evaluating material.

    • Content/type
    • Simplicity of language/graded material
    • Print type/size
    • Size of books/material
    • Pictures/illustrations

    Since there is a possibility that most of the materials will be common the above evaluation does not have to be carried out for each of the 55 centers in the sample. The evaluator should see that the quality of all books and materials are evaluated. This work should be carried out by an expert on material preparation. The report should clearly indicate inappropriate material giving explanations and pointing out lacunae. Personnel and their training
    For covering this, the Preraks and nodal preraks in the sample centres should be interview by using a questionnaire. The following may be covered:

    • What was the procedure adopted for selection: consent and participation of community?
    • Whether he/she is from the local community
    • What were the criteria for selection?
    • Were they trained?
    • Training duration.
    • Contents of training.
    • Who were Resource persons for training?
    • Where were they trained?
    • Size of group
    • Reading material provided
    • Was training adequate-Preraks' opinion?

    On the basis of the above responses, the evaluator has to comment on the quality of personnel, quality of training and adequacy of contents taking into consideration the needs of CE programmed. The evaluators may come out with suggestions for improvement in the CE Scheme itself in addition to specific suggestions to the district.

    Preparedness to Launch Actual CE Activities
    The evaluating agency should come out with their judgment on how the ZSS proposes to launch actual Continuing Education Activities. Output of the First External Evaluation
    The evaluating agency should submit a report of about 40-50 pages. It should give recommendations for betterment in the light of their finding in clear terms.

    Some observations on the functioning of CE and Nodal CE centers may be given in the report.

    Second External Evaluation:
    After Four Years

    The report of the first external evaluation carried out after completion of 2 years of the CE Programmed and the appraisal report prepared by SLM/SDAE after 3 years of the CEP will be available for reference and would be quite useful for planning this external evaluation report and the first appraisal report and discussions with the ZSS functionaries.

    The focus of the second external evaluation is on actual "Programmed, Implementation and Process". The entire CE activities, top be carried out after establishing the centres, can be divided into 2 groups, namely (1) Core functions of the Nodal and CE centres and (2) Target Specific Functional Programmes. The core functions are very important and the evaluators will have to carefully evaluate how these functions are being performed in a district. This can be done only by taking appropriate samples of Nodal and CE centres. The evaluation of activities under the second group will have to be done by considering various activities undertaken in the district and taking an appropriate sample of beneficiaries and the important activities. Of course, data and information necessary for evaluation has to be collected from the ZSS office and Nodal and CE centres.

    EVALUATION OF CORE FUNCTIONS OF NODAL AND CE CENTRES
    Sample Design and size
    Fifty Continuing Education centers and 5 connected Nodal centers should be covered in the sample. For this purpose random selection from each block or region could be done and this would be two stage sampling, village and then CE centers.

    The list of connected nodal centers to the above 50 CE centers should be prepared and five nodal centers among them should be selected in the sample by random method.

    All important core functions should be evaluated in the basis of the sample drawn of CE and Nodal CE centers. The important core functions and the indicators to be covered are listed below.

    Support Function of Nodal CE Centres
    The in-charge Prerak of the nodal CE center has to perform a number of support functions for the CE centers attached. It has to be evaluated as to how effectively he has been going this. A number of questions on this will have to be included in the questionnaire for the Prerak of the nodal CE center and for the Prerak of the CE center. This will also involve finding out how the ZSS has been instructing the preraks of the modal and CE centers and giving them support.
    Library
    1. Library has been set up or not.
    2. Number of books and the variety
    3. Circulation, whether books are issued, to what extent, who and how many are the borrowers?
    4. Display and maintenance of library and its records
    5. Timings of the library and how many days in a week kept open.
    Reading Room
    1. Timings of the reading room
    2. How many and on what days kept open in a week?
    3. What are the reading materials subscribed?
    4. How many people visit the reading room?
    5. Any volunteer help received for running the reading room?
    Information and Development Centre

    • Type of information disseminated.
    • Method of dissemination.
    • Use of various media.
    • How many people approach for information?
    • What are the initiatives taken by the prerak to disseminate information?
    • Types of material available at the center.
    • Resource persons.
    • Participation/support of the government departments, other agencies
    • Issues on which the Charcha Mandals are organized, how many, number of participants?
    • Have these activities of the center helped generate demand for target oriented programmes?
    Cultural Centre
    • Major cultural activities taken up.
    • Frequency of organization.
    • Number of participants - who performed and also audience.
    • Process of identification and nature of association of local cultural groups and
    • individuals.
    • Community participation - contribution.
    Sports Centre
    • Sports organized
    • Frequency
    • Participation
    • Local community participation-contribution.
    • Level of organization (Mohalla, village, Panchayat)

    Most of the above information will have to be collected from the preraks of nodal and CE centers. These aspects should also be covered in focus-group discussions. In addition, meeting people of the village/community informally will help the evaluator.

    Evaluating Target Specific Functional Programmes
    There are four types of Target specific Functional Programmes, namely, equivalency, income generating, quality of life improvement and individual interest promotion programmes. By discussing with the ZSS functionaries and from the records, a list of programmes/activities undertaken under each type during the last 2 years should be enlisted. Important programmes among them should then be selected based on the nature of the programmed and the number of participants. Each such selected activity should be evaluated. This will need studying the records at various levels and also interviewing the beneficiaries/participants of the programmed by taking appropriate samples. The assessment of programmed benefits to the learners/beneficiaries/participants should be a vitally important part of the evaluation and should be relevant to each. While evaluating training programmes the attitudinal and behavioral changes can be evaluated but cognitive and educational delivery systems must be focused on.

    Sample Design and Size As stated above, samples of beneficiaries/participant/learner in each of the important activities identified above have to be drawn. About 10-20 persons in each activity should be included in the sample. The sample plan will have to be drawn based on the locations of beneficiaries in each activity. Geographical spread of the conducted activity must be ensured. For certain activities the nodal and CE centers selected in the sample for the study of core functions might be useful and for certain activities new centers will have to be visited. The plan of sampling will be district specific. The list of points/indicators to be covered in evaluation for these four Target Specific Functional Programmes is presented below for reference.

    1. Equivalency Programmes

    • Type of programmes taken up.
    • Method of need assessment.
    • Collaborative efforts by ZSS with State Resource Centres (SRCs), National and state
    • Open Schools (NOS & SOS).
    • Technical support provided by ZSS/RRC/NOS/SOS.
    • Material produced/procured/provided to the learners.
    • Net working with NGOs.
    • Certification.
    • Benefits accrued to the learners.

    2. Income Generating Programmes

    • Selection of vocations, trade skills.
      • Whether need based.
      • Method of need assessment.
    • Number of programmes organized.
    • Number of beneficiaries, Agencies associated
    • Method of mobilization
    • Delivery system: feed back from beneficiaries.
    • Networking and coordination mechanism with various development agencies.
    • Community participation.
    • Benefits accrued to the learners.

    3. Quality of Life Improvement Programmes
    Under CE, such educational programmes are to be undertaken which help to improve the quality of life. Indicators of quality of life are family welfare, health, nutrition, drinking water, sanitation, population education, environment, social services, communal harmony etc. The programmes undertaken may be studied with respect to the following indicators:

    • Number of programmes organized
    • Number of beneficiaries
    • Agencies associated.
    • Delivery system of education programmes.
    • Media used and presentation methods.
    • Outreach.
    • Programmed utility.
    4. Individual Interest Promotion Programmes (IIPS)
    • Method of identification of the interest areas.
    • Resource information on identified interest areas.
    • Facilities/Guidelines provided and for how many, networking and coordination of
    • relevant agencies.
    • Feedback form beneficiaries and benefits accrued.

    Evaluating the Mopping up Operation In mopping up operation is a vitally important activity in achieving our goal of full literacy and has been identified as an essential component of the CE programmed. It is, therefore, necessary to evaluate the mopping up operation carefully. The sampling procedure and the size of the sample would be as per the PLP Evaluation Guidelines.

    • Door to door resurvey by ZSS
      (a) Methodology: what, how, when, by, whom
      (b) Lists of learners under mopping up
    • What was the mopping up target?
      (a) Unenrolled non-literates
      (b) Drop-outs during TLC/PL
      (c) Learners still to complete P-III & those who could not reach NLM norms
      (d) New entrants
    • Efforts taken to cover the above four categories
    • Arrangements for teaching-learning
    • Availability of Primers
    • Testing learners on sample basis
    • Estimation of district level achievement on the basis of above testing
    • How many reached NLM norms?
    • How many still to reach?

    Third External Evaluation:
    After Seven Years

    By the time the third external evaluation is taken up by the eternal agency, 2 external evaluations would have been carried out for the district and also two appraisals reposts would have been prepared by SLMA/SDAE. These documents would be important reference material for the external evaluation agency. The Monitoring Proforma which are filled in and submitted by the ZSS on monthly basis would also be important sources of information.

    Main Coverage
    This evaluation will have to cover all the aspects covered in the second External Evaluation (carried out after the completion of four years) namely, core functions of CE and nodal CE centers and target specific functional programmes was well as mopping up operation. For this, the methodology given in Chapter IV has to be adopted.

    Review of Administrative and organizational aspects
    Since by this time, three years have lapsed after the First External Evaluation covering Administrative and organizational aspects, it will be necessary to review these aspects quickly during this third external evaluation. The external agency should, therefore, take a quick review of the various aspects covered in the first external evaluation, including, the action taken by the ZSS on recommendations made. On the basis of this review the external agency should give a score between 0-5 to the ZSS. The working out of the composite score is explained in the following pages.

    Evaluating Sustainability
    As per the present funding pattern, the Central Government provides 100% financial assistance to a district for the first three years. Thereafter, for the next 2-year the costs are to be shared on a 50-50 basis by the central and state governments. Subsequently, the State/UT government is to bear full costs. In this connection, the following paragraph quoted by the NLM booklet on CE is quite important for the evaluator.

    "Clearly, the effectiveness of a CEC and the scope of its activities will be significantly determined by the extent of support enjoyed by it from the community. The CECs must be perceived by the people as arising from their own initiative to meet their explicit needs. To achieve this objective, the ZSS will be expected to devise all possible ways to enlist community support and mobilize financial and material resources from the community itself so that the CECs and their programmes becomes self-sufficient in due course of time."

    It is in this context that sustainability has to be evaluated. To undertake the sustainability aspect, the following will have to be covered:
    1. Support enjoyed from the community.
    2. Financial aspect of sustainability
    3. Infrastructure availability
    4. Personal planning and planning for training new-personnel
    5. Material and equipment planning
    6. Monitoring arrangements
    7. Institutionalization

    The sustainability aspect will have to evaluate by going through the records, official CE plan for the future and by discussing with the key persons of the ZSS and some taluka level functionaries. Some focus-group discussions would be useful in assessing community support.

    Index of Performance of CE
    It is proposed that the external evaluation agency, during the third evaluation will compute an Index of Performance of CE for the district concerned. For this, the following scheme of weightage for each of the 8 aspects to be covered is suggested. For each aspect, a score of 0 to 5 points will be allotted by the agency depending upon the performance of the district and then by using the weightage scheme the composite index has to be computed.

    Scheme of Weightage

    AspectWeightage
    1. Establishment of CE Centres 10
    2. Core functions of Nodal and CE centers 20
    3. Equivalency Programmes 10
    4. Income Generating Programmes 10
    5. Quality of Life Improvement Programmes 10
    6. Individual Interest Promotion Programmes 10
    7. Performance of Mopping Up Operation 20
    8. Sustainability beyond five years 10
    Total 100

    For the most important aspects namely Core Functions of the Nodal and CE centers (S.No.2) and Mopping-up operation (S.No-7) weightage of 20 each has been given and for all the remaining aspects weightage of 10 each has been given. The score for each aspect (on a scale of 0-5) has to be multiplied by its weightage and all these are to be added for all the eight aspects. This total is then to be divided by 5 to give the composite score in terms of percentage. For example consider a district with the following scores as given by the external agency: 'A' district for example

    AspectScoreweightageScore Weightage
    1. Establishment of CE Centres31030
    2. Core functions of Nodal and CE centers42080
    3. Equivalency Programmes21020
    4. Income Generating Programmes41040
    5. Quality of Life Improvement Programmes21020
    6. Individual Interest Promotion Programmes21020
    7. Performance of Mopping Up Operation 32060
    8. Sustainability beyond 5 years31030
    100 300
    Composite score 300/5=60

    The district is at 60% level of performance

    Appraisal Reports by the SLMA/SDAE

    As per the present guidelines for evaluating Continuing Education programmes the SLMA/SDAE has to carry out 2 comprehensive appraisals, the first after completion of three years and the second after the completion of the fifth year of the programme. The coverage by these appraisals would be as follows:

    1st Appraisal: Should cover what is to be covered in the second external evaluation.
    2nd Appraisal: Should cover what is to be covered in the third external evaluation.

    The appraisal exercise has to be carried out by the SLMA/SDAE through an agency, which has been empanelled by it. A copy of the Appraisal Report should be submitted to the NLM. The methodology to be adopted for 1st and 2nd appraisal will be similar to what is proposed for the 2nd and 3rd external evaluations respectively.