Monitoring and evaluation of any scheme, specially in the social sector, has great significance as it helps one to know whether the scheme is on the right track. The National Literacy Mission has of late being laying stress on vigorous monitoring and systematic evaluation of adult education campaigns launched under its aegis in the country. It has developed and circulated guidelines for concurrent and final evaluation of total literacy campaigns and post literacy programmes. And, is in the process of bringing out a comprehensive set of guidelines on continuing education. There is greater emphasis on the impact that the programme has had in its catchment area. What effect, for example, has it had on improving the value attached to education, or to empowering women and men, or to building awareness of social issues such as dowry, small families, harmonious caste relations, or to highlighting the need for conserving the environment?
Earlier, the Directorate of Adult Education made strenuous efforts to collect the information and data directly from the districts. The extension of the literacy campaigns to the entire nation has made it imperative to design a new reliable system that ensures the steady flow of information needed by the National Literacy Mission.
The previous monitoring system laid excessive emphasis on targets. This often tended to give only quantitative information and there was practically no way of knowing whether the information had been cross-checked and verified. The management information system was made more reliable by decentralising it. The state directorates of adult/mass education now hold the responsibility of monitoring literacy campaigns in their respective states.
The Director of Adult Education in each state/UT is required to hold a monthly meeting on a fixed date. The objectives of these meetings are to review the progress of literacy campaigns, post literacy programmes and continuing education programmes in various districts. These meetings are attended by secretaries of the Zilla Saksharta Samitis, Directors of State Resource Centres, state/area coordinators of the Bharat Gyan-Vigyan Samiti or representatives of the nodal voluntary agency and of Jan Shikshan Sansthans. A representative from the National Literacy Mission or the Directorate of Adult Education is also present. Apart from obtaining correct, factual and up-to-date statistical information, the focus of discussion is also on assessment of qualitative aspects such as problems encountered in accelerating the pace of literacy campaign, efficacy of corrective measures initiated in the past, activities planned for the coming month, and so on.
The State Directorates scrutinise, analyse and compile the data from all the districts carefully and send consolidated state reports to the Central Directorate of Adult Education in the prescribed format. National-level status reports on literacy and post literacy are also compiled and published periodically.
The visit reports of the members of the executive committee of the National Literacy Mission, journalists, literacy consultants, departmental officers also form an important part of the monitoring system.
Another important step taken towards revitalising the state-level monitoring system was the computerisation of state directorates. Under the new decentralised monitoring system, the state directorates have been assigned a vital role in monitoring district campaigns. The task essentially required proper computer facilities. Realising this, the National Literacy Mission made a special provision and provided additional budget to the state directorates to purchase computer systems. Now, most of the states have installed computers and the system is being efficiently used. All the state reports are centrally processed by the computer and analysed to ascertain the strengths and weaknesses of the programme to facilitate further implementation and policy formulation.
The post literacy programme is a transition from a structured literacy programme to an unstructured self-learning phase. The projects are highly contextual to the district, and can consist of several target groups, with some continuing non-formal education, some with adult night schools and some with simply self-learning. Reporting on post literacy campaign is thus on the lines of the project activities. A basic statistical report should therefore be accompanied by a qualitative account of coverage.
Monthly monitoring meetings are held at state and district levels. Given the diverse nature of the various elements of a post literacy campaign, closer monitoring is done through project functionaries, people's committees and Panchayats at the village and block levels.
Monitoring of continuing education programmes in a scientific way is very important for quality and functional control. The overall responsibility of monitoring the programme lies with the ZSS. Appropriate bodies are created at the sub-district level in the form of Block Monitoring Committees and, if feasible, Panchayat Monitoring Committees.
The prerak of the CEC must submit his/her monthly progress report to the prerak of the Nodal CEC. The Nodal CECs submit their collated monthly reports cluster to the Block Coordinator, who in turn must send them to the district-level committee i.e. the ZSS. The ZSS report will be reviewed by the State Literacy Mission and National Literacy Mission every month on the basis of a pre-determined MIS format.
The in-charge of the Nodal CEC is expected to visit the CECs in his/her cluster and hold meetings with all the volunteers at least once a month. The Block Coordinator must visit each Nodal CEC and hold meetings with the prerak at least once a month. The ZSS functionaries through their core team at the district level must hold periodic meetings with the Block Coordinators and also conduct regular field visits.
To ensure the participation of the community, the establishment of Village Education Committees (VECs), Committees of Neo-literates, User Committees, etc., are encouraged for day-to-day functioning of the programme.
The National Literacy Mission now recommends a three-step evaluation:
Self-evaluation of learning outcomes is a built-in feature of the three literacy primers. Each primer contains a set of three tests. It is only when the learners qualify in these tests, they reach the levels of literacy and numeracy laid down in National Literacy Mission norms. This self-evaluation would enable a learner to perceive his own pace and progress of learning and should heighten his motivation.
Concurrent or process evaluation is a kind of mid-term appraisal of an ongoing project to assess its strengths and weaknesses and to suggest corrective measures accordingly. It is conducted in two stages during the course of the campaign. The first stage is undertaken when at least 50 per cent of the enrolled learners have completed the first primer. The second stage is completed within three months after the first stage. The concurrent evaluation is carried by competent evaluation agencies from outside the concerned districts. The state directorate nominates three evaluation agencies of which the district chooses one to conduct the concurrent evaluation.
Final or summative evaluation of a literacy campaign is undertaken when at least 60 per cent of the enrolled learners have completed or are nearing completion of the third primer. This evaluation is carried out by reputed evaluating agencies from outside the state and empanelled with the National Literacy Mission. On receiving a request from the concerned Zilla Saksharta Samiti through the state directorate, the Mission suggests a panel of three to five agencies from which the district chooses one to carry out the final evaluation. The main focus of the evaluation is the learners and the evaluation is done in the random sample method. A minimum sample of 5 per cent or 10,000 (whichever is less) is to be selected randomly. Care is taken to avoid sample loss, absenteeism and proxy attendance. The criteria for considering a learner literate is achievement of at least 50 per cent marks in each of the abilities (reading, writing and numeracy) and 70 per cent as the aggregate. So far, final evaluation has been undertaken in 281 districts.
Similar final evaluation exercise is carried out for the districts going through the post literacy phase. For projects of one year duration, the final evaluation is carried out at the end of the programme and is more or less an impact evaluation. The revised guidelines for this have been evolved through a series of regional and national workshops.
The selection of agencies for conducting the evaluation of post literacy programmes is the same as that for total literacy campaigns. The agencies are expected to keep in mind the objectives of the post literacy programme while carrying out its evaluation and assessing its impact. These agencies are oriented from time to time with the latest developments in the field of literacy and objectives of the National Literacy Mission. The state directorates also conduct such orientation workshops for the evaluating agencies. Post literacy evaluation concentrates not merely on evaluating learning skills but also other themes like convergence with development programmes, skill upgradation, etc.
The evaluation guidelines have been standardised and are issued by the National Literacy Mission from time to time. Special impact studies of the campaigns on the social, cultural and economic environment of the areas are also being undertaken by the State Resource Centres and such other organisations under the aegis of the Mission. It is hoped that the new approach of evaluating of literacy campaigns and post literacy programmes will ensure complete transparency and enhance the credibility of the results and impact assessments.