Evaluation of Education Quality in the Faculty of Mechanical Engineering and Robotics

 

KOWAL Janusz1, KOWALSKI Wlodzimierz2, SAPINSKI Bogdan3

1 University of Mining and Metallurgy, jkowal@uci.agh.edu.pl
2 University of Mining and Metallurgy, wkowalski@uci.agh.edu.pl
3 University of Mining and Metallurgy, deep@uci.agh.edu.pl

 

Abstract: The article presents forms of the participation of students in the education quality evaluation system in the Faculty of Mechanical Engineering and Robotics of the University of Mining and Metallurgy (UMM). The international cooperation of the Faculty as a part of Tempus project, the subject of which contains the education quality in Faculties of Mechanical Engineering, is also presented. The authors discuss research conducted at the UMM in the field of implementation of a uniform university system of lesson evaluation done by students.

Keywords: education, study program, quality

 

1 Traditional forms of lessons’ evaluation

Some forms of an education quality system have been applied to the Faculty of Mechanical Engineering and Robotics for a long time. Traditional inspections of lessons of university teachers by their supervisors, as well as mutual attendance of university teachers cooperating within the same organization unit (group, department) which are commonly practiced at university, can be considered as such forms. The participation of assistants in lessons conducted by their supervisors can also be considered as one of these traditional forms. Attending and - more generally - the mutual inspections of lessons served as an inspiration for an exchange of opinions concerning essential lesson contents, as well as the methodology applied. In this area, the initiative for such type of opinion exchange belonged to the group of university teachers who generally can be called supervisors.

Similarly, different forms of lesson evaluation by students have been applied for a long time. It can be stated that all university teachers have had the equal initiative in this domain; perhaps the initiative of younger university teachers was greater. Different kinds of questionnaires were applied. In practice, they were always prepared by the person had initiated this kind of estimation as a part of his/her lessons. The information about the estimation results was reserved by the organizers for their own purposes.

2 Present forms of students’ lesson evaluation applied to the Faculty

For several years attempts have been made in the Faculty of Mechanical Engineering and Robotics as well as in other faculties to standardize the used forms of education quality evaluation. The essential inspiration was the publication in 1996 by the General Council of the Higher Education of the announcement of classification of faculties in with respect to their didactic level, similarly to the classification regarding the scientific level, applied by the Committee of Scientific Research. In that paper the General Council presented the proposals of two kinds of commonly known questionnaires of lesson evaluation to be done by students. In the Faculty of Mechanical Engineering and Robotics they were recognized as a standard worth of propagation.

Before the beginning of the last year summer semester the Faculty Council took the resolution concerning the student estimation of lessons in normal system studies. Here is the text of the resolution:

Bearing in mind further improvement of the didactic process in the Faculty, in compliance with recommendations contained in the paper of the General Council of Higher Education ”System of Education Quality Estimation in Universities”, the Faculty Council takes the following resolution:

  1. The Faculty Council recommends scientific-didactic and didactic workers of the Faculty of Mechanical Engineering and Robotics to subject the chosen lessons conducted in the Faculty to the estimation of students, from the beginning of the current semester.
  2. The Faculty Council recommends the questionnaire models proposed in the paper of the General Council of the Higher Education to be used for the lesson estimation.
  3. The Council leaves the choice of lessons subjected to the student estimation as well as the choice of questionnaire model to the teachers.
  4. In the summer semester of the present academic year the estimation contents remain known to the workers executing the estimation.
  5. After the end of the semester, Departments will present in the Dean’s office information concerning estimations executed, including the list of lessons subjected to the estimation and the number of students participating in it. The summary information will be presented to the Deputy Rector for Education.
  6. The Dean can execute the estimation of chosen lessons at his own initiative.

The above resolution was taken after a discussion mainly concerning point 3 and 4. As a result of the discussion it was stated – considering the former application of different individual forms of inquiry - that a sufficient step forward will be the standardizing of questionnaire contents and the official information for the didactic workers of the Faculty that the execution of questionnaires of such type and the propagation of application of this form of lesson estimation are expected, especially in context of the General Council suggestion concerning a very probable institutional introduction of the obligation of a lesson inquiry application in the future. It was admitted that the leaving the questionnaire results exclusively to the knowledge of the inquiring persons would favor honest evaluation and that it would be an opportunity (especially for the didactic workers who had not used the inquiry of lessons before) to see ”educational efforts” of teachers as they look in ”students’ eyes”. At the same time the information was spread that the remaining of the questionnaire results exclusively to the knowledge of workers would be temporary and this rule could be changed soon, but not before the beginning of the next semester. Many of didactic workers have used this opportunity for the estimation of their own lessons. It is estimated that about 30% of the Faculty students have participated in the lesson inquiry.

The aim of point 6 of the Faculty Council resolution was to facilitate the solution of conflict situations. They are rare in the Faculty. They are expressed in group requests for ”the change of the teaching person” presented by students to the Dean. Students explain their position by an inadequate especially too stressing form of teaching. It has to be admitted that students do not abuse their right to criticize the teaching form and their pronouncements were not without a cause. The application of the lesson estimation by the Dean, in conformity with the point 6 of the Faculty Council resolution, significantly simplified the accommodation of conflicts and - except for one case - the change of the teaching person was not necessary.

In the current semester the lesson inquiry based on the rules determined in the Faculty Council resolution is continued.

Since the previous academic year the opinions of graduates have also been collected in the Faculty. It is done on the occasion of handing of the University degree certificates. The questionnaire results are very interesting. The graduates are generally satisfied with the choice of the University, faculty and specialization. Choosing the faculty as the place of their studies they were guided mainly by the rank of the University, their interests and good circumstances in the given field. They state (about 70% of them) that during their studies they were inspired to independent thinking by teachers. The way of teaching is estimated by them mostly as a good one. They often consider (about 40%) that the number of hours devoted to one subject was not sufficient. In general (65%) they are satisfied with forms of lessons and giving the credits. They think that during studies their opinions and demands were fairly well taken into consideration (60%). Almost 90% of graduates have found satisfactory work within several months after the end of studies. They consider that knowledge of foreign languages and fluent use of computer are the additional skills making it easier to find a job.

3 International cooperation of the Faculty in the field of the education quality estimation

In the academic year 1995/96 the international project Tempus CME 01068-95 entitled Internal Quality Assurance System within Mechanical Engineering Faculties was executed in the Faculty. The UMM, the Faculty of Mechanical Engineering and Robotics (Poland), GKN International College of Engineering (Germany) and Glasgow Caledonian University (Great Britain) were partners of the project.

The aim of the project was to support the reorganization and so-called institutional development of the Faculty in the field of the internal system of quality assurance, and especially

The results of the project execution are very satisfying. The Faculty workers participated in many training tours and they had an opportunity to observe the didactic lesson organization and the applied forms of their estimation. Several meetings and seminars were organized. At the end of the project the international conference was organized at the UMM. It was entitled ”Quality Assurance within Engineering Higher Education” (EQAS’96) [1]. The conference EQAS’98 in Zakopane (September 13th-16th, 1998) was its continuation. It had preceded ”The Meeting of Deans of Mechanical Engineering Faculties” in September 1998. Within the project, studies on a group of 27 students were executed in the Faculty [2].

The results of studies made according to the project Tempus significantly influenced the resolution of Faculty Council concerning the lesson estimation by students, discussed above.

4 Work concerning a standardized system of education quality estimation executed at the University

The question of the education quality estimation at the UMM was taken into consideration by the Senate Committee for Education. For this purpose it created a team consisting, among others, of workers of Social Sciences Institute of the UMM and faculty deputy deans for education. In June’97 the team elaborated the Report [3] in which principles of the education quality estimation were proposed as the subject for further discussion. The following observations were basic for the elaboration of report conclusions:

  1. Some faculties examine the student opinion by means of questionnaires of different kind, as well as by inspections of lessons. Faculties which do not execute inquiries among students, mostly declare their willingness to apply such inquiries, but not before the elaboration of a uniform inquiry system for the whole University. Faculties, which up till now have studied student opinions, used different questionnaires. Also the aims of those studies were various. Some faculties have used questionnaires for the periodic estimation of workers, but most often these questionnaires serve as a tool for a casual solution of didactic problems.
  2. It is typical that the questionnaires applied at present contain questions formulated in such a way that both sides - the estimated as well as the estimating - are put in an embarrassing and conflict position. An unfortunate question construction, for instance: ”Is the teacher properly prepared for lessons?” may result in the fact that questionnaire is applied relactuntly and those different new questionnaire variants are created. This fact does not promote the standardization of estimation systems of the education quality level.
  3. All this creates a need for the elaboration of a new questionnaire which would be professional and uniform at the whole University and which could in a conflictless way fill the existing gap in the internal estimation of the education level in faculties and which at the same time would make mutual comparisons possible.
  4. As the result of the observation of the education quality estimation condition in faculties it can be stated that there is a big need for a questionnaire that could give solid and credible information concerning the didactic process. The condition of the success of such an undertaking is to base it on the previous solutions and on the experience of people professionally resolving problems of this kind.
  5. An attempt was made to construct such a questionnaire on the base of the experience of other universities and foreign questionnaires, as well as on the analysis of the questionnaires applied up till now. The project guiding idea is to move the emphasis from the university teacher estimation to the study of student feelings caused by the participation in lessons. It is assumed that it will make it possible to obtain dependable information concerning the quality of teacher didactic work, without causing unnecessary stresses and protests.

In the report conclusion, the following proposition was presented:

It is suggested that a separated unit should execute the inquiry among students. It should consist of experts. This fact will assure a professional estimation level and the objectivity of results.

  1. The Dean charges the University Center of Student Opinion Inquiry (UOBOS) with the execution of the study of student opinions concerning lesson, course or specialization.
  2. The contents of the questionnaire and principles of the scoring for each answer should be known at the beginning of an academic year to all university teachers who will subject their lessons to an inquiry.
  3. Students fulfill questionnaires during the last lesson of a given course.
  4. Each questionnaire is coded and only the Dean knows the names of teachers concealed by code numbers.
  5. The team executes a comprehensive analysis of the inquiry data, prepares the report and presents it to the Dean.

5 Summary

At the UMM up till now the order concerning the application of an uniform questionnaire has not been given. The Faculty of Mechanical Engineering and Robotics continues the estimation by students of the lesson quality according to principles formulated in the Faculty Council resolution and on the basis of experience attained during project Tempus execution. The Faculty is ready to implement an uniform system of the lesson estimation.

References

  1. J. KOWAL, J. SZPYTKO. Staff development in Engineering Education – a Pilot Study and Experience in the UMM Cracow. In Proceedings of the International Conference on the Quality Assurance within Engineering Higher Education. Cracow, Poland, September 22 – 25 1997. 90 - 94 s.
  2. S. K. GHOSH, J. SZPYTKO. Educational Versus Professional Engineering Development Programs for Current Business Environment. In Conference Proceedings – Second International Conference Quality Assurance within Engineering Higher Education. Zakopane, Poland, September 13-16 1998. 53-57 s.
  3. Humanities and Arts in a Balanced Engineering Education. In Conference Proceedings SFFI Annual Conference 1997. Cracow, Poland, September 8-10 1997