SEROW, Robert C. & ZOROWSKI, Carl F.
Raleigh, NC, USA - North Carolina State University, serow@poe.coe.ncsu.edu
Abstract: In 1992 the National Science Foundation funded through its Engineering Education Coalition Program (EEC) an eight southeastern engineering school consortium (SUCCEED) to create a new undergraduate engineering education model for the twenty first century. SUCCEED was one of eight such coalitions funded to foster systemic reform to revitalize engineering education. In addition to developing new curriculum models, components and processes the coalitions were charged to disseminate their results for adoption at other engineering schools and programs. In its first five years SUCCEED concentrated on defining its curriculum model and developing some seventy curriculum components and processes. The coalition is now involved in the implementation and institutionalization of the model and its components among its members. It has also begun the task of disseminating the results of its developments nationally.
To assist in this later process research was conducted on the factors that promote the successful diffusion of instructional innovations. The diffusion of a good product or idea does not always occur as readily as expected. This subject was reported on by Everett M. Rogers as related to technological innovations. He developed a model of factors that influence how an innovation diffuses. The study reported here was to validate this model and to see if other criteria applied to educational innovations by conducting case studies of SUCCEED products that have diffused successfully.
Each of the products studied benefited from three principal factors related to prior research and theory on diffusion. These were: first, a high level of product quality, achieved in part through active solicitation of user input; second, active partnering with external organizations that can assist in the diffusion process; and third, relatively low product cost. Understanding how these factors impact the studied products will prove useful to education innovation developers in diffusing good products and ideas. It will also be of assistance to more rapidly diffuse new developments in engineering education.
Keywords: diffusion, innovation, education, engineering, dissemination
In the early 1990s, the U. S. National Science Foundation (NSF) established the Engineering Education Coalitions (EEC) program as a means of creating a new undergraduate education model for the twenty-first century. One of the coalitions within EEC is SUCCEED, a consortium of eight university-based colleges of engineering located in the southeastern United States. The overall goal that SUCCEED shares with the other coalitions is to revitalize engineering education for future industrial and national needs in the United States. In addition to developing new curriculum models, components and processes, the coalitions were charged with the task of disseminating their results nationally for adoption and implementation at other engineering schools and colleges.
This paper will examine the dissemination process, as it has been implemented in three SUCCEED projects. In each case, the project originated as a small-scale effort intended to meet the immediate needs of a local audience. Over time, and with the support of SUCCEED and NSF, these projects have expanded their vision and have developed instructional materials that are now used across the United States and beyond. The purpose of this paper is to extract from these cases some general guidelines about the diffusion of instructional innovations. Our expectation is that understanding how these projects achieved such success will prove useful to other engineering educators in their efforts to disseminate good products and ideas. Such information may help to overcome the ”not invented here” syndrome that in the past has too often served as a deterrent to the adoption of new ideas and products in engineering education.
The decision to emphasize dissemination of the coalitions’ output has followed several decades of research showing that the diffusion of good products and ideas does not always occur as rapidly as expected. According to Everett M. Rogers, a noted communications theorist, the success of an innovation depends not only on its technical qualities, but also on the process by which its basic ideas are communicated. In his words, ”Many technologists believe that advantageous innovations will sell themselves, that the obvious benefits of a new idea will be widely realized by potential adopters, and that innovation will therefore diffuse rapidly. Seldom is this the case. Most innovations, in fact, diffuse at a disappointingly slow rate.” (Rogers, 1994, p. 7).
Rogers and others have identified the optimal conditions for dissemination of innovations, the most important of which are as follows:
Relative advantage--the degree to which the innovation is perceived as an improvement over existing practice.
Compatibility--preference for innovations that address core issues and stay within a recognized sphere of applicability.
Complexity--often found to be inversely related to the diffusion of an innovation, while simplicity, or ease of use, makes for wider and more rapid acceptance.
Trialability and Observability--ease with which adjustments can be made to an innovation, and visibility of results (Rogers, 1994; see also Tornatzky and Klein, 1982).
These ideas provided some of the principal benchmarks against which the diffusion of SUCCEED projects were studied. To be precise, SUCCEED’s Assessment and Evaluation (A& E) team, of which both authors are members, viewed the case studies in part as a means of testing the applicability of Rogers’ Diffusion of Innovation model to instructional innovation in engineering education. In the remainder of this paper, we will describe the methods and results of three of the case studies.
It is important to mention that during the first five years of its projected ten-year life cycle, SUCCEED did not require its funded projects to follow any particular path to the external dissemination of products. Instead the focal point of the coalition’s activity was the definition of its curriculum model and the development and testing of some seventy curriculum components and processes. In addition, networks within the coalition were established with an eye toward sharing ideas across SUCCEED’s eight member institutions. Among the vehicles used for this purpose were an annual conference, a newsletter, attendance of project staff and coalition managers at site visits of NSF teams, and the encouragement of inter-institutional collaboration on projects. By contrast, the external diffusion of output during the coalition’s early years was largely left to the efforts of the individual projects. The upshot of this situation is that when the coalition’s A&E team began to study product dissemination, we were not given a checklist of standards and regulations for project performance by either NSF or SUCCEED. Instead, we were able to turn to the research literature for help in identifying the appropriate criteria.
Selecting the subjects of the case studies required careful deliberation. While SUCCEED has sponsored many useful projects since its inception in 1992, we were particularly interested in those whose major products had been extensively disseminated beyond the Coalition. After reviewing project reports and speaking with numerous SUCCEED participants, we narrowed our focus to a small number of products, three of which are the subjects of this paper. They are Visualizations in Material Science (ViMS), a multimedia CD-ROM developed by John Russ and his colleagues in the Department of Materials Science at North Carolina State University; Mars Navigator, also a multimedia CD-ROM, which had been produced by a faculty-student team directed by Kurt Grammol of Georgia Tech (now of the University of Oklahoma); and the Entrepreneurs Seminar Series (ESS), a package of six videotaped lectures developed at NC State under the leadership of Tom Miller.
Three other projects were also studied. Although each of these had enjoyed some limited success in product diffusion, none approached the levels established by ViMS, Mars Navigator and ESS. Nor did we find that they matched up particularly closely with the guidelines for successful dissemination described below. It is possible that the three excluded cases will be the subject of a subsequent paper. However, considerations of confidentiality and collegiality require that we not make further reference to these projects in the current report.
All of the data pertaining to these projects were collected during the 1997-98 and 1998-99 academic years. In gathering this information, we followed the standard methods of the case study--a format now recognized internationally as a useful vehicle for assessing the impact of research and development projects in industry, government, and education (Kingsley, 1993; Yin, 1994). The major data gathering technique was the individual interview. In all, 38 individuals were interviewed for these studies, either in person or via telephone or e-mail. The interviewees included the project directors, campus and coalition administrators, local and national colleagues who had played key roles in project development, representatives of publishing houses and federal agencies, and university instructors from across the country who had adopted the products for use in their own classes. We also reviewed dozens of project documents, including funding proposals, annual reports, journal articles, conference papers, course syllabi, and, of course, the products themselves. Finally, we stayed current with each project by visiting its site on the World Wide Web and by following up any references to it on sites maintained by engineering educators, the media and software user groups.
One of the first general observations to emerge from the case studies was that each of the three products had been disseminated in collaboration with an outside partner: In the case of ViMS, the partner was a commercial publisher, PWS Kent of Boston; for Mars Navigator, a U. S. government agency, the Jet Propulsion Laboratory (JPL); and for ESS, a professional association, the Institute of Electrical and Electronics Engineering, Inc. (IEEE). All three organizations cooperated fully in our studies and proved to be excellent sources of information about product diffusion. From PWS Kent, for example, we obtained a list of the 64 colleges and universities that had adopted the ViMS CD for an engineering course (or, occasionally, for a course in another subject, such as Physics). The information snowballed from there, as we were able to speak with course instructors, who not only had their own comments about ViMS but also passed along remarks from their students. Similar procedures were followed in the cases of Mars Navigator and ESS.
As a result of this process, we were able to confirm that all three products were being widely used. The 64 institutions that have adopted the ViMS CD translate into a total student exposure that is well into the thousands. And that does not take into account that ViMS is also available on the Web (http://vims.ncsu.edu). Likewise, IEEE reports that 150 ESS sets have been sold so far, mostly to organizations that show the tapes at meetings and classes. Meanwhile, JPL has filled some 20,000 requests from teachers, students, and hobbyists for copies of the Mars Navigator.
Beyond calculating the extent of product diffusion, a further result of the studies was the creation of a data base against which to apply the Rogers criteria outlined above. In brief, the data suggested that all three products were rated highly by users on the basis of both relative advantage and compatibility with existing practice. Mars Navigator, for instance, was not only praised by the individuals with whom we spoke but has also received a major award for instructional media given by the John Wiley & Sons publishing company and by NEEDS, the National Engineering Education Delivery System (Additional information about the award can be found at the NEEDS home page: http://www1.needs.org). In much the same vein, ViMS was seen as a valuable supplement to traditional lectures and textbooks. As one instructor stated, ”It keeps students’ attention. It’s like watching a movie.” Others suggested that engineering majors are primarily visual learners and that the animation, video and graphics are especially effective for this purpose. As to whether ViMS helps students master the subject content, controlled experiments conducted at the home university and posted on the Web (http://vims.ncsu.edu/Home/mm_text.pdf) have shown an advantage of about one full letter grade for students in sections using ViMS.
It is also worth noting that problems reported by users of the products have usually been quickly remedied--a point that squares nicely with the Rogers criteria of simplicity and trialability. The Materials Science Department at North Carolina State University has continuously solicited student feedback in order to upgrade the content and delivery of ViMS. In fact, students who have recently completed the introductory Materials Science course are often hired to work on the development of forthcoming editions of ViMS. The logic behind this policy is worth considering. As described by the principal investigator, ”The way we do it is to hire undergraduates who have taken the course. We prefer students who got B’s with a lot of sweat. They’re paid $6 an hour and given some really cool toys to play with. Most have worked out very well.” In other words, it is the average student who is seen as the best source of information about the product’s strengths and weaknesses. The use of the ”typical consumer” as a lodestar for product development is, of course, a classic tactic in marketing research, and one that apparently has some value in developing instructional materials as well. Another example of product adaptability in response to user feedback concerns ESS, which. decided to add full transcripts of each of the six videotaped lectures when users reported difficulty in understanding the regional accents and local examples used by some of the lecturers.
Another factor that may help to explain the popularity of these products is that each has a relatively low purchase price. Although not addressed in the recent literature on Diffusion of Innovation, product cost was included in our case studies in recognition of the desire of many instructors and institutions to hold down instructional expenses. This was particularly important for us because at least two of the products (ViMS and Mars Navigator) are most often adopted as supplementary materials rather than as the primary text for the courses or programs in which they are used. (In other words, users will purchase them after having already paid $80 to $100 or more for the main textbook.) The retail prices of both the ViMS CD-ROM ($33.95) and the ESS tapes ($129 for IEEE members) compare very favorably with those of similar instructional products, while Mars Navigator is available at no cost from JPL (or at a nominal charge from a private company).
In summary, the lessons learned from these case studies come down to satisfying three criteria for successful diffusion of educational innovations: partnering, product quality, and low cost. The formation of external represents a important step toward effective marketing of the product. Because relatively few educators have either the capital or the contacts required to single-handedly mount an extensive marketing campaign, the ideal approach entails collaborating with an outside organization that can supply both. For example, conversations with representatives of publishing houses suggest that these organizations are eager to discover viable new instructional materials, especially those that can be directly targeted to courses with high enrollments, such as introductory and capstone courses in the various engineering majors. Publishing offers more flexibility today than in the past, when the print textbook was pretty much the only outlet available. The textbook is still the cornerstone of educational publishing, of course, but it is likely to be supplemented by course packs, custom publishing, video and electronic software. Moreover, marketing of instructional products is not the exclusive preserve of commercial publishers; as we have seen in two cases, governments agencies and professional associations are also eager to put good materials out on the market, providing that the product is directly linked to the organization’s mission of public or member service.
The second major lesson has to do with product quality. Not even the most vigorous publicity campaign can offset the effects of poor product design. Rather, consistent with prior research on diffusion of innovation, the innovations studied here were considered by our interviewees to offer a decisive advantage over other options, yet also preserved the best features from existing instructional practice. Each was also relatively simple to use and could be adapted to the interests and needs of varying audiences. The key to product quality was the willingness of the project staff to solicit user feedback and then to incorporate those comments in a cycle of continual product improvement.
Finally, ViMS, Mars Navigator, and ESS passed one additional test that does not appear in the literature on instructional innovation but which matters a great deal to engineering educators: Each of these products were relatively inexpensive to obtain and to use--an important consideration at a time when students and faculty are growing more resistant to the escalating costs of instructional materials. It may be that as more and more products become routinely available over the Web, cost will no longer be so prominent a consideration. However, as long as students are required to pay for instructional materials, and as long as faculty members are required to listen to their students complain about such costs, an innovation that is priced competitively will have an advantage.
As was mentioned at the start of this paper, we hope that the results discussed today will be useful in promoting new products and ideas for engineering education. We are taking our own advice in this matter, as the SUCCEED A&E team has recently embarked on an effort to get other projects to follow the principles of partnership, product quality and low cost. Right now, it’s too early to tell how this will work out, but perhaps we can report back to you at next year’s conference.
Kingsley, G. The Use of Case Studies in R&D Impact Evaluations. In B. Bozeman and J. Melkers (Eds.), Evaluating R&D Impacts: Methods and Practice. Boston: Kluwer, 1993, (pp. 17-42).
Rogers, E. Diffusion of Innovations, 4th ed. New York, NY: Free Press, 1994.
Tornatzky, L. G., and Klein, K. J. Innovation Characteristics and Adoption-Implementation: A Meta-Analysis of Findings. IEEE Transactions on Engineering Management, EM29, 1982.
Yin, R. Case Study Research: Design and Methods, 2d ed. Thousand Oaks, CA: Sage, 1994.