Cuyamaca College
Environmental Technology Program
Evaluating The Technology Transfer Project
It is important to evaluate technology transfer programs and projects on an ongoing basis to determine their success and to incorporate lessons learned in future activities. Measuring the results of technology transfer activities is an important component of technology transfer projects.
The three basic reasons to measure technology transfer results are to (1) provide accountability, (2) facilitate the process of technology transfer, and (3) demonstrate the value of a particular technology transfer project.
Provide Accountability
Any publicly funded activity has to be accountable to higher authority, usually to a program office, an agency, the chief executive (the President) and, ultimately, the U.S. Congress and the taxpayer. Each of these levels expects answers to the following questions: Did the project achieve the anticipated results? To what extent and why or why not? What unanticipated results were achieved? These and similar questions will be asked by management during the life of the project and at its conclusion.
Facilitate the Technology Transfer Process
We learn from both successes and failures encountered during the course of a project. For example, a particular workshop or demonstration project may not have included all of the stakeholders, and thus encountered some opposition or delay. Although the project may have been successful, the exclusion is an obstacle that needs to be assessed, with the appropriate lessons noted.
In another instance, a site demonstration may have been especially successful because the project managers obtained access to the site and ensured that the organizations responsible for the site were brought into the process at an early date. In another example, a potential partner may have been discouraged from participating in an exploratory demonstration or pilot because of unexpected terms and conditions that were not fully presented to the potential partner in a timely fashion. In each of these examples, there is significant opportunity to learn and thereby to improve the technology transfer process.
Demonstrate Value
Officials at all levels always want to know what the results are and whether the project achieved its goals.
For instance, recent studies of just over 400 examples of the application of technology from the National Aeronautics and Space Administration (NASA) demonstrated that the technology users benefited from sales or savings of $33.5 billion because of successful technology transfer. In a similar study for the Agricultural Research Service (ARS), sales or savings totaling $14.3 billion were identified in 87 cases where users applied ARS technology or science.
In addition, congressional hearings and studies by the General Accounting Office (GAO) during the past several years indicate increasing interest in some type of measure for technology transfer activities.
Technology transfer is rarely accomplished in a short period of time. The process may not be complete for five or ten years. Therefore, it is not always possible to wait for the end results; it is sometimes necessary to seek interim indicators that will demonstrate progress toward the particular project goal.
It is important to identify intermediate products whenever possible. For example, a project that uses remediation technology might include plans to use a variety of workshops and demonstrations as interim steps. Appropriate indicators could include:
(1) target organizations most likely to make use of the technology
(2) the number of organizations represented at the workshop and the distribution of attendees compared to the target audience originally identified
(3) the degree of interest and satisfaction gleaned from a brief evaluation by participants at the conclusion of the workshop
(4) a willingness to explore further participation with the project staff as determined by follow-on inquiries
(5) a follow-up of attendees at a three- to six-month interval after the workshop to determine short-term results.
As a project approaches completion, or in instances where the completion of a transfer is likely to extend past the end date of the project, other results might include indicators such as (1) companies started or expanded, (2) jobs created or saved, or (3) sales of services or products based upon the technology involved in the project. In addition, sales or services of technology overseas or the development of subsidiaries could indicate internationally competitive capability.
How Can You Measure Technology Transfer?
Some experts say a transfer is successful only when it becomes a profitable product or process, while others claim a transfer is successful when the technology is at least reviewed for possible use by another person or organization.
A great deal of technology is never commercialized due to (1) lack of financing; (2) management failures; or (3) unanticipated obstacles in the timing of the introduction, such as a shift in the economy, changed market conditions, or the sudden appearance of a competing technology.
Organizations with limited resources for technology transfer may have to concentrate on less costly mechanisms, such as publications. In fact, publishing often leads to the transfer of knowledge. For example, a 1979 study by the University of Denver's Denver Research Institute, reviewing various means used by the NASA Technology Utilization Program, revealed that the most cost-effective means of technology transfer was via its publication, Tech Briefs.
Measures of success in technology transfer need to be designed to cover the technology transfer mission and the resources available to the organization. Even if commercialization is a primary goal, the technology's value to further research or improvement of a process should not be overlooked or minimized.
Three Dimensions of Technology Transfer Measurement
The three dimensions with which we are concerned in the measurement of technology transfer are (1) the transfer mechanisms, (2) the time frame in which the transfer occurs, and (3) the area of impact (or results). Each of these dimensions will have an important role in determining the kind of measure that will provide the best information and be most indicative of either progress or actual results from the particular transfer.
(1) Transfer Mechanisms
Usually formal mechanisms, such as contracts, CRADAs, facilities-use agreements, licenses, and personnel exchanges, provide information that can be measured more easily. This is not true of the less formal activities, such as technical assistance; informal collaborations; and most workshop, publication, or data base activities.
(2) Time Frame
Technology transfer is often a rather lengthy process, so measures should be appropriate for the time frame in which the information is to be collected. The type of the measure will also depend on the stage of development of the technology. Technology that is ready for widespread application can have valid measurements in the short or midterm.
On the other hand, a number of transfer activities will be long-term; therefore, you will have to use surrogate or process measures for interim reporting. The three categories of short, mid, and long-term are useful in assessing the types of measurements possible in the time frame dimension.
(3) Impacts
A third dimension in measuring technology transfer is the consideration of the economic or technical impacts.
Attention is usually focused on economic impacts because of interest in forecasting sales, cost savings, and jobs created or saved. However, this information is often difficult to collect, especially if proprietary information is involved.
Technical impacts are more easily determined because they are usually included as a part of any project's goals, and the degree of improvement or substitution that has been sought can be described and given a value--for example, a description of sites where the technology has been selected for implementation or actually implemented.
Technology Transfer Program Criteria
A meaningful technology transfer plan should be based on a combination of criteria:
- Assists focus areas and cross cutting technology areas
- Assists with implementing waste management, environmental restoration, and decontamination and decommissioning activities
- Increases the commercial value of industrial technology
- Increases the involvement of private industry
- Decreases the cost of waste management cleanup
- Encourages the commercialization of technologies
- Increases U.S. competitiveness or creates jobs
- Responds to political mandates
Steps in Developing a Technology Transfer Evaluation Plan
To develop a technology transfer plan, these six steps should be followed:
(1) Identify the technology transfer criterion being served.
(2) Describe how this project serves the criterion.
(3) Describe the technology transfer mechanisms to be used.
(4) Identify the technology transfer measures to be used, and describe how the information will be acquired.
(5) Assess the value of the data to be collected against the costs of collection.
(6) Assign priorities to the proposed measures.For example, the plan may focus on increasing the involvement of private industry programs. This criterion is being served through a variety of means by which specific interested industrial representatives will be involved.
In the case of the Ames Laboratory Technology Maturation Project, seven technology transfer mechanisms are planned: (1) a CRADA, (2) licenses, (3) facilities use, (4) personnel exchange, (5) publications/data bases, (6) demonstrations/workshops, and (7) informal collaborations.
Determining Measures
(1) Number of new technologies selected or implemented
(2) Percentage of site treatment plans or other compliance instruments that consider new or emerging technologies in the decision-making and peer review process
(3) Cost savings through implementation of new or improved technologies
(4) Time savings through implementation of new or improved technologies
(5) Improvement in client/customer satisfaction among users, regulators, stakeholders, etc.The development of a technology transfer evaluation plan is an added responsibility for the program manager. The process has the advantage of identifying at the beginning the kind of measurement strategy that makes most sense in terms of the use of the data and the cost of collecting it. Measuring technology transfer after the fact is a very costly and usually not very satisfactory way of identifying success.
Using a timely process of technology transfer measurement will provide all parties the feedback necessary to continually improve its identification, development, and transfer of innovative technologies. In addition, information garnered from the measurements will help satisfy the Department, Congress, and the public that the effort is committed to its mission.
 |
 Last update 05/12/98 by Cuyamaca Web Team |