Evaluating The Technology Transfer Project
It is important to evaluate technology transfer programs and projects on an ongoing
basis to determine their success and to incorporate lessons learned in future activities.
Measuring the results of technology transfer activities is an important component of
technology transfer projects.
The three basic reasons to measure technology transfer results are to (1) provide
accountability, (2) facilitate the process of technology transfer, and (3) demonstrate the
value of a particular technology transfer project.
Any publicly funded activity has to be accountable to higher authority, usually to a
program office, an agency, the chief executive (the President) and, ultimately, the U.S.
Congress and the taxpayer. Each of these levels expects answers to the following
questions: Did the project achieve the anticipated results? To what extent and why or why
not? What unanticipated results were achieved? These and similar questions will be asked
by management during the life of the project and at its conclusion.
Facilitate the Technology Transfer Process
We learn from both successes and failures encountered during the course of a project.
For example, a particular workshop or demonstration project may not have included all of
the stakeholders, and thus encountered some opposition or delay. Although the project may
have been successful, the exclusion is an obstacle that needs to be assessed, with the
appropriate lessons noted.
In another instance, a site demonstration may have been especially successful because
the project managers obtained access to the site and ensured that the organizations
responsible for the site were brought into the process at an early date. In another
example, a potential partner may have been discouraged from participating in an
exploratory demonstration or pilot because of unexpected terms and conditions that were
not fully presented to the potential partner in a timely fashion. In each of these
examples, there is significant opportunity to learn and thereby to improve the technology
Officials at all levels always want to know what the results are and whether the
project achieved its goals.
For instance, recent studies of just over 400 examples of the application of technology
from the National Aeronautics and Space Administration (NASA) demonstrated that the
technology users benefited from sales or savings of $33.5 billion because of successful
technology transfer. In a similar study for the Agricultural Research Service (ARS), sales
or savings totaling $14.3 billion were identified in 87 cases where users applied ARS
technology or science.
In addition, congressional hearings and studies by the General Accounting Office (GAO)
during the past several years indicate increasing interest in some type of measure for
technology transfer activities.
Technology transfer is rarely accomplished in a short period of time. The process may
not be complete for five or ten years. Therefore, it is not always possible to wait for
the end results; it is sometimes necessary to seek interim indicators that will
demonstrate progress toward the particular project goal.
It is important to identify intermediate products whenever possible. For example, a
project that uses remediation technology might include plans to use a variety of workshops
and demonstrations as interim steps. Appropriate indicators could include:
(1) target organizations most likely to make use of the technology
(2) the number of organizations represented at the workshop and the distribution of
attendees compared to the target audience originally identified
(3) the degree of interest and satisfaction gleaned from a brief evaluation by
participants at the conclusion of the workshop
(4) a willingness to explore further participation with the project staff as determined
by follow-on inquiries
(5) a follow-up of attendees at a three- to six-month interval after the workshop to
determine short-term results.
As a project approaches completion, or in instances where the completion of a transfer
is likely to extend past the end date of the project, other results might include
indicators such as (1) companies started or expanded, (2) jobs created or saved, or (3)
sales of services or products based upon the technology involved in the project. In
addition, sales or services of technology overseas or the development of subsidiaries
could indicate internationally competitive capability.
How Can You Measure Technology Transfer?
Some experts say a transfer is successful only when it becomes a profitable product or
process, while others claim a transfer is successful when the technology is at least
reviewed for possible use by another person or organization.
A great deal of technology is never commercialized due to (1) lack of financing; (2)
management failures; or (3) unanticipated obstacles in the timing of the introduction,
such as a shift in the economy, changed market conditions, or the sudden appearance of a
Organizations with limited resources for technology transfer may have to concentrate on
less costly mechanisms, such as publications. In fact, publishing often leads to the
transfer of knowledge. For example, a 1979 study by the University of Denver's Denver
Research Institute, reviewing various means used by the NASA Technology Utilization
Program, revealed that the most cost-effective means of technology transfer was via its
publication, Tech Briefs.
Measures of success in technology transfer need to be designed to cover the technology
transfer mission and the resources available to the organization. Even if
commercialization is a primary goal, the technology's value to further research or
improvement of a process should not be overlooked or minimized.
Three Dimensions of Technology Transfer Measurement
The three dimensions with which we are concerned in the measurement of technology
transfer are (1) the transfer mechanisms, (2) the time frame in which the transfer occurs,
and (3) the area of impact (or results). Each of these dimensions will have an important
role in determining the kind of measure that will provide the best information and be most
indicative of either progress or actual results from the particular transfer.
(1) Transfer Mechanisms
Usually formal mechanisms, such as contracts, CRADAs, facilities-use agreements,
licenses, and personnel exchanges, provide information that can be measured more easily.
This is not true of the less formal activities, such as technical assistance; informal
collaborations; and most workshop, publication, or data base activities.
(2) Time Frame
Technology transfer is often a rather lengthy process, so measures should be
appropriate for the time frame in which the information is to be collected. The type of
the measure will also depend on the stage of development of the technology. Technology
that is ready for widespread application can have valid measurements in the short or
On the other hand, a number of transfer activities will be long-term; therefore, you
will have to use surrogate or process measures for interim reporting. The three categories
of short, mid, and long-term are useful in assessing the types of measurements possible in
the time frame dimension.
A third dimension in measuring technology transfer is the consideration of the economic
or technical impacts.
Attention is usually focused on economic impacts because of interest in forecasting
sales, cost savings, and jobs created or saved. However, this information is often
difficult to collect, especially if proprietary information is involved.
Technical impacts are more easily determined because they are usually included as a
part of any project's goals, and the degree of improvement or substitution that has been
sought can be described and given a value--for example, a description of sites where the
technology has been selected for implementation or actually implemented.
Technology Transfer Program Criteria
A meaningful technology transfer plan should be based on a combination of criteria:
- Assists focus areas and cross cutting technology areas
- Assists with implementing waste management, environmental restoration, and
decontamination and decommissioning activities
- Increases the commercial value of industrial technology
- Increases the involvement of private industry
- Decreases the cost of waste management cleanup
- Encourages the commercialization of technologies
- Increases U.S. competitiveness or creates jobs
- Responds to political mandates
Steps in Developing a Technology Transfer Evaluation Plan
To develop a technology transfer plan, these six steps should be followed:
(1) Identify the technology transfer criterion being served.
(2) Describe how this project serves the criterion.
(3) Describe the technology transfer mechanisms to be used.
(4) Identify the technology transfer measures to be used, and describe how the information
will be acquired.
(5) Assess the value of the data to be collected against the costs of collection.
(6) Assign priorities to the proposed measures.
For example, the plan may focus on increasing the involvement of private industry
programs. This criterion is being served through a variety of means by which specific
interested industrial representatives will be involved.
In the case of the Ames Laboratory Technology Maturation Project, seven technology
transfer mechanisms are planned: (1) a CRADA, (2) licenses, (3) facilities use, (4)
personnel exchange, (5) publications/data bases, (6) demonstrations/workshops, and (7)
(1) Number of new technologies selected or implemented
(2) Percentage of site treatment plans or other compliance instruments that consider new
or emerging technologies in the decision-making and peer review process
(3) Cost savings through implementation of new or improved technologies
(4) Time savings through implementation of new or improved technologies
(5) Improvement in client/customer satisfaction among users, regulators, stakeholders,
The development of a technology transfer evaluation plan is an added responsibility for
the program manager. The process has the advantage of identifying at the beginning the
kind of measurement strategy that makes most sense in terms of the use of the data and the
cost of collecting it. Measuring technology transfer after the fact is a very costly and
usually not very satisfactory way of identifying success.
Using a timely process of technology transfer measurement will provide all parties the
feedback necessary to continually improve its identification, development, and transfer of
innovative technologies. In addition, information garnered from the measurements will help
satisfy the Department, Congress, and the public that the effort is committed to its
Back I Index I Forward