Evaluation of idea generation methods for conceptual design: Effectiveness metrics and design of experiments

Jami J. Shah, Santosh V. Kulkarni, Noe Vargas-Hernandez

Research output: Contribution to journalArticlepeer-review

356 Scopus citations


A wide range of formal methods have been devised and used for idea generation in conceptual design. Experimental evidence is needed to support claims regarding the effectiveness of these methods in promoting idea generation in engineering design. Towards that goal this paper presents a set of effectiveness metrics experimental methods, data collection and analysis techniques. Statistically based Design of Experiments (DOE) principles were used in developing the guidelines. Four classes of operating variables were considered to characterize the design problem and the environment. The effectiveness metrics proposed are based on outcome and consists of the quantity, quality, novelty, and variety of ideas generated. Two experimental approaches have been developed. In the Direct Method, the influence of the type of design problem and various parameters related to the procedure of an idea generation method is measured by using the method in its entirety. In the Indirect Method, each idea generation method is decomposed into key components and its overall effectiveness is predicted by experimentally studying the effectiveness of its components and their mutual interactions.

Original languageEnglish (US)
Pages (from-to)377-384
Number of pages8
JournalJournal of Mechanical Design, Transactions of the ASME
Issue number4
StatePublished - Dec 2000

ASJC Scopus subject areas

  • Mechanics of Materials
  • Mechanical Engineering
  • Computer Science Applications
  • Computer Graphics and Computer-Aided Design


Dive into the research topics of 'Evaluation of idea generation methods for conceptual design: Effectiveness metrics and design of experiments'. Together they form a unique fingerprint.

Cite this