Staff Publications

Staff Publications

  • external user (warningwarning)
  • Log in as
  • language uk
  • About

    'Staff publications' is the digital repository of Wageningen University & Research

    'Staff publications' contains references to publications authored by Wageningen University staff from 1976 onward.

    Publications authored by the staff of the Research Institutes are available from 1995 onwards.

    Full text documents are added when available. The database is updated daily and currently holds about 240,000 items, of which 72,000 in open access.

    We have a manual that explains all the features 

Record number 510494
Title Probing the power of Apollo : methodological challenges and opportunities of the Delphi methods for developing rankings
Author(s) Meijering, Jurian V.
Source Wageningen University. Promotor(en): Adri van den Brink, co-promotor(en): Hilde Tobi. - Wageningen : Wageningen University - ISBN 9789462579187 - 184
Department(s) WASS
Landscape Architecture
Publication type Dissertation, internally prepared
Publication year 2016
Keyword(s) delphi method - methodology - ranking - landscape architecture - delphimethode - methodologie - rangordening - landschapsarchitectuur
Categories Research Methodology / Landscape Architecture (General)
Abstract

The Delphi method is a structured data-collection process aimed at facilitating experts to achieve a certain level of agreement on a complex problem. In several subsequent rounds experts give their opinion on the problem of interest, usually by means of a standardized questionnaire. After each round experts receive controlled opinion feedback in the form of a summary of the findings. Based on this feedback experts are allowed to reconsider their opinion in the next round.

Although the Delphi method seems to provide opportunities for developing rankings, research into its application to the development of rankings seems to be non-existent. Furthermore, the Delphi method has several unresolved methodological issues, in particular regarding the measurement of agreement and the provision of controlled opinion feedback. Therefore, the general objective of this PhD project was to find out what the methodological challenges and opportunities are of the Delphi method for developing rankings.

To achieve this objective, four studies were conducted. In the first study, simulations were performed to find out how various agreement indices behave within and across the rounds of a Delphi study. In the second study, the Delphi method was applied to develop a ranking of research domains in landscape architecture. An experiment within the study investigated the effect of two types of controlled opinion feedback on the drop-out rate, experts’ degree of opinion change, and the level of agreement among experts. The third study investigated the methodological characteristics of six urban sustainability rankings, which resulted in a recommendation to define and operationalize the concept ‘urban sustainability’ using the Delphi method. Therefore, in the fourth and final study the Delphi method was applied to find out which components experts find most relevant for defining and measuring urban sustainability. An experiment within this study provided insight into the effect of feeding back experts’ initial ratings on the degree of opinion change and the level of agreement.

Overall, this PhD project showed how the Delphi method may be used to: (1) obtain a ranking of objects on a ranking attribute and (2) obtain a definition and operationalization of a complex ranking attribute. It was also shown that these applications of the Delphi method do not come without challenges. First, selection criteria and search strategies need to be developed by which sufficient numbers of different types of experts may be found. Second, for the first Delphi questionnaire a parsimonious list of items (e.g. ranking objects or components of a ranking attribute) needs to be drawn up that largely covers the potentially great diversity of existing items. Third, careful considerations need to be made about which types of information (i.e. summary statistics, rationales, experts’ own initial ratings) to feed back to experts after every round as this may influence various Delphi outcome measures. Fourth, the choice for a particular consensus, agreement, or association index needs to be carefully made and justified as it determines the level of agreement among experts that is obtained. Finally, this project showed that by means of a simulation study and two real-world Delphi experiments new knowledge about the functioning of the Delphi method could be acquired. More of these studies are needed to establish evidence-based guidelines and to uncover the full potential of the Delphi method for developing rankings.

Comments
There are no comments yet. You can post the first one!
Post a comment
 
Please log in to use this service. Login as Wageningen University & Research user or guest user in upper right hand corner of this page.