A threshold for a q-sorting methodology for computer-adaptive surveys

Conference paper


Sabbaghan, S, Gardner, L and Chua, CEH (2017). A threshold for a q-sorting methodology for computer-adaptive surveys. pp. 2896-2906
AuthorsSabbaghan, S, Gardner, L and Chua, CEH
TypeConference paper
Abstract

© 2017 Proceedings of the 25th European Conference on Information Systems, ECIS 2017. All rights reserved. Computer-Adaptive Surveys (CAS) are multi-dimensional instruments where questions asked of respondents depend on the previous questions asked. Due to the complexity of CAS, little work has been done on developing methods for validating their content and construct validity. We have created a new q-sorting technique where the hierarchies that independent raters develop are transformed into a quantitative form, and that quantitative form is tested to determine the inter-rater reliability of the individual branches in the hierarchy. The hierarchies are then successively transformed to test if they branch in the same way. The objective of this paper is to identify suitable measures and a “good enough” threshold for demonstrating the similarity of two CAS trees. To find suitable measures, we perform a set of bootstrap simulations to measure how various statistics change as a hypothetical CAS deviates from a “true” version. We find that the 3 measures of association, Goodman and Kruskal's Lambda, Cohen's Kappa, and Goodman and Kruskal's Gamma together provide information useful for assessing construct validity in CAS. In future work we are interested in both finding a “good enough” threshold(s) for assessing the overall similarity between tree hierarchies and diagnosing causes of disagreements between the tree hierarchies.

Year2017
JournalProceedings of the 25th European Conference on Information Systems, ECIS 2017
Accepted author manuscript
License
All rights reserved
File Access Level
Open
Publication dates
Print01 Jan 2017
Publication process dates
Deposited03 Nov 2020
ISBN9780991556700
Page range2896-2906
Permalink -

https://openresearch.lsbu.ac.uk/item/8v07x

Download files


Accepted author manuscript
  • 67
    total views
  • 29
    total downloads
  • 2
    views this month
  • 0
    downloads this month

Export as

Related outputs

A Variant Q-Sorting Methodology for Building Diagnostic Trees
Sabbaghan, S., Chua, C. and Gardner, L. (2021). A Variant Q-Sorting Methodology for Building Diagnostic Trees. IEEE Transactions on Engineering Management. https://doi.org/10.1109/TEM.2021.3078582.
Statistical measurement of trees’ similarity
Sabbaghan, S., Cecil Eng Huang Chua and Lesley Ann Gardner (2020). Statistical measurement of trees’ similarity. Quality and Quantity: international journal of methodology. https://doi.org/10.1007/s11135-019-00957-8
A test of a computer-adaptive survey using online reviews
Sabbaghan, S, Chua, CEH and Gardner, LA (2018). A test of a computer-adaptive survey using online reviews.
Computer-Adaptive Surveys (CAS) as a means of answering questions of why
Sabbaghan, S, Gardner, L and Chua, CEH (2017). Computer-Adaptive Surveys (CAS) as a means of answering questions of why.
A Q-sorting methodology for Computer-Adaptive Surveys - Style "Research"
Sabbaghan, S, Gardner, L and Chua, CEH (2016). A Q-sorting methodology for Computer-Adaptive Surveys - Style "Research".