管理科学与工程系学术论坛第一百讲

主 题：微任务众包中群体智慧的聚合与评估

主讲人: 魏煊 (亚利桑那大学商学院，博士生)

时 间：2019-07-26 14:00

地 点：明商1008会议室

语 言：中英文

讲座摘要:   

Microtask crowdsourcing has emerged as a cost-effective approach to obtaining large-scale labeled data in a wide range of applications. However, significant concerns still remain for the major stakeholders—task requesters and crowdsourcing platforms. Task requesters attempt to maximize the data quality given the limited budget and the crowdsourcing platforms need to evaluate the participating workers to foster the creation of a healthy and sustainable crowdsourcing ecosystem. Existing literature on learning from crowds has mainly focused on the single-label (i.e., binary and multi-class) setting, which prevents the application of microtask crowdsourcing to a wide range of business applications in which each item can be associated with multiple labels simultaneously. In this paper, we consider the problem of learning from crowds in the general multi-label setting, which includes previously studied single-label crowdsourcing problems as special cases. We propose a new Bayesian hierarchical model for the underlying annotation process of crowd workers and introduce a mixture of Bernoulli distribution to capture the unknown label dependency. An efficient variational inference procedure is then developed to jointly infer ground truth labels, worker reliability in terms of sensitivity and specificity, and label dependency. Results based on extensive simulation experiments and a real-world MTurk experiment indeed confirm that the proposed approach outperforms other competing methods, highlighting the necessity to model both worker quality and label dependency when learning from crowdsourced multi-label annotations.

主讲人简介 :

Xuan Wei is a Ph.D. candidate in the Department of Management Information Systems at the University of Arizona. His research interests include crowd intelligence, false news detection, intention mining in social media, etc. He received his B.S. degree in Mathematics and Applied Mathematics from the Shanghai Jiao Tong University, China. He has published papers in journals and conference proceedings such as TKDD, International Journal of Intelligent Systems, PACIS, and ICDE. He also has several working papers that receive Major Revision at journals such as MISQ, ISR, JOC, and DSS. He has presented his research at major conferences and workshops including INFORMS, CIST, and the INFORMS Workshop on Data Science. He received Best Paper Award in WITS 2018, and Best Paper Award Runner-up in INFORMS Workshop on Data Science 2017.