Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
Pluralis v0.1:迈向用于人工智能风险与可靠性的多元文化、多模态、多语言基准测试
Alicia Parrish, Rajat Shinde, Sanket Badhe, Xinyi Bai, Sree Bhargavi Balija, Hua-Rong Chu, Emilio Ferrara, Armstrong Foundjem, Rajat Ghosh, Aakash Gupta, Xuanli He, Ong Chen Hui, Minji Jung, Madhangi Karimanal, Faiza Khan Khattak, Boryoung Kim, Eugenia Kim, Liliya Lavitas, Seok Min Lim, Victor Lu, Jim Moirangthem, Dhivya Nagasubramanian, Deepak Pandita, Sita Rajagopal, Geetha Raju, Evgeniia Razumovskaia, Aravind Reddy, Federico Ricciuti, Nobin Sarwar, Sungpil Shin, Sunayana Sitaram, Snehal Thorat, Tharindu Cyril Weerasooriya, Jasmijn Bastings, Joachim Baumann, Kongtao Chen, Murali Emani, Mariya Hendriksen, Jiho Jin, Jun Seong Kim, Younghoon Ko, Alicja Kwasniewska, Minjae Lee, Tom Wei-cyuan Lin Kashyap Ramanandula Manjusha, Junho Myung, Junyeong Park, Roma Patel, Shyam Ratan, Sudarsun Santhiappan, Priyanka Suresh, Tuesday, Ksheeraj Sai Vepuri Laura Amortegui-Ordonez, Claire Dennis, Minsuk Kahng, Chris Knotz, Alice Oh, Balaraman Ravindran, Soojung Ryu William Bartholomew, Hiwot Tesfaye, Lora Aroyo
机构
*
Google DeepMind(谷歌DeepMind)
;
University of Alabama in Huntsville(阿拉巴马大学亨茨维尔分校)
;
Google(谷歌)
;
University of Missouri Columbia(密苏里大学哥伦比亚分校)
;
Chunghwa Telecom Laboratories(春木电信实验室)
;
University of Southern California(南加州大学)
;
Polytechnique Montreal(蒙特利尔理工学院)
;
Nutanix
;
ThinkEvolve Labs(ThinkEvolve实验室)
;
UCL(伦敦大学学院)
;
Infocomm Media Development Authority(信息通信媒体发展局)
;
Monark Health(Monark健康)
;
Seoul National University(首尔国立大学)
;
Microsoft(微软)
;
Centre for Responsible AI (CeRAI), Wadhwani School of Data Science and AI (WSAI), Indian Institute of Technology Madras(负责任人工智能中心(CeRAI)、瓦达威人工智能学校(WSAI)、印度理工学院马德拉斯分校)
;
University of Maryland, Baltimore County(马里兰大学巴尔的摩县分校)
;
Microsoft Research India(微软印度研究院)
;
Stanford University(斯坦福大学)
;
Argonne National Laboratory(阿贡国家实验室)
;
University of Oxford(牛津大学)
;
KAIST(韩国科学技术院)
;
Yonsei University(延世大学)
;
Amazon(亚马逊)
;
UIUC(伊利诺伊大学香槟分校)
;
Rochester Institute of Technology(罗切斯特理工学院)
;
Xenoscube Inc.(Xenoscube公司)
;
Korea AI Safety Institute (K-AISI)(韩国人工智能安全研究所(K-AISI))
;
MLCommons
;
CommonGround
;
Artifex Labs(Artifex实验室)
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
ZeroBench:当代大型多模态模型的一个不可能的视觉基准测试
Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma, Akash Gupta, Samuel Roberts, Ioana Croitoru, Simion-Vlad Bogolin, Jialu Tang, Florian Langer, Vyas Raina, Vatsal Raina, Hanyi Xiong, Vishaal Udandarao, Jingyi Lu, Shiyang Chen, Sam Purkis, Tianshuo Yan, Wenye Lin, Gyungin Shin, Qiaochu Yang, Anh Totti Nguyen, David I. Atkinson, Aaditya Baranwal, Alexandru Coca, Mikah Dang, Sebastian Dziadzio, Jakob D. Kunz, Kaiqu Liang, Alexander Lo, Brian Pulfer, Steven Walton, Charig Yang, Kai Han, Samuel Albanie
机构
*
University of Cambridge(剑桥大学)
;
University of Alberta(阿尔伯塔大学)
;
The University of Hong Kong(香港大学)
;
University of Oxford(牛津大学)
;
Northeastern University(东北大学)
;
Astadeus
;
College of Southern Maryland(马里兰州南部学院)
;
University of Geneva(日内瓦大学)
;
University of Oregon(俄勒冈大学)