多模态大模型 - arXivDaily 专题

2209.14792 2022-09-30 cs.CV cs.AI cs.LG 62%

Make-A-Video: Text-to-Video Generation without Text-Video Data

Uriel Singer, Adam Polyak, Thomas Hayes, Xi Yin, Jie An, Songyang Zhang, Qiyuan Hu, Harry Yang, Oron Ashual, Oran Gafni, Devi Parikh, Sonal Gupta, Yaniv Taigman

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2107.13144 2022-09-15 cs.CV cs.AI 62%

Content-aware Directed Propagation Network with Pixel Adaptive Kernel Attention

Min-Cheol Sagong, Yoon-Jae Yeo, Seung-Won Jung, Sung-Jea Ko

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

2208.06658 2022-08-16 cs.CV cs.AI 62%

ULDGNN: A Fragmented UI Layer Detector Based on Graph Neural Networks

Jiazhi Li, Tingting Zhou, Yunnong Chen, Yanfang Chang, Yankun Zhen, Lingyun Sun, Liuqing Chen

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2206.09378 2022-06-22 cs.CL cs.CV 62%

A Self-Guided Framework for Radiology Report Generation

Jun Li, Shibo Li, Ying Hu, Huiren Tao

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.CL

Comments 11 pages, 3 figures, accepted by Medical Image Computing and Computer Assisted Intervention 2022(MICCAI 2022)

2205.11273 2022-06-22 cs.CV cs.AI 62%

GR-GAN: Gradual Refinement Text-to-image Generation

Bo Yang, Fangxiang Feng, Xiaojie Wang

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by ICME 2022

2110.02624 2022-05-02 cs.CV cs.AI 62%

CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation

Aditya Sanghi, Hang Chu, Joseph G. Lambourne, Ye Wang, Chin-Yi Cheng, Marco Fumero, Kamal Rahimi Malekshan

专题命中多模态生成：image-text(abstract)；分类 cs.CV、cs.AI

Comments Accepted by CVPR 2022

2204.13590 2022-04-29 cs.CV cs.AI cs.LG cs.RO 62%

Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms

Nachuan Ma, Jiahe Fan, Wenshuo Wang, Jin Wu, Yu Jiang, Lihua Xie, Rui Fan

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments accepted to Transportation Safety and Environment

2203.14101 2022-04-21 cs.LG cs.AI cs.CL 62%

A Roadmap for Big Model

Sha Yuan, Hanyu Zhao, Shuai Zhao, Jiahong Leng, Yangxiao Liang, Xiaozhi Wang, Jifan Yu, Xin Lv, Zhou Shao, Jiaao He, Yankai Lin, Xu Han, Zhenghao Liu, Ning Ding, Yongming Rao, Yizhao Gao, Liang Zhang, Ming Ding, Cong Fang, Yisen Wang, Mingsheng Long, Jing Zhang, Yinpeng Dong, Tianyu Pang, Peng Cui, Lingxiao Huang, Zheng Liang, Huawei Shen, Hui Zhang, Quanshi Zhang, Qingxiu Dong, Zhixing Tan, Mingxuan Wang, Shuo Wang, Long Zhou, Haoran Li, Junwei Bao, Yingwei Pan, Weinan Zhang, Zhou Yu, Rui Yan, Chence Shi, Minghao Xu, Zuobai Zhang, Guoqiang Wang, Xiang Pan, Mengjie Li, Xiaoyu Chu, Zijun Yao, Fangwei Zhu, Shulin Cao, Weicheng Xue, Zixuan Ma, Zhengyan Zhang, Shengding Hu, Yujia Qin, Chaojun Xiao, Zheni Zeng, Ganqu Cui, Weize Chen, Weilin Zhao, Yuan Yao, Peng Li, Wenzhao Zheng, Wenliang Zhao, Ziyi Wang, Borui Zhang, Nanyi Fei, Anwen Hu, Zenan Ling, Haoyang Li, Boxi Cao, Xianpei Han, Weidong Zhan, Baobao Chang, Hao Sun, Jiawen Deng, Chujie Zheng, Juanzi Li, Lei Hou, Xigang Cao, Jidong Zhai, Zhiyuan Liu, Maosong Sun, Jiwen Lu, Zhiwu Lu, Qin Jin, Ruihua Song, Ji-Rong Wen, Zhouchen Lin, Liwei Wang, Hang Su, Jun Zhu, Zhifang Sui, Jiajun Zhang, Yang Liu, Xiaodong He, Minlie Huang, Jian Tang, Jie Tang

专题命中多模态生成：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments This report has been withdrawn by the authors due to critical issues in Section 2.3.1 of Article 2

2201.05986 2022-04-20 cs.CV cs.MM 62%

Audio-Driven Talking Face Video Generation with Dynamic Convolution Kernels

Zipeng Ye, Mengfei Xia, Ran Yi, Juyong Zhang, Yu-Kun Lai, Xuwei Huang, Guoxin Zhang, Yong-jin Liu

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.MM

Comments in IEEE Transactions on Multimedia

2111.11133 2022-04-07 cs.CV cs.CL cs.LG 62%

L-Verse: Bidirectional Generation Between Image and Text

Taehoon Kim, Gwangmo Song, Sihaeng Lee, Sangyun Kim, Yewon Seo, Soonyoung Lee, Seung Hwan Kim, Honglak Lee, Kyunghoon Bae

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to CVPR 2022 as Oral Presentation (18 pages, 14 figures, 4 tables)

2204.02030 2022-04-06 cs.CL cs.AI 62%

$\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation

Yu Bao, Hao Zhou, Shujian Huang, Dongqi Wang, Lihua Qian, Xinyu Dai, Jiajun Chen, Lei Li

专题命中多模态生成：multi-modal(abstract)；分类 cs.CL、cs.AI

Comments 12 pages, 5 figures, 6 tables. Accepted as a long paper in the main conference of ACL-2022

2109.03892 2022-03-28 cs.CL cs.AI cs.LG 62%

Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models

Steven Y. Feng, Kevin Lu, Zhuofu Tao, Malihe Alikhani, Teruko Mitamura, Eduard Hovy, Varun Gangal

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Accepted to AAAI 2022. Code at https://github.com/styfeng/VisCTG

2203.07613 2022-03-16 cs.CL cs.CV 62%

CARETS: A Consistency And Robustness Evaluative Test Suite for VQA

Carlos E. Jimenez, Olga Russakovsky, Karthik Narasimhan

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments ACL 2022

2112.04928 2021-12-10 cs.CV cs.CL cs.LG 62%

Self-Supervised Image-to-Text and Text-to-Image Synthesis

Anindya Sundar Das, Sriparna Saha

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments ICONIP 2021 : The 28th International Conference on Neural Information Processing

Journal ref ICONIP 2021. Lecture Notes in Computer Science, vol 13111, pp 415-426. Springer, Cham

2111.12417 2021-11-25 cs.CV cs.AI 62%

NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion

Chenfei Wu, Jian Liang, Lei Ji, Fan Yang, Yuejian Fang, Daxin Jiang, Nan Duan

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

2110.15727 2021-11-01 cs.CL cs.AI 62%

Calling to CNN-LSTM for Rumor Detection: A Deep Multi-channel Model for Message Veracity Classification in Microblogs

Abderrazek Azri, Cécile Favre, Nouria Harbi, Jérôme Darmont, Camille Noûs

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

Journal ref Joint European Conference on Machine Learning and Knowledge Discovery in Databases (ECML PKDD 2021), Sep 2021, Bilbao, Spain. pp.497-513

2108.08955 2021-08-23 cs.CV cs.CL 62%

CIGLI: Conditional Image Generation from Language & Image

Xiaopeng Lu, Lynnette Ng, Jared Fernandez, Hao Zhu

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 5 pages

2108.04350 2021-08-11 cs.CV cs.AI 62%

VirtualConductor: Music-driven Conducting Video Generation System

Delong Chen, Fan Liu, Zewen Li, Feng Xu

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.AI

Comments Accepted by IEEE International Conference on Multimedia and Expo (ICME) 2021, demo track. Best demo

2107.04619 2021-07-13 cs.CV cs.AI cs.LG cs.RO 62%

Diverse Video Generation using a Gaussian Process Trigger

Gaurav Shrivastava, Abhinav Shrivastava

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments International Conference on Learning Representations, 2021

2101.00265 2021-05-24 cs.CV cs.CL cs.LG 62%

VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words

Xiaopeng Lu, Tiancheng Zhao, Kyusong Lee

专题命中多模态生成：cross-modal(abstract)；分类 cs.CV、cs.CL

Comments Accepted to ACL2021 (10 pages)

2011.07680 2020-11-17 cs.CL cs.AI cs.LG 62%

Reinforced Medical Report Generation with X-Linear Attention and Repetition Penalty

Wenting Xu, Chang Qi, Zhenghua Xu, Thomas Lukasiewicz

专题命中多模态生成：multi-modal(abstract)；分类 cs.CL、cs.AI

2010.01986 2020-10-06 cs.LG cs.AI cs.CL stat.ML 62%

A Spherical Hidden Markov Model for Semantics-Rich Human Mobility Modeling

Wanzheng Zhu, Chao Zhang, Shuochao Yao, Xiaobin Gao, Jiawei Han

专题命中多模态生成：multi-modal(abstract)；分类 cs.CL、cs.AI

2005.11267 2020-05-25 cs.AI cs.CL cs.RO 62%

Givenness Hierarchy Theoretic Cognitive Status Filtering

Poulomi Pal, Lixiao Zhu, Andrea Golden-Lasher, Akshay Swaminathan, Tom Williams

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

Comments To be published in the proceedings of the 2020 Annual Meeting of the Cognitive Science Society (COGSCI). Supplemental materials available at https://osf.io/qse7y/

1911.11378 2019-11-27 cs.LG cs.CV cs.MM eess.IV stat.ML 62%

Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions

Osaid Rehman Nasir, Shailesh Kumar Jha, Manraj Singh Grover, Yi Yu, Ajit Kumar, Rajiv Ratn Shah

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.MM

1905.13456 2019-10-10 eess.IV cs.AI cs.CV 62%

Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection

Changhee Han, Leonardo Rundo, Ryosuke Araki, Yudai Nagano, Yujiro Furukawa, Giancarlo Mauri, Hideki Nakayama, Hideaki Hayashi

专题命中多模态生成：multimodal(abstract)；分类 cs.CV、cs.AI

Comments 12 pages, 7 figures, accepted to IEEE ACCESS

1909.09788 2019-09-24 cs.CL cs.AI cs.NE 62%

Visuallly Grounded Generation of Entailments from Premises

Somaye Jafaritazehjani, Albert Gatt, Marc Tanti

专题命中多模态生成：multimodal(abstract)；分类 cs.CL、cs.AI

Comments Proceedings of the 12th International Conference on Natural Language Generation (INLG 2019), 11 pages, 5 figures

1810.10395 2018-10-25 cs.CV cs.AI cs.LG 62%

LoGAN: Generating Logos with a Generative Adversarial Neural Network Conditioned on color

Ajkel Mino, Gerasimos Spanakis

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.AI

Comments 6 page, ICMLA18

1808.04446 2018-10-15 cs.CV cs.CL cs.LG stat.ML 62%

Visual Reasoning with Multi-hop Feature Modulation

Florian Strub, Mathieu Seurin, Ethan Perez, Harm de Vries, Jérémie Mary, Philippe Preux, Aaron Courville, Olivier Pietquin

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments In Proc of ECCV 2018

1809.10274 2018-09-28 cs.LG cs.CL cs.CV stat.ML 62%

Semantically Invariant Text-to-Image Generation

Shagan Sah, Dheeraj Peri, Ameya Shringi, Chi Zhang, Miguel Dominguez, Andreas Savakis, Ray Ptucha

专题命中多模态生成：multi-modal(abstract)；分类 cs.CV、cs.CL

Comments 5 papers, 5 figures, Published in 2018 25th IEEE International Conference on Image Processing (ICIP)

1809.01997 2018-09-11 cs.AI cs.CL cs.LG 62%

Dual Ask-Answer Network for Machine Reading Comprehension

Han Xiao, Feng Wang, Jianfeng Yan, Jingyao Zheng

专题命中多模态生成：cross-modal(abstract)；分类 cs.CL、cs.AI

Comments 8 pages, 5 figures, 4 tables. Code is available at https://github.com/hanxiao/daanet

1. 多模态生成 4959 篇

Make-A-Video: Text-to-Video Generation without Text-Video Data

Content-aware Directed Propagation Network with Pixel Adaptive Kernel Attention

ULDGNN: A Fragmented UI Layer Detector Based on Graph Neural Networks

A Self-Guided Framework for Radiology Report Generation

GR-GAN: Gradual Refinement Text-to-image Generation

CLIP-Forge: Towards Zero-Shot Text-to-Shape Generation

Computer Vision for Road Imaging and Pothole Detection: A State-of-the-Art Review of Systems and Algorithms

A Roadmap for Big Model

Audio-Driven Talking Face Video Generation with Dynamic Convolution Kernels

L-Verse: Bidirectional Generation Between Image and Text

$\textit{latent}$-GLAT: Glancing at Latent Variables for Parallel Text Generation

Retrieve, Caption, Generate: Visual Grounding for Enhancing Commonsense in Text Generation Models

CARETS: A Consistency And Robustness Evaluative Test Suite for VQA

Self-Supervised Image-to-Text and Text-to-Image Synthesis

NÜWA: Visual Synthesis Pre-training for Neural visUal World creAtion

Calling to CNN-LSTM for Rumor Detection: A Deep Multi-channel Model for Message Veracity Classification in Microblogs

CIGLI: Conditional Image Generation from Language & Image

VirtualConductor: Music-driven Conducting Video Generation System

Diverse Video Generation using a Gaussian Process Trigger

VisualSparta: An Embarrassingly Simple Approach to Large-scale Text-to-Image Search with Weighted Bag-of-words

Reinforced Medical Report Generation with X-Linear Attention and Repetition Penalty

A Spherical Hidden Markov Model for Semantics-Rich Human Mobility Modeling

Givenness Hierarchy Theoretic Cognitive Status Filtering

Text2FaceGAN: Face Generation from Fine Grained Textual Descriptions

Combining Noise-to-Image and Image-to-Image GANs: Brain MR Image Augmentation for Tumor Detection

Visuallly Grounded Generation of Entailments from Premises

LoGAN: Generating Logos with a Generative Adversarial Neural Network Conditioned on color

Visual Reasoning with Multi-hop Feature Modulation

Semantically Invariant Text-to-Image Generation

Dual Ask-Answer Network for Machine Reading Comprehension