Xuemeng Song
    Associate Professor
    Department of Computer Science and Technology
    Shandong University
    Address： Shandong University, Jimo, Tsingtao, China, 266237
    Email： sxmustc at gmail dot com
News Biography Awards Services Publications Research
News
Opening for master/PhD students: I am recruiting self-motivated master/PhD students to collaborate on research domains such as information retrieval and multimedia analysis. (Enrollment of 2024 Sep).
Biography
Xuemeng Song is an Associate Professor in Shandong University, and IEEE senior member. She got her PhD degree from National University of Singapore (NUS) and her Bacherlor degree from Universitiy of Science and Technology of China (USTC) in 2016 and 2012, respectively. Her research interests are information retrieval and social media analysis. She has published more than 70 papers in the top venues (e.g., IEEE TIP, IEEE TMM, ACM SIGIR, ACM MM, and ACM TOIS) and 4 books by Morgan & Claypool Publishers and Springer. In addition, she has served as reviewers for many top conferences and journals, such as TKDD, TMM, ICMR, and MMM.

近几年来，宋雪萌博士在信息检索和多媒体分析领域展开了一系列基础研究工作，取得了较好的研究积累。其在中国计算机学会推荐的国内外著名学术期刊与会议上发表论文70余篇，出版学术专著4部。据Google Scholar统计，其已发表论文共计被引用2300余次，h-index为27。 担任国际期刊IEEE TCSVT, IET Image Processing编委（AE）。同时，受到多项科研奖励，如2021年山东省科学技术进步奖一等奖，2022年AI 2000多媒体领域最具影响力学者提名奖(AMiner)，入选2022年全球AI华人青年学者榜单（百度），入选2023年AI华人女性青年学者榜单（百度）。 主持国家自然科学基金面上项目、青年项目、科技部重点研发计划子课题、山东省优秀青年基金、山东大学未来计划青年学者、阿里巴巴AIR计划等。

  • Google Scholar: https://scholar.google.com/citations?user=29gP4okAAAAJ&hl=en
  • DBLP: https://dblp.org/pid/147/9141.html
    • Awards
  • 2022, Global Top Chinese Young Scholars in Artificial Intelligence, Baidu
  • 2022, Fellow of 2022 Women in AI, AMiner (Only 23 Chinese Women Scholars)
  • 2022, AI 2000 Most Influential Scholar Award Honorable Mention, AMiner
  • 2021, First Prize of Science and Technology Progress Award, Shandong Province of China
  • 2021, Best Paper Honorable Mention, ChinaMM
  • 2018, Future Talent Scholars, Shandong University
    •
    Services
  • Chair of the ACM MM workshop on “Multimedia Computing towards Fashion Recommendation”, 2022
  • Invited keynote speaker of the workshop "Towards a Complete Analysis of People: From Face and Body to Clothes (T-CAP)" on ICIAP, 2022
  • Guest editor of the special issue “Research on multimodal representation learning with pretraining” for the Journal of Software, 2022
  • Invited speaker of the CCF-TCVRV (The sixth forum at Wuhan Textile University), 2021
  • Participate in the course construction of "Exploration of demonstration teaching on pattern recognition" of college artificial intelligence teaching resources construction project of the Ministry of Education, 2020
  • Guest editor of the special issue “Deep Learning for Multi-modal Social Media Analysis and Applications” for the journal Information Processing and Management, 2018
  • Editor board member of the journal Information Processing and Management since 2018
  • Program committee member of top conferences such as ACM SIGIR, MM, SIGKDD, WSDM, AAAI, IJCAI, and reviewers for top journals, such as ACM TOIS, IEEE TMM, IEEE TIP, and IEEE TKDE.
    • Publications
    1. Graph Learning for Fashion Compatibility Modeling.
      Weili Guan, Xuemeng Song, Xiaojun Chang, Liqiang Nie. Synthesis Lectures on Information Concepts, Retrieval, and Services. Springer, 2022.
    2. Answer Questions with Right Image Regions: A Visual Attention Regularization Approach.
      Yibing Liu, Yangyang Guo, Jianhua Yin, Xuemeng Song, Weifeng Liu, Liqiang Nie, Min Zhang. ACM TOMM, 2022.
    3. Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis.
      Teng Sun, Wenjie Wang, Liqiang Jing, Yiran Cui, Xuemeng Song, Liqiang Nie. In ACM MM, 2022. (Full Paper).
    4. Bi-directional Heterogeneous Graph Hashing towards Efficient Outfit Recommendation.
      Weili Guan, Xuemeng Song, Haoyu Zhang, Meng Liu, Chung-Hsing Yeh, Xiaojun Chang. In ACM MM, 2022. (Full Paper).
    5. Divide-and-Conquer Predictor for Unbiased Scene Graph Generation
      Xianjing Han, Xingning Dong, Xuemeng Song, Tian Gan, Yibing Zhan, Yan Yan, Liqiang Nie. IEEE TCSVT, 2022.
    6. DBiased-P: Dual-Biased Predicate Predictor for Unbiased Scene Graph Generation
      Xianjing Han, Xuemeng Song, Xingning Dong, Yinwei Wei, Meng Liu, Liqiang Nie. IEEE TMM, 2022.
    7. V2P: Vision-to-Prompt based Multi-Modal Product Summary Generation.
      Xuemeng Song, Liqiang Jing, Dengtian Lin, Zhongzhou Zhao, Haiqing Chen, Liqiang Nie. In ACM SIGIR, 2022. (Full Paper). Code
    8. Personalized Fashion Compatibility Modeling via Metapath-guided Heterogeneous Graph Learning.
      Weili Guan, Fangkai Jiao, Xuemeng Song, Haokun Wen, Chung-Hsing Yeh and Xiaojun Chang. In ACM SIGIR, 2022. (Full Paper)
    9. Stacked Hybrid-Attention and Group Collaborative Learning for Unbiased Scene Graph Generation.
      Xingning Dong, Tian Gan, Xuemeng Song, Jianlong Wu, Yuan Cheng, Liqiang Nie. In CVPR, 2022.
    10. MERIt: Meta-Path Guided Contrastive Learning for Logical Reasoning.
      Fangkai Jiao, Yangyang Guo, Xuemeng Song, Liqiang Nie. In ACL, 2022.
    11. Response Generation by Jointly Modeling Personalized Linguistic Styles and Emotions.
      Teng Sun, Chun Wang, Xuemeng Song, Fuli Feng, Liqiang Nie. ACM TOMM, 2022.
    12. Modality-Oriented Graph Learning Toward Outft Compatibility Modeling.
      Xuemeng Song, Shiting Fang, Xiaolin Chen, Yinwei Wei, Zhongzhou Zhao, Liqiang Nie. TMM, 2021. Pdf   Code
    13. Collocation and Try-on Network: Whether an outfit is compatible.
      Na Zheng, Xuemeng Song, Qingying Niu, Xue Dong, Yibing Zhan, Liqiang Nie. In ACM MM, 2021. (Full Paper) Pdf
    14. Complementary Factorization towards Outfit Compatibility Modeling.
      Tianyu Su, Xuemeng Song, Na Zheng, Weili Guan, Yan Li, Liqiang Nie. In ACM MM, 2021. (Full Paper) Pdf
    15. Multimodal Compatibility Modeling via Exploring the Consistent and Complementary Correlations.
      Weili Guan, Haokun Wen, Xuemeng Song, Chung-Hsing Yeh, Xiaojun Chang, Liqiang Nie. In ACM MM, 2021. (Full Paper) Pdf
    16. Comprehensive Linguistic-Visual Composition Network for Image Retrieval.
      Haokun Wen, Xuemeng Song, Xin Yang, Yibing Zhan, Liqiang Nie. In SIGIR, 2021. (Full Paper) Pdf   Code
    17. Adversarial-Enhanced Hybrid Graph Network for User Identity Linkage.
      Xiaolin Chen, Xuemeng Song, Guozhen Peng, Shanshan Feng, Liqiang Nie. In SIGIR, 2021. (Full Paper) Pdf
    18. Multimodal Activation: Awakening Dialog Robots without Wake Words.
      Liqiang Nie, Mengzhao Jia, Xuemeng Song, Ganglu Wu, Harry Cheng, and Jian Gu. In SIGIR, 2021. (Full Paper) Pdf
    19. Fine-Grained Privacy Detection with Graph-Regularized Hierarchical Attentive Representation Learning.
      Xiaolin Chen, Xuemeng Song, Ruiyang Ren, Lei Zhu, Zhiyong Cheng, Liqiang Nie. ACM Trans. Inf. Syst. 38(4): 37:1-37:26, 2020. Pdf
    20. Auxiliary Template-Enhanced Generative Compatibility Modeling.
      Jinhuan Liu, Xuemeng Song, Zhaochun Ren, Liqiang Nie, Zhaopeng Tu, Jun Ma. In IJCAI, 2020. (Full Paper) Pdf
    21. MGCM: Multi-modal generative compatibility modeling for clothing matching.
      Jinhuan Liu, Xuemeng Song, Zhumin Chen, Jun Ma. Neurocomputing 414: 215-224, 2020. Pdf
    22. Multi-modal Try-on-guided Fashion Compatibility Modelin.
      Xue Dong, Jianlong Wu, Xuemeng Song, Hongjun Dai, Liqiang Nie. In SIGIR, 2020. (Full Paper) Pdf
    23. Generative Attribute Manipulation Scheme for Flexible Fashion Search.
      Xin Yang, Xuemeng Song, Xianjing Han, Haokun Wen, Jie Nie, Liqiang Nie. In SIGIR, 2020. (Full Paper) Pdf
    24. Large-Scale Question Tagging via Joint Question-Topic Embedding Learning.
      Liqiang Nie, Yongqi Li, Fuli Feng, Xuemeng Song, Meng Wang, Yinglong Wang. ACM Trans. Inf. Syst. 38(2): 20:1-20:23, 2020. Pdf
    25. Compatibility Modeling: Data and Knowledge Applications for Clothing Matching.
      Xuemeng Song, Liqiang Nie, Yinglong Wang. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers. 2019. Link
    26. Multimodal Learning toward Micro-Video Understanding.
      Liqiang Nie, Meng Liu, Xuemeng Song. Synthesis Lectures on Image, Video, and Multimedia Processing. Morgan & Claypool Publishers. 2019. Link
    27. An End-to-End Attention-Based Neural Model for Complementary Clothing Matching.
      Jinhuan Liu, Xuemeng Song, Liqiang Nie, Tian Gan, Jun Ma. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2019. Pdf
    28. Neural Compatibility Modeling with Probabilistic Knowledge Distillation.
      Xianjing Han, Xuemeng Song, Yiyang Yao, Xin-Shun Xu, Liqiang Nie. IEEE Transactions on Image Processing (TIP), 2019. Pdf
    29. GP-BPR: Personalized Compatibility Modeling for Clothing Matching.
      Xuemeng Song, Xianjing Han, Yunkai Li, Jingyuan Chen, Xin-Shun Xu, Liqiang Nie. In MM, 2019. (Full Paper) Pdf  Code
      Dataset: We have released the dataset IQON3000 and features Features to facilitate the research community.
    30. Personalized Capsule Wardrobe Creation with Garment and User Modeling.
      Xue Dong, Xuemeng Song, Fuli Feng, Peiguang Jing, Xin-Shun Xu, Liqiang Nie. In MM, 2019. (Full Paper) Pdf  Code
    31. Virtually Trying on New Clothing with Arbitrary Poses.
      Na Zheng, Xuemeng Song, Zhaozheng Chen, Linmei Hu, Da Cao, Liqiang Nie. In MM, 2019. (Full Paper) Pdf  Code
    32. Seeking Micro-influencers for Brand Promotion.
      Tian Gan, Shaokun Wang, Meng Liu, Xuemeng Song, Yiyang Yao, Liqiang Nie. In MM, 2019. (Full Paper)
    33. Efficient Discrete Latent Semantic Hashing for Scalable Cross-modal Retrieval.
      Xu Lu, Lei Zhu, Zhiyong Cheng, Xuemeng Song, Huaxiang Zhang. Signal Processing, 2019.
    34. Neural Fashion Experts: I Know How to Make the Complementary Clothing Matching.
      Jinhuan Liu, Xuemeng Song, Zhumin Chen, Jun Ma. Neurocomputing, 2019. Pdf
    35. Prototype-guided Attribute-wise Interpretable Scheme for Clothing Matching.
      Xianjing Han, Xuemeng Song, Jianhua Yin, Yinglong Wang, Liqiang Nie. In SIGIR, 2019. (Full paper) Pdf  Code
    36. Supervised Hierarchical Cross-Modal Hashing.
      Changchang Sun, Xuemeng Song, Fuli Feng, Wayne Xin Zhao, Hao Zhang, Liqiang Nie. In SIGIR, 2019. (Full paper) Pdf
    37. User Attention-guided Multimodal Dialog Systems.
      Chen Cui, Wenjie Wang, Xuemeng Song, Minlie Huang, Xin-Shun Xu, Liqiang Nie. In SIGIR, 2019. (Full paper) Pdf
    38. Neural Compatibility Modeling with Attentive Knowledge Distillation.
      Xuemeng Song, Fuli Feng, Xianjing Han, Xin Yang, Liqiang Nie, Wei Liu. In SIGIR, 2018. (Full paper) Pdf  Code
    39. A Personal Privacy Preserving Framework: I Let You Know Who Can See What.
      Xuemeng Song, Xiang Wang, Liqiang Nie, Xiangnan He, Zhumin Chen, Wei Liu. In SIGIR, 2018. (Full paper) Pdf
    40. Adaptive Collaborative Similarity Learning for Unsupervised Multi-view Feature Selection.
      Xiao Dong, Lei Zhu, Xuemeng Song, Jingjing Li, Zhiyong Cheng. In IJCAI, 2018. (Full paper) Pdf
    41. SDMCH: Supervised Discrete Manifold-Embedded Cross-Modal Hashing.
      Xin Luo, Xiao-Ya Yin, Liqiang Nie, Xuemeng Song, Yongxin Wang, Xin-Shun Xu. In IJCAI, 2018. (Full paper) Pdf
    42. A^3NCF: An Adaptive Aspect Attention Model for Rating Prediction.
      Zhiyong Cheng, Ying Ding, Xiangnan He, Lei Zhu, Xuemeng Song, Mohan S. Kankanhalli. In IJCAI, 2018. (Full paper) Pdf
    43. Scalable Graph Based Non-negative Multi-view Embedding for Image Ranking.
      Shuhan Qi, Xuan Wang, Xi Zhang, Xuemeng Song, Zoe L. Jiang. Neurocomputing, 274: 29-36, 2018. Pdf
    44. Venue Prediction for Social Images by Exploiting Rich Temporal Patterns in LBSNs.
      Jingyuan Chen, Xiangnan He, Xuemeng Song, Hanwang Zhang, Liqiang Nie, Tat-Seng Chua. In MMM, 2018. (Full paper) Pdf
    45. Video Logo Removal Detection Based on Sparse Representation.
      Xiao Jin, Yuting Su, Liang Zou, Chengqian Zhang, Peiguang Jing, Xuemeng Song. Multimedia Tools and Applications, 2018.
    46. NeuroStylist: Neural Compatibility Modeling for Clothing Matching.
      Xuemeng Song, Fuli Feng, Jinhuan Liu, Zekun Li, Liqiang Nie, Jun Ma. In MM, 2017. (Full paper) Pdf  Code
      Dataset: We have released the dataset FashionVC to facilitate the research community.
    47. Unifying virtual and physical worlds: Learning toward local and global consistency.
      Xiang Wang, Liqiang Nie, Xuemeng Song, Dongxiang Zhang, Tat-Seng Chua. ACM Transactions on Information Systems (TOIS), 2017. Pdf
    48. Modeling Disease Progression via Multisource Multitask Learners: A Case Study With Alzheimer's Disease.
      Liqiang Nie, Luming Zhang, Lei Meng, Xuemeng Song, Xiaojun Chang, Xuelong Li. IEEE Transactions on Neural Network Learning System (TNNLS), 2017. Pdf
    49. Micro Tells Macro: Predicting the Popularity of Micro-Videos via a Transductive Model.
      Jingyuan Chen, Xuemeng Song, Liqiang Nie, Xiang Wang, Hanwang Zhang, Tat-Seng Chua. In MM, 2016. (Full paper) Pdf
    50. Learning from Multiple Social Networks.
      Liqiang Nie, Xuemeng Song, Tat-Seng Chua. Synthesis Lectures on Information Concepts, Retrieval, and Services. Morgan & Claypool Publishers, 2016. Link
    51. Volunteerism Tendency Prediction via Harvesting Multiple Social Networks.
      Xuemeng Song, Zhao-Yan Ming, Liqiang Nie, Yi-Liang Zhao, Tat-Seng Chua. ACM Transactions on Information Systems (TOIS), 2016. Pdf
    52. Fusing Social Networks with Deep Learning for Volunteerism Tendency Prediction.
      Yongpo Jia, Xuemeng Song, Jingbo Zhou, Li Liu, Liqiang Nie, David S. Rosenblum. In AAAI, 2016. (Full paper) Pdf
    53. Interest Inference via Structure-Constrained Multi-Source Multi-Task Learning.
      Xuemeng Song, Liqiang Nie, Luming Zhang, Maofu Liu, Tat-Seng Chua. In IJCAI, 2015. (Full paper) Pdf
    54. Multiple Social Network Learning and Its Application in Volunteerism Tendency Prediction.
      Xuemeng Song, Liqiang Nie, Luming Zhang, Mohammad Akbari, Tat-Seng Chua. In SIGIR, 2015. (Full paper) Pdf
    55. Enrichment of User Profiles Across Multiple Online Social Networks for Volunteerism Matching for Social Enterprise. Xuemeng Song. In SIGIR, 2014. Pdf
    Research

    Fashion Analysis towards Clothing Matching

    According to the Goldman Sachs, the 2016 online retail market of China for fashion products, including apparel, footwear, and accessories, has reached 187.5 billion US dollars, which demonstrates people’s great demand for clothing. In fact, apart from physiological needs, people also have esteem needs of clothes as dressing properly is of importance in daily life. As each outfit usually involves multiple complementary items (e.g., tops, bottoms, and shoes), the key to a proper outfit lies in the harmonious clothing matching to a great extent. However, not everyone is a naturalborn fashion stylist, which makes choosing the matching clothes a tedious and even annoying daily routine. It thus deserves our attention to develop an effective clothing matching scheme to help people figure out the suitable match for a given item and make a harmonious outfit.

    Composing Text and Image for Image Retrieval

    Image retrieval refers to retrieving images that meet the user’s search intent. Traditional image retrieval systems only allow users to use either the text or image query to express their search intent. However, in many cases, it is intractable for users to describe their search intent via a single textual query, meanwhile it is also difficult for users to find the ideal images to exactly convey their intent. Consequently, to allow users to flexibly express their search intent, composing text and image for image retrieval (CTI-IR) is recently proposed and gaining increasing research attention.

    User Profiling across Multiple Social Networks

    User profiling, which aims to infer users' unobservable information based on observable information such as individual's behavior or utterances, is the basis for many applications, such as personalized recommendation, and expert finding. Traditional user profiling conducted with traditional medium, such as document records, is always hindered by the limited data sources. Recent years, the proliferation of social media has opened new opportunities for user profiling. Moreover, as different social networks provide different services, increasing number of people are involved in multiple social networks. Different aspects can be revealed by different social networks. Therefore, to comprehensively learn users' profiles, it is time to shift from a single social network to multiple social networks.