skip to main content
10.1145/2806416.2806478acmconferencesArticle/Chapter ViewAbstractPublication PagescikmConference Proceedingsconference-collections
research-article

A Soft Computing Approach for Learning to Aggregate Rankings

Published:17 October 2015Publication History

ABSTRACT

This paper presents an approach to combine rank aggregation techniques using a soft computing technique -- Genetic Programming -- in order to improve the results in Information Retrieval tasks. Previous work shows that by combining rank aggregation techniques in an agglomerative way, it is possible to get better results than with individual methods. However, these works either combine only a small set of lists or are performed in a completely ad-hoc way. Therefore, given a set of ranked lists and a set of rank aggregation techniques, we propose to use a supervised genetic programming approach to search combinations of them that maximize effectiveness in large search spaces. Experimental results conducted using four datasets with different properties show that our proposed approach reaches top performance in most datasets. Moreover, this cross-dataset performance is not matched by any other baseline among the many we experiment with, some being the state-of-the-art in learning-to-rank and in the supervised rank aggregation tasks. We also show that our proposed framework is very efficient, flexible, and scalable.

References

  1. M. S. Beg and N. Ahmad. Soft computing techniques for rank aggregation on the world wide web. World Wide Web, 6(1):5--22, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. J. C. Borda. Mémoire sur les élections au scrutin. Histoire de l'Académie Royale des Sciences, 1781.Google ScholarGoogle Scholar
  3. G. V. Cormack, C. L. Clarke, and S. Buettcher. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 758--759. ACM, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. R. Cummins, M. Lalmas, and C. O'Riordan. Learning aggregation functions for expert search. In ECAI, pages 535--540, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. M. G. de Carvalho, A. H. F. Laender, M. A. Gonçalves, and A. S. da Silva. A genetic programming approach to record deduplication. IEEE Trans. Knowl. Data Eng., 24(3):399--412, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. G. de Castro Mendes Gomes, V. C. de Oliveira, J. M. Almeida, and M. A. Gonçalves. Is learning to rank worth it? A statistical analysis of learning to rank methods in the LETOR benchmarks. JIDM, 4(1):57--66, 2013.Google ScholarGoogle Scholar
  7. C. Dwork, R. Kumar, M. Naor, and D. Sivakumar. Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web, pages 613--622. ACM, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. C. Dwork, R. Kumar, M. Naor, and D. Sivakumar. Rank aggregation methods for the web. In Proceedings of the 10th International Conference on World Wide Web, WWW '01, pages 613--622, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. R. Fagin, R. Kumar, and D. Sivakumar. Efficient similarity search and classification via rank aggregation. In Proceedings of the 2003 ACM SIGMOD international conference on Management of data, pages 301--312. ACM, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. W. Fan, E. A. Fox, P. Pathak, and H. Wu. The effects of fitness functions on genetic programming-based ranking discovery for web search. Journal of the American Society for Information Science and Technology, 55(7):628--636, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. R. Feldt and P. Nordin. Using factorial experiments to evaluate the effect of genetic programming parameters. In Genetic programming, pages 271--282. Springer, 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. C. D. Ferreira, J. Santos, R. da S Torres, M. A. Gonçalves, R. C. Rezende, and W. Fan. Relevance feedback based on genetic programming for image retrieval. Pattern Recognition Letters, 32(1):27--37, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. D. A. Hull, J. O. Pedersen, and H. Schütze. Method combination for document filtering. In Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, pages 279--287. ACM, 1996. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. J. R. Koza. Genetic programming: on the programming of computers by means of natural selection, volume 1. MIT press, 1992. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. K. W. Lam and C. H. Leung. Rank aggregation for meta-search engines. In Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pages 384--385. ACM, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. K. Li, N. Du, and A. Zhang. A link prediction based unsupervised rank aggregation algorithm for informative gene selection. In Bioinformatics and Biomedicine (BIBM), 2012 IEEE International Conference on, pages 1--6. IEEE, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Y.-T. Liu, T.-Y. Liu, T. Qin, Z.-M. Ma, and H. Li. Supervised rank aggregation. In Proceedings of the 16th International Conference on World Wide Web, WWW '07, pages 481--490, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. D. C. G. Pedronette and R. d. S. Torres. Image re-ranking and rank aggregation based on similarity of ranked lists. In Computer analysis of images and patterns, pages 369--376. Springer, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. D. C. G. Pedronette and R. d. S. Torres. Combining re-ranking and rank aggregation methods. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pages 170--178. Springer, 2012.Google ScholarGoogle ScholarCross RefCross Ref
  20. M. Pujari and R. Kanawati. Supervised rank aggregation approach for link prediction in complex networks. In Proceedings of the 21st World Wide Web Conference, WWW 2012, Lyon, France, April 16-20, 2012 (Companion Volume), pages 1189--1196, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. T. Qin, T.-Y. Liu, J. Xu, and H. Li. Letor: A benchmark collection for research on learning to rank for information retrieval. Inf. Retr., 13(4), Aug. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. T. Qin, T.-Y. Liu, X.-D. Zhang, Z. Chen, and W.-Y. Ma. A study of relevance propagation for web search. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '05, pages 408--415, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. M. E. Renda and U. Straccia. Web metasearch: rank vs. score based rank aggregation methods. In Proceedings of the 2003 ACM symposium on Applied computing, pages 841--846. ACM, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. J. Sese and S. Morishita. Rank aggregation method for biological databases. GENOME INFORMATICS SERIES, pages 506--507, 2001.Google ScholarGoogle Scholar
  25. J. A. Shaw, E. A. Fox, J. A. Shaw, and E. A. Fox. Combination of multiple searches. In The Second Text REtrieval Conference (TREC-2, pages 243--252, 1994.Google ScholarGoogle Scholar
  26. K. Subbian and P. Melville. Supervised rank aggregation for predicting influencers in twitter. In PASSAT/SocialCom 2011, Privacy, Security, Risk and Trust (PASSAT), 2011 IEEE Third International Conference on and 2011 IEEE Third International Conference on Social Computing (SocialCom), Boston, MA, USA, 9-11 Oct., 2011, pages 661--665, 2011.Google ScholarGoogle Scholar
  27. R. d. S. Torres, A. X. Falcão, M. A. Gonçalves, J. P. Papa, B. Zhang, W. Fan, and E. A. Fox. A genetic programming framework for content-based image retrieval. Pattern Recognition, 42(2):283--292, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. M. N. Volkovs, H. Larochelle, and R. S. Zemel. Learning to rank by aggregating expert preferences. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM '12, pages 843--851, 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. M. N. Volkovs and R. S. Zemel. Crf framework for supervised preference aggregation. In Proceedings of the 22Nd ACM International Conference on Conference on Information and Knowledge Management, CIKM '13, pages 89--98, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. S. Wu. The weighted condorcet fusion in information retrieval. Inf. Process. Manage., 49(1):108--122, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. J.-Y. Yeh, J.-Y. Lin, H.-R. Ke, and W.-P. Yang. Learning to rank for information retrieval using genetic programming. In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval (LR4IR 2007), 2007.Google ScholarGoogle Scholar
  32. C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '01, pages 334--342, 2001. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A Soft Computing Approach for Learning to Aggregate Rankings

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      CIKM '15: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management
      October 2015
      1998 pages
      ISBN:9781450337946
      DOI:10.1145/2806416

      Copyright © 2015 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 17 October 2015

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      CIKM '15 Paper Acceptance Rate165of646submissions,26%Overall Acceptance Rate1,861of8,427submissions,22%

      Upcoming Conference

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader