ABSTRACT
This paper presents an approach to combine rank aggregation techniques using a soft computing technique -- Genetic Programming -- in order to improve the results in Information Retrieval tasks. Previous work shows that by combining rank aggregation techniques in an agglomerative way, it is possible to get better results than with individual methods. However, these works either combine only a small set of lists or are performed in a completely ad-hoc way. Therefore, given a set of ranked lists and a set of rank aggregation techniques, we propose to use a supervised genetic programming approach to search combinations of them that maximize effectiveness in large search spaces. Experimental results conducted using four datasets with different properties show that our proposed approach reaches top performance in most datasets. Moreover, this cross-dataset performance is not matched by any other baseline among the many we experiment with, some being the state-of-the-art in learning-to-rank and in the supervised rank aggregation tasks. We also show that our proposed framework is very efficient, flexible, and scalable.
- M. S. Beg and N. Ahmad. Soft computing techniques for rank aggregation on the world wide web. World Wide Web, 6(1):5--22, 2003. Google ScholarDigital Library
- J. C. Borda. Mémoire sur les élections au scrutin. Histoire de l'Académie Royale des Sciences, 1781.Google Scholar
- G. V. Cormack, C. L. Clarke, and S. Buettcher. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 758--759. ACM, 2009. Google ScholarDigital Library
- R. Cummins, M. Lalmas, and C. O'Riordan. Learning aggregation functions for expert search. In ECAI, pages 535--540, 2010. Google ScholarDigital Library
- M. G. de Carvalho, A. H. F. Laender, M. A. Gonçalves, and A. S. da Silva. A genetic programming approach to record deduplication. IEEE Trans. Knowl. Data Eng., 24(3):399--412, 2012. Google ScholarDigital Library
- G. de Castro Mendes Gomes, V. C. de Oliveira, J. M. Almeida, and M. A. Gonçalves. Is learning to rank worth it? A statistical analysis of learning to rank methods in the LETOR benchmarks. JIDM, 4(1):57--66, 2013.Google Scholar
- C. Dwork, R. Kumar, M. Naor, and D. Sivakumar. Rank aggregation methods for the web. In Proceedings of the 10th international conference on World Wide Web, pages 613--622. ACM, 2001. Google ScholarDigital Library
- C. Dwork, R. Kumar, M. Naor, and D. Sivakumar. Rank aggregation methods for the web. In Proceedings of the 10th International Conference on World Wide Web, WWW '01, pages 613--622, 2001. Google ScholarDigital Library
- R. Fagin, R. Kumar, and D. Sivakumar. Efficient similarity search and classification via rank aggregation. In Proceedings of the 2003 ACM SIGMOD international conference on Management of data, pages 301--312. ACM, 2003. Google ScholarDigital Library
- W. Fan, E. A. Fox, P. Pathak, and H. Wu. The effects of fitness functions on genetic programming-based ranking discovery for web search. Journal of the American Society for Information Science and Technology, 55(7):628--636, 2004. Google ScholarDigital Library
- R. Feldt and P. Nordin. Using factorial experiments to evaluate the effect of genetic programming parameters. In Genetic programming, pages 271--282. Springer, 2000. Google ScholarDigital Library
- C. D. Ferreira, J. Santos, R. da S Torres, M. A. Gonçalves, R. C. Rezende, and W. Fan. Relevance feedback based on genetic programming for image retrieval. Pattern Recognition Letters, 32(1):27--37, 2011. Google ScholarDigital Library
- D. A. Hull, J. O. Pedersen, and H. Schütze. Method combination for document filtering. In Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, pages 279--287. ACM, 1996. Google ScholarDigital Library
- J. R. Koza. Genetic programming: on the programming of computers by means of natural selection, volume 1. MIT press, 1992. Google ScholarDigital Library
- K. W. Lam and C. H. Leung. Rank aggregation for meta-search engines. In Proceedings of the 13th international World Wide Web conference on Alternate track papers & posters, pages 384--385. ACM, 2004. Google ScholarDigital Library
- K. Li, N. Du, and A. Zhang. A link prediction based unsupervised rank aggregation algorithm for informative gene selection. In Bioinformatics and Biomedicine (BIBM), 2012 IEEE International Conference on, pages 1--6. IEEE, 2012. Google ScholarDigital Library
- Y.-T. Liu, T.-Y. Liu, T. Qin, Z.-M. Ma, and H. Li. Supervised rank aggregation. In Proceedings of the 16th International Conference on World Wide Web, WWW '07, pages 481--490, 2007. Google ScholarDigital Library
- D. C. G. Pedronette and R. d. S. Torres. Image re-ranking and rank aggregation based on similarity of ranked lists. In Computer analysis of images and patterns, pages 369--376. Springer, 2011. Google ScholarDigital Library
- D. C. G. Pedronette and R. d. S. Torres. Combining re-ranking and rank aggregation methods. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pages 170--178. Springer, 2012.Google ScholarCross Ref
- M. Pujari and R. Kanawati. Supervised rank aggregation approach for link prediction in complex networks. In Proceedings of the 21st World Wide Web Conference, WWW 2012, Lyon, France, April 16-20, 2012 (Companion Volume), pages 1189--1196, 2012. Google ScholarDigital Library
- T. Qin, T.-Y. Liu, J. Xu, and H. Li. Letor: A benchmark collection for research on learning to rank for information retrieval. Inf. Retr., 13(4), Aug. 2010. Google ScholarDigital Library
- T. Qin, T.-Y. Liu, X.-D. Zhang, Z. Chen, and W.-Y. Ma. A study of relevance propagation for web search. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '05, pages 408--415, 2005. Google ScholarDigital Library
- M. E. Renda and U. Straccia. Web metasearch: rank vs. score based rank aggregation methods. In Proceedings of the 2003 ACM symposium on Applied computing, pages 841--846. ACM, 2003. Google ScholarDigital Library
- J. Sese and S. Morishita. Rank aggregation method for biological databases. GENOME INFORMATICS SERIES, pages 506--507, 2001.Google Scholar
- J. A. Shaw, E. A. Fox, J. A. Shaw, and E. A. Fox. Combination of multiple searches. In The Second Text REtrieval Conference (TREC-2, pages 243--252, 1994.Google Scholar
- K. Subbian and P. Melville. Supervised rank aggregation for predicting influencers in twitter. In PASSAT/SocialCom 2011, Privacy, Security, Risk and Trust (PASSAT), 2011 IEEE Third International Conference on and 2011 IEEE Third International Conference on Social Computing (SocialCom), Boston, MA, USA, 9-11 Oct., 2011, pages 661--665, 2011.Google Scholar
- R. d. S. Torres, A. X. Falcão, M. A. Gonçalves, J. P. Papa, B. Zhang, W. Fan, and E. A. Fox. A genetic programming framework for content-based image retrieval. Pattern Recognition, 42(2):283--292, 2009. Google ScholarDigital Library
- M. N. Volkovs, H. Larochelle, and R. S. Zemel. Learning to rank by aggregating expert preferences. In Proceedings of the 21st ACM International Conference on Information and Knowledge Management, CIKM '12, pages 843--851, 2012. Google ScholarDigital Library
- M. N. Volkovs and R. S. Zemel. Crf framework for supervised preference aggregation. In Proceedings of the 22Nd ACM International Conference on Conference on Information and Knowledge Management, CIKM '13, pages 89--98, 2013. Google ScholarDigital Library
- S. Wu. The weighted condorcet fusion in information retrieval. Inf. Process. Manage., 49(1):108--122, 2013. Google ScholarDigital Library
- J.-Y. Yeh, J.-Y. Lin, H.-R. Ke, and W.-P. Yang. Learning to rank for information retrieval using genetic programming. In Proceedings of SIGIR 2007 Workshop on Learning to Rank for Information Retrieval (LR4IR 2007), 2007.Google Scholar
- C. Zhai and J. Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. In Proceedings of the 24th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR '01, pages 334--342, 2001. Google ScholarDigital Library
Index Terms
A Soft Computing Approach for Learning to Aggregate Rankings
Recommendations
Rank aggregation using ant colony approach for metasearch
Metasearch engines provide a plethora of information to the user through World Wide Web. They are the prominent sources of query-based search and centralized human---world interactions. Metasearch engine shows a list of Web sites to a particular query ...
Meta-search: aggregating search engine results using rank-distance
SISAP '11: Proceedings of the Fourth International Conference on SImilarity Search and APplicationsIn this demonstration, we present a meta-search engine based on rank-distance, which is a similarity measure between partial rankings. Our meta-search engine fetches results from four main search engines: Google, Yahoo, Bing and Ask.com and combines ...
Effective rank aggregation for metasearching
Nowadays, mashup services and especially metasearch engines play an increasingly important role on the Web. Most of users use them directly or indirectly to access and aggregate information from more than one data sources. Similarly to the rest of the ...
Comments