Short Note
Dr. Kornai and I met each other at a human language technologies conference in the United States of America in the early 2000s. Shortly after we met, Andras Kornai mentioned to me that the founders of LinkedIn, whom he knew, might be interested in using technology from the Semantic Hacking project, on which I had worked [2,3], to detect deceptive profile postings. Although I eventually contacted one of the LinkedIn founders, nothing came of this suggestion.
Andras Kornai and I worked together on some proposals, but our only funded work together was the Metalearning project [4,5,6]. This project was inspired by the earlier European Union METAL project [1]. The question was if, for a given set of data, the appropriate machine learning algorithm, or parameter settings, could be determined scientifically, rather than in an ad hoc manner.
The principal investigator for the Metalearning project was Professor Eugene Santos of Dartmouth College. Although, as stated above, this project had the goal of determining the appropriate machine learning algorithm, or parameter settings, to be used, based on the data set being analyzed, the results for this short project, which lasted less than one year, were more modest. Andras Kornai, based on results from this project, proposed that a standard (meta)data set be created [4], on which competing systems could be trained and tested. Santos et al. [6] described this project in terms of the creation of a flexible, generic metalearning process that supports algorithm selection based on studying the algorithms’ past performance behaviors. Poulin et al. [5] reflected on this project in terms of its original, but, unmet goals.
Since the time of the Metalearning project, deep learning has become a widely used technique in machine learning. While deep learning undoubtedly has many suitable applications, arguably the machine learning community is still not adequately addressing the question posed by both the METAL and metalearning projects.
References
1. Brazdil, Pavel; Carrier, Christophe Giraud; Soares, Carlos; Vilalta, Ricardo, Metalearning: Applications to Data Mining Berlin: Springer, 2008.
2. Cybenko, G.; Giani, A.; Thompson, P. “Cognitive Hacking: A for the Mind” IEEE Computer August 2002, vol. 35, no. 8, p. 50-56.
3. Cybenko, G.; Giani, A.; Thompson, P. “Cognitive Hacking” In Zelkowitz, Marvin (ed.) Advances in Computers, vol. 60, 2004, p. 36-75.
4. Kornai, Andras “Supervised metalearning” unpublished draft.
5. Poulin, Chris; Thompson, Paul; and Bryan, Craig. “Public Health Surveillance: Predictive Analytics and Big Data” in Luxton, David D. (ed.) Artificial Intelligence and Mental Health Care, Oxford: Academic Press, 2015, p. 205 – 230.
6. Santos, Eugene, Jr.; Kirkpatrick, Alex; Nguyen, Hien; Gu, Qi; Grooms, Andy; and Poulin, Chris. “Flexible Algorithm Selection Framework for Large Scale Metalearning” 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology 2012.