6
$\begingroup$

I've recently been exploring Reinforcement Learning (RL) methods in my work and the exploration-exploitation dilemma is always mentioned. It almost feels like the exploration-exploitation dilemma stemmed from RL research. However, when we think about Simulated Annealing (SA) and Genetic Algorithms (GA), isn't there also the notion of exploration and exploitation? Specifically, in SA, you take a random step (explore) with some probability and an improving step otherwise (exploit). In GA, you mutate (explore) with some probability and select good parents for crossover (exploit).

My question: which paper first mentioned this? It feels as though https://pubsonline.informs.org/doi/abs/10.1287/orsc.2.1.71 is when it was first mentioned but I'd like expert opinions on this.

Improve this question
$\endgroup$
6
  • $\begingroup$ Exploration vs. Exploitation is a big thing in Bayesian optimization. Bayesian optimization publications go back to the early 70s. I don't know when exploration vs. exploitation started being considered in that literature, and when it was called as such. $\endgroup$
    – Mark L. Stone
    Commented Jun 8, 2022 at 0:46
  • $\begingroup$ Well, perhaps "Efficient Global Optimization of Expensive Black-Box Functions" researchgate.net/profile/Donald-Jones-5/publication/… published in 1998 might have been the 1st publication in which the "modern" version of Bayesian Optimization was considered, which formulated trade off between exploration and exploitation, although not quite called as such. That is after your 1991 paper.. $\endgroup$
    – Mark L. Stone
    Commented Jun 8, 2022 at 1:07
  • $\begingroup$ You might also want to consider adaptive experimental design for bandit problems going back to the 70s, although you might not necessarily consider that they qualify. $\endgroup$
    – Mark L. Stone
    Commented Jun 8, 2022 at 1:07
  • $\begingroup$ An earlier paper is Axelrod, R. (1987). The evolution of strategies in the iterated Prisoner's Dilemma. In Genetic algorithms and simulated annealing - books.google.ca/… $\endgroup$
    – Rob
    Commented Jun 8, 2022 at 1:41
  • 1
    $\begingroup$ In optimization, one often uses the words "intensification" and "diversification" $\endgroup$
    – fontanf
    Commented Jun 8, 2022 at 8:47

1 Answer 1

Reset to default
1
$\begingroup$

With reference to the paper in the original post "Exploration and Exploitation in Organizational Learning" (E&E) by James G. March, the paper "The Lost Experiment in Exploration and Exploitation" by Donncha Kavanagh provides some interesting historical context preceeding the E&E paper.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.