mic_none

Talk:Neural network (machine learning)/Archives/2024/September Source: en.wikipedia.org/wiki/Talk:Neural_network_(machine_learning)/Archives/2024/September


Early work

Today's deep neural networks are based on early work in statistics over 200 years ago. The simplest kind of feedforward neural network (FNN) is a linear network, which consists of a single layer of output nodes with linear activation functions; the inputs are fed directly to the outputs via a series of weights. The sum of the products of the weights and the inputs is calculated at each node. The mean squared errors between these calculated outputs and the given target values are minimized by creating an adjustment to the weights. This technique has been known for over two centuries as the method of least squares or linear regression. It was used as a means of finding a good rough linear fit to a set of points by Legendre (1805) and Gauss (1795) for the prediction of planetary movement.[1][2][3][4][5]

In 1958, psychologist Frank Rosenblatt described the perceptron, one of the first implemented artificial neural networks,[6][7][8][9] funded by the United States Office of Naval Research.[10] R. D. Joseph (1960)[11] mentions an even earlier perceptron-like device by Farley and Clark[4]: "Farley and Clark of MIT Lincoln Laboratory actually preceded Rosenblatt in the development of a perceptron-like device." However, "they dropped the subject." Farley and Clark[12] (1954) also used computational machines to simulate a Hebbian network. Other neural network computational machines were created by Rochester, Holland, Habit and Duda (1956).[13] The perceptron raised public excitement for research in Artificial Neural Networks, causing the US government to drastically increase funding. This contributed to "the Golden Age of AI" fueled by the optimistic claims made by computer scientists regarding the ability of perceptrons to emulate human intelligence.[14] The first perceptrons did not have adaptive hidden units. However, Joseph (1960)[11] also discussed multilayer perceptrons with an adaptive hidden layer. Rosenblatt (1962)[15]: section 16  cited and adopted these ideas, also crediting work by H. D. Block and B. W. Knight. Unfortunately, these early efforts did not lead to a working learning algorithm for hidden units, i.e., deep learning.

References

  1. ^ Mansfield Merriman, "A List of Writings Relating to the Method of Least Squares"
  2. ^ Stigler, Stephen M. (1981). "Gauss and the Invention of Least Squares". Ann. Stat. 9 (3): 465–474. doi:10.1214/aos/1176345451.
  3. ^ Bretscher, Otto (1995). Linear Algebra With Applications (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
  4. ^ a b Cite error: The named reference DLhistory was invoked but never defined (see the help page).
  5. ^ Stigler, Stephen M. (1986). The History of Statistics: The Measurement of Uncertainty before 1900. Cambridge: Harvard. ISBN 0-674-40340-1.
  6. ^ Haykin (2008) Neural Networks and Learning Machines, 3rd edition
  7. ^ Rosenblatt, F. (1958). "The Perceptron: A Probabilistic Model For Information Storage And Organization in the Brain". Psychological Review. 65 (6): 386–408. CiteSeerX 10.1.1.588.3775. doi:10.1037/h0042519. PMID 13602029. S2CID 12781225.
  8. ^ Werbos, P.J. (1975). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences.
  9. ^ Rosenblatt, Frank (1957). "The Perceptron—a perceiving and recognizing automaton". Report 85-460-1. Cornell Aeronautical Laboratory.
  10. ^ Olazaran, Mikel (1996). "A Sociological Study of the Official History of the Perceptrons Controversy". Social Studies of Science. 26 (3): 611–659. doi:10.1177/030631296026003005. JSTOR 285702. S2CID 16786738.
  11. ^ a b Joseph, R. D. (1960). Contributions to Perceptron Theory, Cornell Aeronautical Laboratory Report No. VG-11 96--G-7, Buffalo.
  12. ^ Farley, B.G.; W.A. Clark (1954). "Simulation of Self-Organizing Systems by Digital Computer". IRE Transactions on Information Theory. 4 (4): 76–84. doi:10.1109/TIT.1954.1057468.
  13. ^ Rochester, N.; J.H. Holland; L.H. Habit; W.L. Duda (1956). "Tests on a cell assembly theory of the action of the brain, using a large digital computer". IRE Transactions on Information Theory. 2 (3): 80–93. doi:10.1109/TIT.1956.1056810.
  14. ^ Russel, Stuart; Norvig, Peter (2010). Artificial Intelligence A Modern Approach (PDF) (3rd ed.). United States of America: Pearson Education. pp. 16–28. ISBN 978-0-13-604259-4.
  15. ^ Rosenblatt, Frank (1962). Principles of Neurodynamics. Spartan, New York.