2023-09-26
Hebbian learning inspired estimation of the linear regression parameters from queries
Publication
Publication
Local learning rules in biological neural networks (BNNs) are commonly referred to as Hebbian learning. [26] links a biologically motivated Hebbian learning rule to a specific zeroth-order optimization method. In this work, we study a variation of this Hebbian learning rule to recover the regression vector in the linear regression model. Zeroth-order optimization methods are known to converge with suboptimal rate for large parameter dimension compared to first-order methods like gradient descent, and are therefore thought to be in general inferior. By establishing upper and lower bounds, we show, however, that such methods achieve near-optimal rates if only queries of the linear regression loss are available. Moreover, we prove that this Hebbian learning rule can achieve considerably faster rates than any non-adaptive method that selects the queries independently of the data.
| Additional Metadata | |
|---|---|
| , , , , , | |
| doi.org/10.48550/arXiv.2311.03483 | |
| Organisation | Machine Learning |
|
Schmidt-Hieber, J., & Koolen-Wijkstra, W. (2023). Hebbian learning inspired estimation of the linear regression parameters from queries. doi:10.48550/arXiv.2311.03483 |
|
