Teaching Machines to Think is a series which I am intending on starting. The focus on the series is on the engineering considerations made when we train machine learning and artificial intelligence algorithms.
We can create a Naive Linear Regression Solver:
- Guess what
m
,b
- Make another guess on the incremental change for the best observed
m
andb
- Compare the proposed improved with our best
m
andb
. If it is better then we take the proposal, if it is worse then we leave the proposedm
andb
alone. - Repeat 2 and 3 till we’re happy.
In this approach it raises a few questions
- Statistic question: What is the computational cost of closed form over this approach?
- Optimization:
- RHC can we do better? What is the ideal step size
- Similated annealing? What is the penalty over time that we should have for our steps?
- Covex optimization? SGD
As starting this series I have decided to grab a new microphone; the Rode NTUSB Microphone. This microphone is quite amazing and fairly priced. It has definitely has many features over my webcam microphone. It is very easy and straight-forward to use.