Teaching Myself Math and ML: Attempt #2
I haven’t made a post since July 2022. When I first started this blog I didn’t know what the point of it would be. But I got inspired by Tim Urban’s Wait But Why and decided that I would write an article in his style about a topic I somewhat understand: computers. I was shocked by the response to that post.
I got numerous emails and dm’s from people reading it, saw it get featured on other peoples blog’s and newsletters, with over 4,000 reads from 80 countries.
That set a high bar for me. I was grateful of course, but in some ways it was a bit of a curse. I haven’t touched my blog since then for fear of bringing down the average post quality. But I recently decided that it doesn’t really matter that much. So this quickly rattled off explanation of what I’m doing right now and why there are two new tabs labeled weekly logs and projects is me lowering the bar that I accidentally set too high. In the future I plan to just write about whatever I find interesting, and I have a few ideas already!
With that context set, let's dive into what I'm actually doing now.
To explain, it would be helpful to briefly give a bit of background. In 2023 I was in Nice during Palantir’s annual winter break. I was trying to understand the trajectory of where AI was going, if I should believe the hype or if I should remain skeptical. I’ve always been interested in AI, ever since I read Tim Urban’s post in 2015 on the AI revolution, and that’s initially why I decided to study Computer Science. However, as I tried to understand what was going on and formulate my own opinion I realized that I was unable to truly understand what any tehcnical paper was saying or how the underlying technology worked. My math and Computer Science skills were not up to par, even after completing a CS degree.
Therefore I devised a plan to teach myself. I picked out a few rigorous textbooks and materials on Calculus and Linear Algebra, Machine Learning and other topics. Calculus by Spivak, Axler’s Linear Algebra, D2L.ai and Karpathy’s Youtube series. I wrote up a syllabus and made plans to study before and after work and to work on projects on the weekend.
This was a failure. Not a complete failure, but a failure. It was a failure in the sense that I never completed those textbooks and I wasn’t able to keep up with the grueling schedule I had set for myself while working a full time job. I was also working on grad school applications for the last quarter of 2024 which took up a lot of my time.
This year I’m trying again, building on my learnings from last year. My syllabus last year had three main issues:
- I had set very specific target dates, ex. finish these 3 chapters by March 1st.
- The materials were overly theoretical.
- I wasn’t solving enough problems / getting enough deliberate practice in.
Building off that, I’ve written a new syllabus for myself this year. One without any specific targets for where I should be, but rather with targets on how long I spend each week. I’m also emphasizing working through problems / projects this year as I feel like that’s where I learn the most.
To construct my new syllabus, I’ve taken heavy inspiration and advice from Justin Skycak, in particular, his post here. I’ve made a few modifications, but I am generally following it.
I have a three pronged approach to my learning. The first is math, which I am teaching myself through Math Academy which I highly recommend. I have a few issues with it that I outlined in this comment, but I am generally very happy with it. For ML learning, the second prong, I am doing this in parallel and following Justin’s recommendations of doing Andrew Ng’s course, followed by reading Deep Learning by Simon J. D. Prince. Finally, I’m giving myself time to work on whatever personal project I want to. I do believe in the power of serendipity and following your curiosity so I’m not setting any strict requirements on what this should be. Right now this happens to be on an indie video game. In the future that may change.
As part of this plan, to keep up the accountability and in a very tiny way, increase my luck surface area I’m logging my weekly progress and logging my projects.
So that’s an explanation of what I’m doing. In the future I may write more posts with updates of how this is going, and I may write random short posts on whatever topic happens to interest me!