Week 2
Hello everyone! It has been a busy week, but I've made some exciting progress on my machine learning journey. Here is what I've been up to: Kaggle Orbit Wars & AWS I completed the baseline implementation for the Kaggle Orbit Wars competition and initially hit a score of around 1030. My score has dipped slightly over the past few days, so I am currently brainstorming ways to improve it. This week also marked my very first time using AWS! I used it to extract data for reinforcement learning. Transparency check: I spent exactly $7.58 USD on AWS resources during the process. Paper Reading & RL Insights I spent a lot of time reading research papers this week. AlphaZero: I was initially excited about using the self-play mechanism from AlphaZero. However, because this specific game has rock-paper-scissors dynamics, standard self-play might not work effectively. AlphaStar: This led me to the AlphaStar paper, which uses self-play combined with League Training . The engineering behind AlphaStar is incredible. Two specific concepts really stood out to me: Pointer Networks and V-trace off-policy correction . I was also impressed by their use of an LSTM core to handle long-term memory. Next Steps Moving forward, I plan to leverage Kaggle, AWS, and GCP credits to train different components of my model. I am giving myself total freedom to experiment, imagine, and test unconventional solutions. Random life update to close out the week: I used to have long hair because I was insecure about my forehead, but I finally decided to shave it all off at home by myself. It honestly feels really weird right now, but it's a fresh start!