I had a dream, but bugs were in the way
How an amateur ML enthusiast succeeded after 1 year of pain
We all had that dream project. It is seemingly far away from reality. It is seemingly impossible to build at that time. But you just want to make it happen.
This is a story about a person with a dream, the obstacles she faced, the struggles she had to overcome, and the life lessons she learned from the entire experience.
The writeathon that inspired this article
When Hashnode first announced their Debugging February Writeathon, I was hyped because I wanted to read everyone's debugging stories and how they overcome their frustrations and struggles. At the same time, I didn't think I could submit anything because I dug and dug into my deepest memory, but I couldn't think of anything.
So where can I find an interesting debugging story?
I searched my journal.
"Yes, this is why you log and keep a progress journal." I thought to myself, as I read tons of journal entries from my amateur days. Among the nostalgic entries, 1 particular debugging adventure stood out. To describe how I was feeling at that time in one word: Desperation 😂
I could feel my past self's desperation oozing out of the pages. It was hilarious and reminiscing. So allow me to share this embarrassing but hopefully, enjoyable debugging story with you.
Part 1: My Dream Project
For those of you who have read my blogs for a while, you probably know that I am a huge Pokemon fan. I've been playing Pokemon since I was 10 and all the projects I built when I was young were Pokemon-related. Like this flash game below:
This project was one of the first I built. Read more about it in My Journey into Tech article here.
So a few years ago, I had a sudden inspiration. I wanted to make a Pokedex!
What’s a Pokedex?
For non-Pokemon fans, a Pokedex is an instrument that when pointed toward a Pokemon, can immediately identify what Pokemon it is.
In developer language, it is a custom object detection app.
Part 2: Research and Pain
After deciding I wanted to build a custom object detection app, the next step to take was research because I have absolutely no clue and background in machine learning and how to build a custom ML model to detect Pokemon.
As an ML enthusiast, I have read a few articles here and there about Tensorflow and YOLO. So I spent the entire day researching deeper into those. It got frustrating and painful at times because I do not understand how machine learning works and most tutorials and articles online assumed the reader to have some background knowledge.
I was lost, frustrated and I didn't know if I could ever find an efficient solution.
By the 2nd day of non-stop researching, I realized that I'm too much of an amateur (or should I say fetus) to really get into Tensorflow and build a custom object detection model. I decided to go with YOLOv3 and darkNet.
I collected a bunch of my favourite Pokemon's pictures (Treecko) to create my first test model.
Part 3: When All Seems Smooth-Sailing
After hours and hours of research, I can finally start training my model and I remembered waking up at 5am that day to begin.
As an amateur, it took me too long to set up Tensorflow on my own laptop... So I used Google Colab because it can run a free GPU on the cloud which means I can train my model on the cloud.
I labelled the images I collected the day before as a custom class 'Treecko'. One by one... It was tedious…
SpongeBob: A FEW HOURS LATER.
Every image is labelled with its corresponding .txt file.
And the training begins! Everything was happening so smoothly, I thought I was slowly reaching my dream.
And voila! The custom object detector was able to detect Treecko! At this point, I was internally screaming. Hours of research, barely understanding anything about machine learning technology, following hours of random tutorials, learning everything on the go...
All the blood, sweat and tears have paid off! ... Or so I thought.
Part 4: It was just the beginning
After enduring hours of headaches, I finally had a working custom object model! All that was left was to incorporate it into a mobile app and turn my phone into a real-life Pokedex (or just Treecko-dex since I only trained one model).
Sounds easy enough right? Wrong.
I started the day again at dawn and formulated the things I needed to do:
Convert the YOLOv3 weights to .pb format (aka Tensorflow)
Convert to .tflite (Tensorflow lite)
Bring .tflite model over to Android Studio (for the mobile app)
Build a working live camera Treecko detection model app
I will skip the boring parts of the story but basically, remember that:
I have no knowledge of machine learning and everything I was doing was from StackOverflow and random articles I found, and I had minimal comprehension due to my lack of knowledge.
Step 1 - Convert to .pb format: done with no issues ✔️
Step 2 - Convert from .pb to .tflite
Gave me…
Error: ‘tflite_convert’ is not recognized as an internal or external command
So okay, I thought. Let’s add that to the PATH. And…
typeError: init*() got an unexpected keyword argument ‘serialized_options’*
In the next few hours, I pip install, upgrade, uninstall and reinstall everything that could be an error based on the answers I got from StackOverflow.
Now looking back, it was dependency hell. There were many conflicting dependencies and versioning such as Protobuf, tensorflow, tensorboard, numpy, tf-nightly, and more! And as a person who was new to all this at that time, all I get were errors after errors trying to fix the problem…
After several hours, the errors were gone thanks to StackOverflow but I didn't really understand why.
Step 3 - Bring .tflite model over to Android Studio
MORE errors.
I hypothesized that my output shape is a 3d array which is not compatible with Android-compatible models, which I analyzed are always 2d arrays, after hours of cloning TFlite model repos.
So I thought: Could it be the model itself? So I created a simple program on Google Colab to check my .tflite model's shape with python. As I feared, the results looked like this.
So now my next question was: Did it not convert properly?
I researched even more. I researched so much that I felt like a mad scientist. I found people having similar issues on GitHub but no one had the solution or the explanation…
With that, I was at a breaking point. Is this project not achievable with my zero to minimal knowledge of machine learning? What can I do at this point if the internet has no answers for me?
I was desperate because I was so close to completing this project. This dream project. But in the end, there was nothing. Nothing left I could do. Despite my lack of understanding and knowledge, I did surprise myself for getting this far. So I patted myself on the back and hung up this project.
Part 5: A Year Later...
After shelving this project, I continued building more side projects and sharpening my web development skills. The machine learning days became fleeting memories but this ML enthusiast still kept up with some news on the latest ML technology.
That's when Google announced its AutoML Vision API, which allows developers who know nothing about ML to make and export custom ML models easily! It was like someone at Google heard my prayers, I thought. (No, they probably didn't)
I got to work immediately. "This time, that dream project will be a reality!" I thought to myself.
I collected over 10,000 images for all 151 Pokemon species. I trained them for hours.
And my dream project was finally completed. You can watch the demo video if you're curious.
So, What Can We Learn?
Let's end this embarrassing story with key takeaways. There are typically 2 reasons why we get stuck when debugging:
1. Lack of Knowledge
When you do not understand the foundational concepts of the project, there's no way you will be able to debug efficiently and get to the root of the problem easily.
In my example, my lack of ML knowledge costs me time - a lot of time - to solve the bugs. Even after solving them, I still didn't understand why because I simply followed solutions from online forums.
Think back to your amateur days. You probably realized that the bugs you were stuck with such as CORS issues, state handling errors, etc. were all due to your lack of knowledge.
How to overcome this category of debugging hell:
Go back to the basics - learn what you don't know, don't take shortcuts
Don't just Google the surface error - get to the root of the error
Ask for help and guidance from someone more knowledgable
2. Lack of Tracing Skills
This happens when you do not understand the entire codebase and have difficulty tracing the error or bug.
In a team with a large codebase, code can get really complex and difficult to read. When you cannot read and understand other team members' code, you will often find that you struggle with getting to the bottom of the error.
How to overcome this category of debugging hell:
Read and re-read the codebase until you internalize it
Ask other team members for help
Thanks for reading!
And that is the story of how an amateur ML enthusiast built her dream project. The ending was quite abrupt but I hope it was still an entertaining debugging story. Feel free to laugh or cry at me in the comments. I laughed at myself countless times reminiscing this story.
If you have submitted a debugging story for the writeathon, feel free to share it in the comments too! I would love to read it! Thanks for reading! Cheers!