Recently, the StackOverflow's 2023 Developer Survey revealed that written mediums, such as blogs and written tutorials, are among the top 5 preferred methods for developers to learn. In other words, this suggests that Hashnode could become the go-to platform for developers' learning needs.

As a Hashnode user for over 3 years, I can confidently say that Hashnode is the best blogging platform and community for developers. When I learned about the Hashnode APIs Hackathon, I thought this would be an excellent opportunity to showcase my idea of building Hashnode Academy - a learning hub for developers.

The Idea

The idea for Hashnode Academy came about when my friend and I talked about how there are so many technical articles being published everyday on Hashnode now. I am very proud to see how far Hashnode has come, as I've seen this platform in its early days when there are only several active bloggers putting out articles at the time.

Unfortunately, it has now become almost impossible to keep up with the sheer amount of articles. I find myself often questioning:

• How can I find articles on Hashnode and learn XXX at the level I'm at? Not every article has the difficulty/experience level tag.

• How can I find a structured guided list of high quality articles to learn XXX if there are no series on this?

• Where can I discover and read from hidden/upcoming blogs? There are too many blogs and too little time to find a new awesome blogger.

• How can I help new bloggers gain more exposure and showcase their expertise on a topic?

After a few days of incubating this idea, I came up with Hashnode Academy: an AI-powered learning path generator app. I envisioned it to be a hub where:

1. Users can prompt what they wish to learn and their current level (i.e. beginner, novice, intermediate, advanced, expert)

2. The app will then fetch articles via Hashnode APIs

3. An optimized learning path of article recommendations will be generated, arranging from user's current difficulty level to expert

4. Users can regenerate learning paths by tweaking filters until they are satisfied

This app aims to solve the questions that were addressed earlier. Not only will Hashnode Academy helps us in finding high quality articles that can guide us on our learning journeys, it will also be a way to discover new bloggers in a rather populated community. In return, writers will be empowered to write higher quality articles and users get to enjoy diversity and a structured reading experience.

The Challenge Begins

Jan 28. Immediately after deciding on the idea, my brain went into 'planning' mode. I knew I didn't have much time. It was only me and a friend.

This is a simplified diagram on what was in my head at the time. As Thomas Edison once said, "Everything is 1% inspiration and 99% perspiration". Time to get sweaty!

1) Research

The biggest challenge in our research was figuring how to teach our ML model what makes one article more advanced than another. Even with human brains, it is not always easy to tell whether article A's content on a certain topic should be learned before or after reading article B.

After consulting several research papers, we came up with a list of features we want to include in our model to determine an article's difficulty and complexity.

Another issue we foresaw was our own biases and subjectivity with perceived difficulty. What we think is "beginner" level for ourselves may be considered "intermediate" to others. How do we have that standardization?

Therefore, we came up with a simple chart:

2) Data Preparation

For the sake of the timeline of this hackathon (and our mental health), we decided to train the model on a small dataset. In total, we used about 1200 articles that we got from various tech blogs.

We scrape the content of these articles and classified each of them according to our perceived difficulty level. Here comes another challenge. Let's say below is the structure of our data for the classification model.

As you can see from some article titles, labelling the Difficulty Level can be pretty straightforward. Of course, we also read the content to make sure that the "Demystifying GraphQL" article is indeed for beginners. However, we slowly realized a problem.

Some articles may need prerequisites from a different topic. For example, the "GraphQL with Flutter" requires novice knowledge of GraphQL but beginner knowledge of Flutter. How should we classify such articles? For now, we decided to omit such articles from our training dataset.

Also, some articles we found are analytical or persuasive in its content and are not tutorial articles. We decided these types of articles are irrelevant for the model.

3) Data Modeling

With time ticking, we decided to include 2 features for the model to predict: N-grams and TF-IDF. We use the SciKit Learn library to train the model. Here's an overarching view on the model creation process.

What we got is a model with an average of 89% accuracy in predicting difficulty levels. We noted that it might due to the small amount of data we used and many other factors that may affect these results. If we had more time, we would aim to increase the accuracy and include the rest of the features we had planned to include.

4) Export Model (and build the app)

After more tweaking and optimizations, we exported the model for use. Now, time to build Hashnode Academy. Here's a high-level diagram on how it works.

Fetching articles from Hashnode was easy thanks to their APIs. For this project, we used the Feed query to get articles based on the tag (i.e. topic) that the user wants to learn. For example, if the user wants to learn GraphQL, then the query would look something like:

query Feed(
  $first: Int!,
  $after: String
  $filter: FeedFilter
) {
  feed(
    first: $first,
    after: $after,
    filter: $filter
  ) {
    edges {
      node {
        title
        content {
          markdown
        }
      }
    }
    pageInfo {
      hasNextPage
      endCursor
    }
  }
}

variables = {
    "first": 50,  //Fixed to 50 as per Hashnode API limit
    "after": "",  //Start with no after cursor
    "filter": {
        "tags": ["56744723958ef13879b9555c"] //graphql tag ID
    }
}

5) Delivery

With only a day left in the hackathon, there's still the blog post and the UI of the app to be done. Fortunately, I have the experience to write an article like this in 2-3 hours, so there were hardly any hiccups in this step.

My team celebrated the completion and end of this hackathon with a mix of relief, joy and slight regrets.

Thank you for reading this rather long story of how Hashnode Academy was built. With that, I present to you this short demo video of the MVP of Hashnode Academy, your go-to Tech Learning Hub!

Technologies Used

• Next.js

• Python

• SciKit library

GitHub repo: https://github.com/victoria-lo/hashnode-academy

Next Steps

In retrospective, there were a few things we could have improved in our data preparation and data modeling step of the implementation.

For example, while preparing the data, our dataset is limited in terms of its variety in each difficulty level. It is arduous to find an equivalent number of articles in each difficulty level, and due to lack of time, we proceeded.

Nevertheless, we were proud of the fact that we achieved a decent model to predict difficulty within this short amount of time. Completing a working demo of the app was a milestone we want to highlight as well. Without our tenacity and experience in allocating our roles based on our strengths, it may be hard to finish this complex project, along with its deliverables (yep - this blog post and demo video) on time.

Our next phase for Hashnode Academy will include but not limited to:

• Training a bigger and more accurate model to predict complexity of an article

• Using higher quality articles as datasets for training

• Include parameters to train the model to look for relevancy and quality as well

• Adding filters in UI for users to tweak their learning path

• Allow users to prompt their desired learning topic in more detail

• Include more variety of difficulty levels and specifying prerequisites from other topics

• Improvements to UI/UX of landing page

Conclusion

The Hashnode API hackathon brought about lots of challenging moments for our 2-person team. Despite jumping into this hackathon late with only a week left, it was still a project I was highly passionate about and wanted to complete no matter what.

Our perseverance proved to yield satisfactory results. Yet because this is our dream project, we acknowledged that we have some regrets regarding the quality of the model and the app's features we had in our vision.

Thank you for reading! I hope you have enjoyed my recounting of this adventure as much as we have enjoyed building Hashnode Academy. Huge thanks to my team member, Steffy Lo, for being my partner in this project. Will be looking forward to seeing all the brilliant projects submitted for this hackathon! Cheers!