Top Subdomains: A Solution to a Common Challenge

April 04, 2024

Originally posted on CheckPhish Community

Introduction

In the beginning, when creating my web scanner scanner.ducks.party, it was a big surprise and a problem for me that there were no actual top subdomains on the Internet. I wanted to scan also subdomains of malicious sites, but using such utilities as OWASP amass turned out to be quite resource-intensive. So the solution came quite quickly - bruteforce popular subdomains to find websites.

Brute-Forcing Popular Subdomains

I'll say up front that later I gave up on this idea because I had collected the necessary amount of data, and analyzing subdomains turned out to be an inefficient, but quite interesting practice. But that's not what I'm talking about now.

Keeping Data Open and Accessible

As an administrator of a web scanner, which is my hobby and does not bring any profit, I think it is important to keep the data as open as possible, as well as to solve the problems I encountered on my way. Top subdomains is one of those problems and now I want to solve it.

Sharing the Results with Checkphish Community

I believe that the Checkphish community is an ideal platform to share my work results. Specifically, I will be sharing the raw data of all subdomains encountered by my bot on a monthly basis. This data will range from the most popular subdomains to those that are only encountered once. The data can be found at https://ducks.party/subdomains.