← Projects

Client Prospector

LaunchedDeprecated

A Python and Tornado pipeline that turned raw WHOIS exports into a qualified call list — thousands of concurrent checks for which small businesses had a website that needed help.

pythontornadoasyncdata pipelinebashconsulting

Consulting work for a software company. The problem they brought me is the one every small services business has: finding the customers.

The insight was that the answer was already public. Every registered domain has a WHOIS record with a registrant, an address, and dates. Every website either does or does not run WordPress, and either does or does not work on a phone. Cross those facts and you have a list of local businesses with a site that needs work and a named human to talk to about it.

WHOIS exportsnoisy proxyScrub and mergerecursive, one datasetFilter by zipa territory, not the worldEnrichthousands in flightProspect listgrouped by registrar, with a name to call
Public facts in, a call list out.

It started as Bash. A recursive scrubber walked a directory tree of WHOIS exports, stripped malformed trailing characters, dropped the columns that carried nothing useful, and concatenated everything into one dataset — the kind of job sed is genuinely good at, until it isn't.

The real tool was the Python rewrite, a command-line program with a handful of verbs:

  • Count the records in a dataset, because knowing the size of the thing is step zero.
  • Filter by zip code range, which is how a national dataset becomes a territory you could actually drive to.
  • Check for WordPress — the enrichment that made the whole thing worth building. It requests /wp-admin on every domain and reads the status, across tens of thousands of hosts, using Tornado's async HTTP client over a curl backend. Done serially at even a second per host, a large dataset is a week of waiting; done with thousands of requests in flight, it is a coffee break. This is where I learned concurrency as a practical discipline rather than an exam question.
  • Check mobile readiness against Google's PageSpeed Insights API, so a site could be qualified as needing work before anyone picked up a phone.
  • Group by registrar or administrative organization, and pull every record for one — because discovering that a single web shop administers three hundred local domains changes who you should be calling.

I put a header on every script — Created by Aaron Trank · For use by Jonesen LLC only — which sums the arrangement up. I wrote the tooling; they ran the business on it.