Somebody had the great idea that we could automatically generate teams for our hockey club using age, arbitrary ranking and maybe a few other things.

I knew that if we had the fun little tool, I didn’t want to be manually pulling data from the a website every week. I thought we should also automatically get the data from the source, where we already have the list of players attending.

We use a fantastic tool called BenchApp–which is even better when people use it–to manage our schedule and who is checking in or out. I figured we could use their API and just pull in all the data we needed and automate it. We have a PRO license after all.

Fortunately, I didn’t have to wait until Monday to be disappointed. Disappointed that I didn’t have the opportunity earlier to have to figure out a clever solution to getting all that sweet, barely-available data!

Getting the data

It was a bit tedious. The page loads with javascript after the initial GET, so I couldn’t just cURL the page. I used a headless browser (Playwright) to capture the page after the content loaded. Once I figured out, from the top level, the season and team IDs, it was easy to iterate through and visit the pages (they are the same for every season and team) and save them for later.

Extracting the data

When later came, oh boy, this was that opportunity that I was definitely not disappointed with. I really thought it was going to be terrible, but Beautiful Soup made it easy. The only real challenge was finding the data in the HTML programmatically.

This script took a while to visit every game page from every season for every team. With having to wait for javascript to load and having to visit pages to change sessions variables, I think it took a little over an hour to capture everything I needed.

Exposing the data

Sharing all the data through Tableau was a no-brainer. It provides a simple interface to navigate the data and check out the automatically generated potential team lineups. See below. The names have been scrambled to protect the innocents.

I can’t wait to replace all this browser sripting with API calls. The BenchApp API has been “in development” for a little over two years–soon! They do a great job over there and maybe they’ll need some, uh, ahem, help.

I started working with Python in earnest last month, so this was a chance to really dive in and learn something new. 10/10 would use again.