Software Engineering

Podcast chat on my career journey from software engineering to data science and independent consulting.

Questions to assess the security posture of a startup, focusing on basic hygiene and handling of sensitive data.

Questions to assess the quality of tech stacks and lifecycles, with a focus on artificial intelligence, machine learning, and analytics.

Quotes from Demetrios Brinkmann on the relationship between MLOps and DevOps, with MLOps allowing for managing changes that come from data.

Questions that prospective data specialists and engineers should ask about development processes before accepting a startup role.

Reviewing the areas that should be assessed to determine a startup’s opportunities and challenges on the data/AI/ML front.

It’s easy to prototype with AI, but production-grade AI apps require even more thorough testing and observability than traditional software.

Discussing considerations that arise when attempting to automate the extraction of structured data from PDFs and similar documents.

Two stories of getting AI functionality to production, which demonstrate the risks inherent in custom development versus starting with a no-code approach.

Getting ChatGPT Plus to elaborate on possible book content and produce a PDF cheatsheet, with the goal of learning about its capabilities.

Reflecting on an interview with Jason Warner, CEO of poolside.

Summarising the work Uri Seroussi and I did to improve Reef Life Survey’s Reef Species of the World app.

For many use cases, libraries like cartopy are better than the likes of Mapbox and Google Maps.

Video and summary of a talk I gave at DataEngBytes Brisbane on what I learned from doing data engineering as part of every data science role I had.

Despite the excitement around large language models, building with machine learning remains an engineering problem with established best practices.

Yes, data science projects have suffered from classic software engineering mistakes, but the field is maturing with the rise of new engineering roles.

Exploring the hackability of speed-based coding tests, using CodeSignal’s Industry Coding Framework as a case study.

Lessons learned building a fish ID web app with fast.ai and Streamlit, in an attempt to reduce my fear of missing out on the latest deep learning developments.

Back-dated meta-post that gathers my posts on Automattic blogs into a summary of the work I’ve done with the company.

Being a data scientist can sometimes feel like a race against software commodities that replace interesting work. What can one do to remain relevant?

Video and summary of a talk I gave at YOW! Data on bootstrap estimation of confidence intervals.

Bootstrap sampling has been promoted as an easy way of modelling uncertainty to hackers without much statistical knowledge. But things aren’t that simple.

Web tools I built to visualise Reef Life Survey data and assist citizen scientists in underwater visual census work.

It seems like anyone who touches data can call themselves a data scientist, which makes the title useless. The work they do can still be useful, though.

Migrating BCRecommender from MongoDB to Elasticsearch made it possible to offer a richer search experience to users at a similar cost, among other benefits.

Giving an overview of the field and common paradigms, and debunking five common myths about recommender systems.

Migrating my web apps away from Parse.com due to reliability issues. Self-hosting is a better solution.

I became a data scientist by doing a PhD, but the same steps can be followed without a formal education program.

A script for importing data into the Parse backend-as-a-service.

Data science has been a hot term in the past few years. Still, there isn’t a single definition of the field. This post discusses my favourite definition.

Iterating on my BCRecommender service with the goal of keeping costs low while providing a valuable music recommendation service.