Browse Posts

Themes from AI Con 2024: data moats, stealthy AI use, Chatty’s UX revolution, and enduring fundamentals.

Building AI is hard and expensive. For most companies, the path to AI success is building with third-party AI interns and cheap AI cogs.

On moving away from weekly blog posts in favour of deeper inconsistent articles and LinkedIn engagement.

Podcast chat on the reality of Data & AI and my consulting focus: Helping climate & nature tech startups ship data-intensive solutions.

Highlights and lessons from my solo expertise biz, including value pricing, fractional cash flow, and distractions from admin & politics.

My views on separating AI hype and bullshit from the real deal. The general ideas apply to past and future hype waves in tech.

Exploring why universal advice on startup data stacks is challenging, and the importance of context-specific decisions in data infrastructure.

Podcast chat on my career journey from software engineering to data science and independent consulting.

Reflections on building a solo expertise business in Data & AI, focusing on climate tech startups. Lessons learned from the first year of transition.

The real world of AI/ML doesn’t fit into a neat diagram, so I created another diagram and a maturity heatmap to model the mess.

Video and key points from the second part of a webinar on a startup’s first data hire, covering tips for defining the role and running the process.

Video and key points from the first part of a webinar on a startup’s first data hire, covering data & AI definitions and high-level recommendations.

Asking identical questions about my MagicGrantMaker idea yielded near-identical responses from the top chatbot models.

Questions to assess the security posture of a startup, focusing on basic hygiene and handling of sensitive data.

Questions to assess the quality of tech stacks and lifecycles, with a focus on artificial intelligence, machine learning, and analytics.

Since we’re far from a utopia where data issues are fully handled by AI, this post presents six questions humans can use to assess data projects.

Expanding on the startup health check question of tracking Kukuyeva’s five business aspects as wide events.

Questions that prospective data specialists and engineers should ask about development processes before accepting a startup role.

Three essential questions to understand where an organisation stands when it comes to Data & AI (with zero hype).

Eight questions that prospective data-to-AI employees should ask about a startup’s work and data culture.

Ten questions that prospective employees should ask about a startup’s team, especially for data-centric roles.

Fourteen questions that prospective employees should ask about a startup’s business model and product, especially for data-focused roles.

Reflections on what it takes to package expertise and deliver timely, actionable advice outside the context of employee relationships.

Reviewing the areas that should be assessed to determine a startup’s opportunities and challenges on the data/AI/ML front.

It’s easy to prototype with AI, but production-grade AI apps require even more thorough testing and observability than traditional software.

The story of how I joined Work on Climate as a volunteer and became its data tech lead, with lessons applied to consulting & fractional work.

Discussing the use of AI to automate underwater marine surveys as an example of the uneven distribution of technological advancement.

Discussing considerations that arise when attempting to automate the extraction of structured data from PDFs and similar documents.

Classifying startups as ML-centric or non-ML is a helpful exercise to uncover the data challenges they’re likely to face.

Two stories of getting AI functionality to production, which demonstrate the risks inherent in custom development versus starting with a no-code approach.

First post in a series on building a minimum viable data stack for startups, introducing key definitions, components, and considerations.

Getting ChatGPT Plus to elaborate on possible book content and produce a PDF cheatsheet, with the goal of learning about its capabilities.

Advice for hiring a startup’s first data person: match skills to business needs, consider contractors, and get help from data people.

Shifting focus to ‘Data & AI for Impact’, with more startup-related content, increased posting frequency, and deeper audience engagement.

Summarising the work Uri Seroussi and I did to improve Reef Life Survey’s Reef Species of the World app.

Video and summary of a talk I gave at DataEngBytes Brisbane on what I learned from doing data engineering as part of every data science role I had.

Reflections on publishing on this website: Writing publicly to share thoughts and documentation beats chasing views and likes.

Yes, data science projects have suffered from classic software engineering mistakes, but the field is maturing with the rise of new engineering roles.

Exploring the hackability of speed-based coding tests, using CodeSignal’s Industry Coding Framework as a case study.

Bing Chat recently quipped that humans are small language models. Here are some of my thoughts on how we small language models can remain relevant (for now).

My perspective after a week of using ChatGPT: This is a step change in finding distilled information, and it’s only the beginning.

Reviewing the first three chapters of the book Causal Machine Learning by Robert Osazuwa Ness.

Discussing my recent career move into climate tech as a way of doing more to help mitigate dangerous climate change.

Lessons learned building a fish ID web app with fast.ai and Streamlit, in an attempt to reduce my fear of missing out on the latest deep learning developments.

Epidemiologists analyse clinical trials to estimate the intention-to-treat and per-protocol effects. This post applies their strategies to online experiments.

Overview of a talk I gave at a deep learning course, focusing on AI ethics as the need for humans to think on the context and consequences of applying AI.

My reasons for switching from WordPress.com to Hugo on GitHub + Cloudflare, along with a summary of the solution components and migration process.

Back-dated meta-post that gathers my posts on Automattic blogs into a summary of the work I’ve done with the company.

Sharing remote teamwork insights, my climate & sustainability activism, Reef Life Survey publications, and progress on Automattic’s Experimentation Platform.

Going deeper into correct testing of different methods for bootstrap estimation of confidence intervals.

Being a data scientist can sometimes feel like a race against software commodities that replace interesting work. What can one do to remain relevant?

Video of a talk I gave on remote data science work at the Data Science Sydney meetup.

Video and summary of a talk I gave at YOW! Data on bootstrap estimation of confidence intervals.

Bootstrap sampling has been promoted as an easy way of modelling uncertainty to hackers without much statistical knowledge. But things aren’t that simple.

Causal Inference by Miguel Hernán and Jamie Robins is a must-read for anyone interested in the area.

Discussing the pluses and minuses of remote work eighteen months after joining Automattic as a data scientist.

Updating my definition of data science to match changes in the field. It is now broader than before, but its ultimate goal is still to support decisions.

Frequently asked questions by visitors to this site, especially around entering the data science field.

Call for BCRecommender maintainers followed by a decision to shut it down, as I don’t have enough time and Bandcamp now offers recommendations.

I wanted a well-paid data science-y remote job with an established company that offers a good life balance and makes products I care about. I got it eventually.

Web tools I built to visualise Reef Life Survey data and assist citizen scientists in underwater visual census work.

There’s a lot of misleading content on the estimation of customer lifetime value. Here’s what I learned about doing it well.

Video and summary of a talk I gave at the Data Science Sydney meetup, about going beyond the what & how of predictive modelling.

Seven common mistakes to avoid when working with data, such as ignoring uncertainty and confusing observed and unobserved quantities.

It seems like anyone who touches data can call themselves a data scientist, which makes the title useless. The work they do can still be useful, though.

A web tool I built to interpret A/B test results in a Bayesian way, including prior specification, visualisations, and decision rules.

Discussing the need for untested assumptions and temporality in causal inference. Mostly based on Samantha Kleinberg’s Causality, Probability, and Time.

Is artificial/machine intelligence a future threat? I argue that it’s already here, with greedy robots already dominating our lives.

Causality is often overlooked but is of much higher relevance to most data scientists than deep learning.

Insights on data collection and machine learning from spending a month sailing, diving, and counting fish with Reef Life Survey.

Some companies present raw data or information as “insights”. This post surveys some examples, and discusses how they can be turned into real insights.

Defining feasible problems and coming up with reasonable ways of measuring solutions is harder than building accurate models or obtaining clean data.

Migrating BCRecommender from MongoDB to Elasticsearch made it possible to offer a richer search experience to users at a similar cost, among other benefits.

Nutritionism is a special case of misinterpretation and miscommunication of scientific results – something many data scientists encounter in their work.

Giving an overview of the field and common paradigms, and debunking five common myths about recommender systems.

Hiring data scientists prematurely is wasteful and frustrating. Here are some questions to ask before you hire your first data scientist.

Migrating my web apps away from Parse.com due to reliability issues. Self-hosting is a better solution.

Progress on my album cover classification project, highlighting lessons that would be useful to others who are getting started with deep learning.

To become proficient at solving data science problems, you need to get your hands dirty. Here, I used album cover classification to learn about deep learning.

I became a data scientist by doing a PhD, but the same steps can be followed without a formal education program.

Recent choices I’ve made to reduce my exposure to fossil fuels, including practical steps that can be taken by Australians and generally applicable lessons.

An overview of my PhD in data science / artificial intelligence. Thesis title: Text Mining and Rating Prediction with Topical User Models.

Progress since leaving my last full-time job and setting on an independent path that includes data science consulting and work on my own projects.

My team’s solution to the Yandex Search Personalisation competition (finished 9th out of 194 teams).

Insights on search personalisation and SEO from participating in a Kaggle competition (finished 9th out of 194 teams).

A script for importing data into the Parse backend-as-a-service.

Exploring an approach to choosing the optimal number of iterations in stochastic gradient boosting, following a bug I found in scikit-learn.

Increasing SEO traffic to BCRecommender by adding content and opening up more pages for crawling. It turns out that thin content is better than no content.

Summary of a Kaggle competition to forecast bulldozer sale price, where I finished 9th out of 476 teams.

Update on BCRecommender traction using three channels: blogger outreach, search engine optimisation, and content marketing.

Data science has been a hot term in the past few years. Still, there isn’t a single definition of the field. This post discusses my favourite definition.

Summary of my approach to the Greek Media Monitoring Kaggle competition, where I finished 6th out of 120 teams.

Ranking 19 channels with the goal of getting traction for BCRecommender.

The recommendation backend for my BCRecommender service for personalised Bandcamp music discovery.

Iterating on my BCRecommender service with the goal of keeping costs low while providing a valuable music recommendation service.

My motivation behind building BCRecommender, a free recommendation & discovery service for Bandcamp music.

Summary of a talk I gave at the Data Science Sydney meetup with ten tips on almost-winning Kaggle competitions.

Discussing the hierarchy of needs proposed by Jay Kreps. Key takeaway: Data-driven algorithms & insights can only be as good as the underlying data.

Pointers to all my Kaggle advice posts and competition summaries.

First post! An email I sent to members of the Data Science Sydney Meetup with tips on how to get started with Kaggle competitions.