How I Built An AI Meal Estimation App – AI Meal Estimates in “PERT Pilot” and Announcing A New App “Carb Pilot”

As I have been working on adding additional features to PERT Pilot, the app I built (now available on the App Store for iOS!) for people like me who are living with exocrine pancreatic insufficiency, I’ve been thinking about all the things that have been challenging with managing pancreatic enzyme replacement therapy (PERT). One of those things was estimating the macronutrients – meaning grams of fat and protein and carb – in what I was eating.

I have 20+ years practice on estimating carbs, but when I was diagnosed with EPI, estimating fat and protein was challenging! I figured out methods that worked for me, but part of my PERT Pilot work has included re-thinking some of my assumptions about what is “fine” and what would be a lot better if I could improve things. And honestly, food estimation is still one of those things I wanted to improve! Not so much the accuracy (for me, after a year+ of practice I feel as though I have the hang of it), but the BURDEN of work it takes to develop those estimates. It’s a lot of work and part of the reason it feels hard to titrate PERT every single time I want to eat something.

So I thought to myself, wouldn’t it be nice if we could use AI tools to get back quick estimates of fat, protein, and carbs automatically in the app? Then we could edit them or otherwise use those estimates.

And so after getting the initial version of PERT Pilot approved and in the App Store for users to start using, I submitted another update – this time with meal estimation! It’s now been live for over a week.

Here’s how it works:

  • Give your meal a short title (which is not used by the AI but is used at a glance by us humans to see the meal in your list of saved meals).
  • Write a simple description of what you’re planning to eat. It can be short (e.g. “hot dogs”) or with a bit more detail (e.g. “two hot dogs with gluten free buns and lots of shredded cheddar cheese”). A little more detail will get you a somewhat more accurate estimates.
  • Hit submit, and then review the generated list of estimated counts. You can edit them if you think they’re not quite right, and then save them.

Here’s a preview of the feature as a video. I also asked friends for examples of what they’d serve if they had friends or family coming over to dinner – check out the meal descriptions and the counts the app generated for them. (This is exactly how I have been using the app when traveling and eating takeout or eating at someone’s house.)

Showing screenshots of PERT Pilot with the meal description input and the output of the estimated macronutrient counts for grams of fat, protein, and carb Showing more screenshots of PERT Pilot with the meal description input and the output of the estimated macronutrient counts for grams of fat, protein, and carb Showing even more screenshots of PERT Pilot with the meal description input and the output of the estimated macronutrient counts for grams of fat, protein, and carb

The original intent of this was to aid people with EPI (PEI/PI) in estimating what they’re eating so they can better match the needed enzyme dosing to it. But I realized…there’s probably a lot of other people who might like a meal estimation app, too. Particularly those of us who are using carb counts to dose insulin several times a day!

I pulled the AI meal estimation idea out into a second, separate app called Carb Pilot, which is also now available on the App Store.

Carb Pilot is designed to make carb counting easier and to save a bunch of clicks for getting an estimate for what you’re eating.

The Carb Pilot logo, which has pieces of fruit on the letters of the word "Carb". Pilot is written in italic script in purple font.

What does Carb Pilot do?

  • Like PERT Pilot, Carb Pilot has the AI meal estimation feature. You can click the button, type your meal description (and a meal title) and get back AI-generated estimates.
  • You can also use voice entry and quickly, verbally describe your meal.
  • You can also enter/save a meal manually, if you know what the counts are, or want to make your own estimates.

Carb Pilot integrates with HealthKit, so if you want, you can enable that and save any/all of your macronutrients there. HealthKit is a great tool for then porting your data to other apps where you might want to see this data along with, say, your favorite diabetes app that contains CGM/glucose data (or for any other reason/combination).

Speaking of “any/all”, Carb Pilot is designed to be different from other food tracking apps.

As a person with diabetes, historically I *just* wanted carb counts. I didn’t want to have to sift through a zillion other numbers when I just needed ONE piece of information. If that’s true for you – whether it’s carbs, protein, calories, or fat – during onboarding you can choose which of these macronutrients you want to display.

Just want to see carbs? That’s the default, and then in the saved meals you’ll ONLY see the carb info! If you change your mind, you can always change this in the Settings menu, and then the additional macronutrients will be displayed again.

Carb Pilot enables you to toggle the display of different nutrients. This shows what it looks like if only carbs are displaying or what happens if you ask the app to display all nutrients for each recorded food item.

It’s been really fun to build out Carb Pilot. Scott has been my tester for it, and interestingly, he’s turned into a super user of Carb Pilot because, in his words, “it’s so easy to use” and to generate macronutrient estimates for what he’s eating. (His use case isn’t for dosing medicine but matching what he’s eating against his energy expenditure for how much exercise/activity he’s been doing.) He’s been using it and giving me feedback and feature requests – I ended up building the voice-entry feature much more quickly than I expected because he was very interested in using it, which has been great! He also requested the ability to display meals in reverse chronological order and to be able to copy a previous meal to repeat it on another day (swipe on a meal and you can copy the description if you want to tweak and use it again, or simply repeat the meal as-is). We also discovered that it supports multiple languages as input for the AI meal estimation feature. How? Well, we were eating outside at a restaurant in Sweden and Scott copied and pasted the entree description from the menu – in Swedish – into Carb Pilot. It returned the counts for the meal, exactly as if he had entered them in English (our default language)!

I’m pointing this out because if you give Carb Pilot a try and have an idea for a feature/wish you could change the app in some way, I would LOVE for you to email me and tell me about it. I have a few other improvements I’m already planning to add but I’d love to make this as useful to as many people who would find this type of app helpful.

Why (was) there a subscription for ongoing AI use?

For both PERT Pilot and Carb Pilot, there is a cost (expense) to using the AI meal estimation. I have to pay OpenAI (which hosts the AI I’m using for the app) to use the AI for each meal estimation, and I have to host a web server to communicate between the app and the AI, which also costs a bit for every time we send a meal estimation request from the app. That’s why I decided to make Carb Pilot free to download and try. I originally played with $1.99 a month for unlimited AI meal estimations, but temporarily have turned that off to see what that does to the server load and cost, so right now it’s free to use the AI features as well.

TLDR:

– PERT Pilot has been updated to include the new meal estimation feature!

– People without EPI can use Carb Pilot for carb, protein, fat, and/or calorie tracking (of just one or any selection of those) tracking, also using the new AI meal estimation features!

You can find PERT Pilot here or Carb Pilot here on the App Store.

One thought on “How I Built An AI Meal Estimation App – AI Meal Estimates in “PERT Pilot” and Announcing A New App “Carb Pilot”

  1. I use the cronometer app for estimating my protein, fat and carb intake. It has ways to input both foods and recipes, etc. It’s good. I got the “professional” version through a dietician I had worked with.

Leave a Reply

Your email address will not be published. Required fields are marked *