Month: May 2023

How I Built An AI Meal Estimation App – AI Meal Estimates in “PERT Pilot” and Announcing A New App “Carb Pilot”

by Dana Lewis

As I have been working on adding additional features to PERT Pilot, the app I built (now available on the App Store for iOS!) for people like me who are living with exocrine pancreatic insufficiency, I’ve been thinking about all the things that have been challenging with managing pancreatic enzyme replacement therapy (PERT). One of those things was estimating the macronutrients – meaning grams of fat and protein and carb – in what I was eating.

I have 20+ years practice on estimating carbs, but when I was diagnosed with EPI, estimating fat and protein was challenging! I figured out methods that worked for me, but part of my PERT Pilot work has included re-thinking some of my assumptions about what is “fine” and what would be a lot better if I could improve things. And honestly, food estimation is still one of those things I wanted to improve! Not so much the accuracy (for me, after a year+ of practice I feel as though I have the hang of it), but the BURDEN of work it takes to develop those estimates. It’s a lot of work and part of the reason it feels hard to titrate PERT every single time I want to eat something.

So I thought to myself, wouldn’t it be nice if we could use AI tools to get back quick estimates of fat, protein, and carbs automatically in the app? Then we could edit them or otherwise use those estimates.

And so after getting the initial version of PERT Pilot approved and in the App Store for users to start using, I submitted another update – this time with meal estimation! It’s now been live for over a week.

Here’s how it works:

  • Give your meal a short title (which is not used by the AI but is used at a glance by us humans to see the meal in your list of saved meals).
  • Write a simple description of what you’re planning to eat. It can be short (e.g. “hot dogs”) or with a bit more detail (e.g. “two hot dogs with gluten free buns and lots of shredded cheddar cheese”). A little more detail will get you a somewhat more accurate estimates.
  • Hit submit, and then review the generated list of estimated counts. You can edit them if you think they’re not quite right, and then save them.

Here’s a preview of the feature as a video. I also asked friends for examples of what they’d serve if they had friends or family coming over to dinner – check out the meal descriptions and the counts the app generated for them. (This is exactly how I have been using the app when traveling and eating takeout or eating at someone’s house.)

Showing screenshots of PERT Pilot with the meal description input and the output of the estimated macronutrient counts for grams of fat, protein, and carb Showing more screenshots of PERT Pilot with the meal description input and the output of the estimated macronutrient counts for grams of fat, protein, and carb Showing even more screenshots of PERT Pilot with the meal description input and the output of the estimated macronutrient counts for grams of fat, protein, and carb

The original intent of this was to aid people with EPI (PEI/PI) in estimating what they’re eating so they can better match the needed enzyme dosing to it. But I realized…there’s probably a lot of other people who might like a meal estimation app, too. Particularly those of us who are using carb counts to dose insulin several times a day!

I pulled the AI meal estimation idea out into a second, separate app called Carb Pilot, which is also now available on the App Store.

Carb Pilot is designed to make carb counting easier and to save a bunch of clicks for getting an estimate for what you’re eating.

The Carb Pilot logo, which has pieces of fruit on the letters of the word "Carb". Pilot is written in italic script in purple font.

What does Carb Pilot do?

  • Like PERT Pilot, Carb Pilot has the AI meal estimation feature. You can click the button, type your meal description (and a meal title) and get back AI-generated estimates.
  • You can also use voice entry and quickly, verbally describe your meal.
  • You can also enter/save a meal manually, if you know what the counts are, or want to make your own estimates.

Carb Pilot integrates with HealthKit, so if you want, you can enable that and save any/all of your macronutrients there. HealthKit is a great tool for then porting your data to other apps where you might want to see this data along with, say, your favorite diabetes app that contains CGM/glucose data (or for any other reason/combination).

Speaking of “any/all”, Carb Pilot is designed to be different from other food tracking apps.

As a person with diabetes, historically I *just* wanted carb counts. I didn’t want to have to sift through a zillion other numbers when I just needed ONE piece of information. If that’s true for you – whether it’s carbs, protein, calories, or fat – during onboarding you can choose which of these macronutrients you want to display.

Just want to see carbs? That’s the default, and then in the saved meals you’ll ONLY see the carb info! If you change your mind, you can always change this in the Settings menu, and then the additional macronutrients will be displayed again.

Carb Pilot enables you to toggle the display of different nutrients. This shows what it looks like if only carbs are displaying or what happens if you ask the app to display all nutrients for each recorded food item.

It’s been really fun to build out Carb Pilot. Scott has been my tester for it, and interestingly, he’s turned into a super user of Carb Pilot because, in his words, “it’s so easy to use” and to generate macronutrient estimates for what he’s eating. (His use case isn’t for dosing medicine but matching what he’s eating against his energy expenditure for how much exercise/activity he’s been doing.) He’s been using it and giving me feedback and feature requests – I ended up building the voice-entry feature much more quickly than I expected because he was very interested in using it, which has been great! He also requested the ability to display meals in reverse chronological order and to be able to copy a previous meal to repeat it on another day (swipe on a meal and you can copy the description if you want to tweak and use it again, or simply repeat the meal as-is). We also discovered that it supports multiple languages as input for the AI meal estimation feature. How? Well, we were eating outside at a restaurant in Sweden and Scott copied and pasted the entree description from the menu – in Swedish – into Carb Pilot. It returned the counts for the meal, exactly as if he had entered them in English (our default language)!

I’m pointing this out because if you give Carb Pilot a try and have an idea for a feature/wish you could change the app in some way, I would LOVE for you to email me and tell me about it. I have a few other improvements I’m already planning to add but I’d love to make this as useful to as many people who would find this type of app helpful.

Why (was) there a subscription for ongoing AI use?

For both PERT Pilot and Carb Pilot, there is a cost (expense) to using the AI meal estimation. I have to pay OpenAI (which hosts the AI I’m using for the app) to use the AI for each meal estimation, and I have to host a web server to communicate between the app and the AI, which also costs a bit for every time we send a meal estimation request from the app. That’s why I decided to make Carb Pilot free to download and try. I originally played with $1.99 a month for unlimited AI meal estimations, but temporarily have turned that off to see what that does to the server load and cost, so right now it’s free to use the AI features as well.

TLDR:

– PERT Pilot has been updated to include the new meal estimation feature!

– People without EPI can use Carb Pilot for carb, protein, fat, and/or calorie tracking (of just one or any selection of those) tracking, also using the new AI meal estimation features!

You can find PERT Pilot here or Carb Pilot here on the App Store.

PERT Pilot – the first iOS app for Exocrine Pancreatic Insufficiency (EPI or PEI) and Pancreatic Enzyme Replacement Therapy (PERT)

by Dana Lewis

Introducing PERT Pilot, the first iOS app designed for people with exocrine pancreatic insufficiency (EPI / PEI) and the only iOS app for specifically recording pancreatic enzyme replacement therapy (PERT) dosing!

*Available to download for FREE on the iOS App Store *
The PERT Pilot logo - PERT is in all caps and bold purple font, the word "Pilot" is in a script font in black placed below PERT.

After originally developing GI symptoms, then working through the long journey to diagnosis with exocrine pancreatic insufficiency (known as EPI or PEI), I’ve had to come up methods to figure out the right dosing of PERT for my EPI. I realized that the methods that I’ve made work for me – logging what I was eating in a spreadsheet and using it to determine the ratios I needed to use to dose my pancreatic enzyme replacement therapy (PERT) – weren’t methods that other people were as comfortable using. I have been thinking about this for the last year or more, and in my pursuit for wanting to encourage others to improve their outcomes with EPI (and realize that it IS possible to get to few symptoms, based on increasing/titrating the enzymes we take based on what we eat), I wrote a very long blog post explaining these methods and also sharing a free web-based calculator to help others to calculate their ratios.

But, that still isn’t the most user-friendly way to enable people to do this.

What else could I do, though? I wasn’t sure.

More recently, though, I have been experimenting with various projects and using ‘large language model’ (LLM) tools like GPT-4 to work on various projects. And a few weeks ago I realized that maybe I could *try* to build an iOS app version of my idea. I wanted something to help people log what they are eating, record their PERT dosing, and more easily see the relationship in what they are eating and what enzymes they are dosing. This would enable them to use that information to more easily adjust what they are dosing for future meals if they’re not (yet) satisfied with their outcomes.

And thus, PERT Pilot was born!

Screenshots from the PERT Pilot app which show the home screen, the calculator where you enter what PERT you're taking and a typical meal, plus the resulting ratios screen that show you the relationship between what you ate and how many enzymes you dosed.

What does PERT Pilot do?

PERT Pilot is designed to help people living with Exocrine Pancreatic Insufficiency (EPI or PEI) more easily deal with pancreatic enzyme replacement therapy (PERT). Aka, “taking enzymes”.

The PERT Pilot calculator enables you log the PERT that you are taking along with a meal, how many pills you take for it, and whether this dosing seems to work for you or not.

PERT Pilot then shows you the relationship between how much PERT you have been taking and what you are eating, supporting you as you fine-tune your enzyme intake.

PERT Pilot also enables you to share what’s working – and what might not be working – with your healthcare provider. PERT Pilot not only lists every meal you’ve entered, but also has a visual graph so you can see each meal and how much fat and protein from each meal were dosed by one pill – and it’s color coded by the outcome you assigned that meal! Green means you said that meal’s dosing “worked”; orange means you were “unsure”, and red matches the meals you said “didn’t work” for that level of dosing.

You can press on any meal and edit it, and you can swipe to delete a meal.

PERT Pilot also has is an education section so you can learn more about EPI and why you need PERT, and how this approach to ratios may help you more effectively dose your PERT in the future.

Why use PERT Pilot if you have EPI or PEI or PI?

PERT Pilot is the first and only specific app for those of us living with EPI (PEI or PI). People who use the approach in PERT Pilot of adapting their PERT dosing to what they are eating for each meal or snack often report fewer symptoms. PERT Pilot was designed and built by someone with exocrine pancreatic insufficiency, just like you!

With PERT Pilot you can:

  • Log your meals and PERT dosing. No other app specifically is designed for PERT dosing.
  • Edit or adjust your meal entry at any time – including if you wake up the next morning and realize your last dose from the day before ‘didn’t work’.
  • Review your dosing and see all of your meals, dosing, and outcomes – including a visual graph that shows you, for each meal, what one pill ‘covered’ so you can see where there are clusters of dosing that worked and if there are any clear patterns in what didn’t work for you.
  • You can also export your data, as a PDF list of all meals or a CSV file (which you can open in tools like Excel or other spreadsheet tools) if you want to analyze your data elsewhere!
  • Your data is your data, period. No one has access to your dosing data, meal data, or outcome data, and nothing you enter into PERT Pilot leaves your device – unless you decide to export your data. (See more in the PERT Pilot Privacy Policy.)

Note: this app was not funded by nor has any relationship to any pharmaceutical or medical-related companies. It’s simply built by a person with EPI for other people with EPI.

Here is a quick demonstration of PERT Pilot in action:

An animated gif of PERT Pilot in action

You can share your feedback about PERT Pilot:

Feel free to email me (Dana+PERTPilot@OpenAPS.org) any time.

I’d love to hear what works or is helpful, but also if something in the app isn’t yet working as expected.

Or, if you use another approved brand of PERT that’s not currently listed, let me know and I can add it in.

And, you can share your feature requests! I’m planning to build more features soon (see below).

What’s coming next for PERT Pilot:

I’m not done improving the functionality! I plan to add an AI meal estimation feature (UPDATE: now available!), so if you don’t know what’s in what you’re eating at a restaurant or someone else’s home cooked meal you can simply enter a description of the meal and have macronutrient estimates generated for you to use or modify.

—

Download PERT Pilot today! It’s free to download, so go ahead and download it and check it out! If you find it useful, please also leave a rating or review on the App Store to help other people find it in the future. You can also share it via social media, and give people a link to download it: https://bit.ly/PERT-Pilot-iOS