research

Effective Pair Programming and Coding and Prompt Engineering and Writing with LLMs like ChatGPT and other AI tools

by Dana Lewis

I’ve been puzzled when I see people online say that LLM’s “don’t write good code”. In my experience, they do. But given that most of these LLMs are used in chatbot mode – meaning you chat and give it instructions to generate the code – that might be where the disconnect lies. To get good code, you need effective prompting and to do so, you need clear thinking and ideas on what you are trying to achieve and how.

My recipe and understanding is:

Clear thinking + clear communication of ideas/request = effective prompting => effective code and other outputs

It also involves understanding what these systems can and can’t do. For example, as I’ve written about before, they can’t “know” things (although they can increasingly look things up) and they can’t do “mental” math. But, they can generally repeat patterns of words to help you see what is known about a topic and they can write code that you can execute (or it can execute, depending on settings) to solve a math problem.

What the system does well is help code small chunks, walk you through processes to link these sections of code up, and help you implement them (if you ask for it). The smaller the task (ask), the more effective it is. Or also – the easier it is for you to see when it completes the task and when it hasn’t been able to finish due to limitations like response length limits, information falling out of the context window (what it knows that you’ve told it); unclear prompting; and/or because you’re asking it to do things for which it doesn’t have expertise. Some of the last part – lack of expertise – can be improved with specific prompting techniques –  and that’s also true for right-sizing the task it’s focusing on.

Right-size the task by giving a clear ask

If I were to ask an LLM to write me code for an iOS app to do XYZ, it could write me some code, but it certainly wouldn’t (at this point in history, written in February 2024), write all code and give me a downloadable file that includes it all and the ability to simply run it. What it can do is start writing chunks and snippets of code for bits and pieces of files that I can take and place and build upon.

How do I know this? Because I made that mistake when trying to build my first iOS apps in April and May 2023 (last year). It can’t do that (and still can’t today; I repeated the experiment). I had zero ideas how to build an iOS app; I had a sense that it involved XCode and pushing to the Apple iOS App Store, and that I needed “Swift” as the programming language. Luckily, though, I had a much stronger sense of how I wanted to structure the app user experience and what the app needed to do.

I followed the following steps:

  1. First, I initiated chat as a complete novice app builder. I told it I was new to building iOS apps and wanted to use XCode. I had XCode downloaded, but that was it. I told it to give me step by step instructions for opening XCode and setting up a project. Success! That was effective.
  2. I opened a different chat window after that, to start a new chat. I told it that it was an expert in iOS programming using Swift and XCode. Then I described the app that I wanted to build, said where I was in the process (e.g. had opened and started a project in XCode but had no code yet), and asked it for code to put on the home screen so I could build and open the app and it would have content on the home screen. Success!
  3. From there, I was able to stay in the same chat window and ask it for pieces at a time. I wanted to have a new user complete an onboarding flow the very first time they opened the app. I explained the number of screens and content I wanted on those screens; the chat was able to generate code, tell me how to create that in a file, and how to write code that would trigger this only for new users. Success!
  4. I was able to then add buttons to the home screen; have those buttons open new screens of the app; add navigation back to the home; etc. Success!
  5. (Rinse and repeat, continuing until all of the functionality was built out a step at a time).

To someone with familiarity building and programming things, this probably follows a logical process of how you might build apps. If you’ve built iOS apps before and are an expert in Swift programming, you’re either not reading this blog post or are thinking I (the human) am dumb and inexperienced.

Inexperienced, yes, I was (in April 2023). But what I am trying to show here is for someone new to a process and language, this is how we need to break down steps and work with LLMs to give it small tasks to help us understand and implement the code it produces before moving forward with a new task (ask). It takes these small building block tasks in order to build up to a complete app with all the functionality that we want. Nowadays, even though I can now whip up a prototype project and iOS app and deploy it to my phone within an hour (by working with an LLM as described above, but skipping some of the introductory set-up steps now that I have experience in those), I still follow the same general process to give the LLM the big picture and efficiently ask it to code pieces of the puzzle I want to create.

As the human, you need to be able to keep the big picture – full app purpose and functionality – in mind while subcontracting with the LLM to generate code for specific chunks of code to help achieve new functionality in our project.

In my experience, this is very much like pair programming with a human. In fact, this is exactly what we did when we built DIYPS over ten years ago (wow) and then OpenAPS within the following year. I’ve talked endlessly about how Scott and I would discuss an idea and agree on the big picture task; then I would direct sub-tasks and asks that he, then also Ben and others would be coding on (at first, because I didn’t have as much experience coding and this was 10 years ago without LLMs; I gradually took on more of those coding steps and roles as well). I was in charge of the big picture project and process and end goal; it didn’t matter who wrote which code or how; we worked together to achieve the intended end result. (And it worked amazingly well; here I am 10 years later still using DIYPS and OpenAPS; and tens of thousands of people globally are all using open source AID systems spun off of the algorithm we built through this process!)

Two purple boxes. The one on the left says "big picture project idea" and has a bunch of smaller size boxes within labeled LLM, attempting to show how an LLM can do small-size tasks within the scope of a bigger project that you direct it to do. On the right, the box simply says "finished project". Today, I would say the same is true. It doesn’t matter – for my types of projects – if a human or an LLM “wrote” the code. What matters is: does it work as intended? Does it achieve the goal? Does it contribute to the goal of the project?

Coding can be done – often by anyone (human with relevant coding expertise) or anything (LLM with effective prompting) – for any purpose. The critical key is knowing what the purpose is of the project and keeping the coding heading in the direction of serving that purpose.

Tips for right-sizing the ask

  1. Consider using different chat windows for different purposes, rather than trying to do it all in one. Yes, context windows are getting bigger, but you’ll still likely benefit from giving different prompts in different windows (more on effective prompting below).Start with one window for getting started with setting up a project (e.g. how to get XCode on a Mac and start a project; what file structure to use for an app/project that will do XYZ; how to start a Jupyter notebook for doing data science with python; etc); brainstorming ideas to scope your project; then separately for starting a series of coding sub-tasks (e.g. write code for the home page screen for your app; add a button that allows voice entry functionality; add in HealthKit permission functionality; etc.) that serves the big picture goal.
  2. Make a list for yourself of the steps needed to build a new piece of functionality for your project. If you know what the steps are, you can specifically ask the LLM for that.Again, use a separate window if you need to. For example, if you want to add in the ability to save data to HealthKit from your app, you may start a new chat window that asks the LLM generally how does one add HealthKit functionality for an app? It’ll describe the process of certain settings that need to be done in XCode for the project; adding code that prompts the user with correct permissions; and then code that actually does the saving/revising to HealthKit.

    Make your list (by yourself or with help), then you can go ask the LLM to do those things in your coding/task window for your specific project. You can go set the settings in XCode yourself, and skip to asking it for the task you need it to do, e.g. “write code to prompt the user with HealthKit permissions when button X is clicked”.

    (Sure, you can do the ask for help in outlining steps in the same window that you’ve been prompting for coding sub-tasks, just be aware that the more you do this, the more quickly you’ll burn through your context window. Sometimes that’s ok, and you’ll get a feel for when to do a separate window with the more experience you get.)

  • Pay attention as you go and see how much code it can generate and when it falls short of an ask. This will help you improve the rate at which you successfully ask and it fully completes a task for future asks. I observe that when I don’t know – due to my lack of expertise – the right size of a task, it’s more prone to give me ½-⅔ of the code and solution but need additional prompting after that. Sometimes I ask it to continue where it cut off; other times I start implementing/working with the bits of code (the first ⅔) it gave me, and have a mental or written note that this did not completely generate all steps/code for the functionality and to come back.Part of why sometimes it is effective to get started with ⅔ of the code is because you’ll likely need to debug/test the first bit of code, anyway. Sometimes when you paste in code it’s using methods that don’t match the version you’re targeting (e.g. functionality that is outdated as of iOS 15, for example, when you’re targeting iOS 17 and newer) and it’ll flag a warning or block it from working until you fix it.

    Once you’ve debugged/tested as much as you can of the original ⅔ of code it gave you, you can prompt it to say “Ok, I’ve done X and Y. We were trying to (repeat initial instructions/prompt) – what are the remaining next steps? Please code that.” to go back and finish the remaining pieces of that functionality.

    (Note that saying “please code that” isn’t necessarily good prompt technique, see below).

    Again, much of this is paying attention to how the sub-task is getting done in service of the overall big picture goal of your project; or the chunk that you’ve been working on if you’re building new functionality. Keeping track with whatever method you prefer – in your head, a physical written list, a checklist digitally, or notes showing what you’ve done/not done – is helpful.

Most of the above I used for coding examples, but I follow the same general process when writing research papers, blog posts, research protocols, etc. My point is that this works for all types of projects that you’d work on with an LLM, whether the output generation intended is code or human-focused language that you’d write or speak.

But, coding or writing language, the other thing that makes a difference in addition to right-sizing the task is effective prompting. I’ve intuitively noticed that has made the biggest difference in my projects for getting the output matching my expertise. Conversely, I have actually peer reviewed papers for medical journals that do a horrifying job with prompting. You’ll hear people talk about “prompt engineering” and this is what it is referring to: how do you engineer (write) a prompt to get the ideal response from the LLM?

Tips for effective prompting with an LLM

    1. Personas and roles can make a difference, both for you and for the LLM. What do I mean by this? Start your prompt by telling the LLM what perspective you want it to take. Without it, you’re going to make it guess what information and style of response you’re looking for. Here’s an example: if you asked it what caused cancer, it’s going to default to safety and give you a general public answer about causes of cancer in very plain, lay language. Which may be fine. But if you’re looking to generate a better understanding of the causal mechanism of cancer; what is known; and what is not known, you will get better results if you prompt it with “You are an experienced medical oncologist” so it speaks from the generated perspective of that role. Similarly, you can tell it your role. Follow it with “Please describe the causal mechanisms of cancer and what is known and not known” and/or “I am also an experienced medical researcher, although not an oncologist” to help contextualize that you want a deeper, technical approach to the answer and not high level plain language in the response.

      Compare and contrast when you prompt the following:

      A. “What causes cancer?”

      B. “You are an experienced medical oncologist. What causes cancer? How would you explain this differently in lay language to a patient, and how would you explain this to another doctor who is not an oncologist?”

      C. “You are an experienced medical oncologist. Please describe the causal mechanisms of cancer and what is known and not known. I am also an experienced medical researcher, although not an oncologist.”

      You’ll likely get different types of answers, with some overlap between A and the first part of answer B. Ditto for a tiny bit of overlap between the latter half of answer B and for C.

      I do the same kind of prompting with technical projects where I want code. Often, I will say “You are an expert data scientist with experience writing code in Python for a Jupyter Notebook” or “You are an AI programming assistant with expertise in building iOS apps using XCode and SwiftUI”. Those will then be followed with a brief description of my project (more on why this is brief below) and the first task I’m giving it.

      The same also goes for writing-related tasks; the persona I give it and/or the role I reference for myself makes a sizable difference in getting the quality of the output to match the style and quality I was seeking in a response.

  • Be specific. Saying “please code that” or “please write that” might work, sometimes, but more often or not will get a less effective output than if you provide a more specific prompt.I am a literal person, so this is something I think about a lot because I’m always parsing and mentally reviewing what people say to me because my instinct is to take their words literally and I have to think through the likelihood that those words were intended literally or if there is context that should be used to filter those words to be less literal. Sometimes, you’ll be thinking about something and start talking to someone about something, and they have no idea what on earth you’re talking about because the last part of your out-loud conversation with them was about a completely different topic!

    LLMs are the same as the confused conversational partner who doesn’t know what you’re thinking about. LLMs only know what you’ve last/recently told it (and more quickly than humans will ‘forget’ what you told it about a project). Remember the above tips about brainstorming and making a list of tasks for a project? Providing a description of the task along with the ask (e.g. we are doing X related to the purpose of achieving Y, please code X) will get you better output more closely matching what you wanted than saying “please code that” where the LLM might code something else to achieve Y if you didn’t tell it you wanted to focus on X.

    I find this even more necessary with writing related projects. I often find I need to give it the persona “You are an expert medical researcher”, the project “we are writing a research paper for a medical journal”, the task “we need to write the methods section of the paper”, and a clear ask “please review the code and analyses and make an outline of the steps that we have completed in this process, with sufficient detail that we could later write a methods section of a research paper”. A follow up ask is then “please take this list and draft it into the methods section”. That process with all of that specific context gives better results than “write a methods section” or “write the methods” etc.

  • Be willing to start over with a new window/chat. Sometimes the LLM can get itself lost in solving a sub-task and lose sight (via lost context window) of the big picture of a project, and you’ll find yourself having to repeat over and over again what you’re asking it to do. Don’t be afraid to cut your losses and start a new chat for a sub-task that you’ve been stuck on. You may be able to eventually come back to the same window as before, or the new window might become your new ‘home’ for the project…or sometimes a third, fourth, or fifth window will.
  • Try, try again.
    I may hold the record for the longest running bug that I (and the LLM) could. Not. solve. This was so, so annoying. No users apparently noticed it but I knew about it and it bugged me for months and months. Every few weeks I would go to an old window and also start a new window, describe the problem, paste the code in, and ask for help to solve it. I asked it to identify problems with the code; I asked it to explain the code and unexpected/unintended functionality from it; I asked it what types of general things would be likely to cause that type of bug. It couldn’t find the problem. I couldn’t find the problem. Finally, one day, I did all of the above, but then also started pasting every single file from my project and asking if it was likely to include code that could be related to the problem. By forcing myself to review all my code files with this problem in mind, even though the files weren’t related at all to the file/bug….I finally spotted the problem myself. I pasted the code in, asked if it was a possibility that it was related to the problem, the LLM said yes, I tried a change and…voila! Bug solved on January 16 after plaguing me since November 8. (And probably existed before then but I didn’t have functionality built until November 8 where I realized it was a problem). I was beating myself up about it and posted to Twitter about finally solving the bug (but very much with the mindset of feeling very stupid about it). Someone replied and said “congrats! sounds like it was a tough one!”. Which I realized was a very kind framing and one that I liked, because it was a tough one; and also I am doing a tough thing that no one else is doing and I would not have been willing to try to do without an LLM to support.

    Similarly, just this last week on Tuesday I spent about 3 hours working on a sub-task for a new project. It took 3 hours to do something that on a previous project took me about 40 minutes, so I was hyper aware of the time mismatch and perceiving that 3 hours was a long time to spend on the task. I vented to Scott quite a bit on Tuesday night, and he reminded me that sure it took “3 hours” but I did something in 3 hours that would take 3 years otherwise because no one else would do (or is doing) the project that I’m working on. Then on Wednesday, I spent an hour doing another part of the project and Thursday whipped through another hour and a half of doing huge chunks of work that ended up being highly efficient and much faster than they would have been, in part because the “three hours” it took on Tuesday wasn’t just about the code but about organizing my thinking, scoping the project and research protocol, etc. and doing a huge portion of other work to organize my thinking to be able to effectively prompt the LLM to do the sub-task (that probably did actually take closer to the ~40 minutes, similar to the prior project).

    All this to say: LLMs have become pair programmers and collaborators and writers that are helping me achieve tasks and projects that no one else in the world is working on yet. (It reminds me very much of my early work with DIYPS and OpenAPS where we did the work, quietly, and people eventually took notice and paid attention, albeit slower than we wished but years faster than had we not done that work. I’m doing the same thing in a new field/project space now.) Sometimes, the first attempt to delegate a sub-task doesn’t work. It may be because I haven’t organized my thinking enough, and the lack of ideal output shows that I have not prompted effectively yet. Sometimes I can quickly fix the prompt to be effective; but sometimes it highlights that my thinking is not yet clear; my ability to communicate the project/task/big picture is not yet sufficient; and the process of achieving the clarity of thinking and translating to the LLM takes time (e.g. “that took 3 hours when it should have taken 40 minutes”) but ultimately still moves me forward to solving the problem or achieving the tasks and sub-tasks that I wanted to do. Remember what I said at the beginning:

    Clear thinking + clear communication of ideas/request = effective prompting => effective code and other outputs

 

  • Try it anyway.
    I am trying to get out of the habit of saying “I can’t do X”, like “I can’t code/program an iOS app”…because now I can. I’ve in fact built and shipped/launched/made available multiple iOS apps (check out Carb Pilot if you’re interested in macronutrient estimates for any reason; you can customize so you only see the one(s) you care about; or if you have EPI, check out PERT Pilot, which is the world’s first and only app for tracking pancreatic enzyme replacement therapy and has the same AI feature for generating macronutrient estimates to aid in adjusting enzyme dosing for EPI.) I’ve also made really cool, 100% custom-to-me niche apps to serve a personal purpose that save me tons of time and energy. I can do those things, because I tried. I flopped a bunch along the way – it took me several hours to solve a simple iOS programming error related to home screen navigation in my first few apps – but in the process I learned how to do those things and now I can build apps. I’ve coded and developed for OpenAPS and other open source projects, including a tool for data conversion that no one else in the world had built. Yet, my brain still tries to tell me I can’t code/program/etc (and to be fair, humans try to tell me that sometimes, too).

    I bring that up to contextualize that I’m working on – and I wish others would work on to – trying to address the reflexive thoughts of what we can and can’t do, based on prior knowledge. The world is different now and tools like LLMs make it possible to learn new things and build new projects that maybe we didn’t have time/energy to do before (not that we couldn’t). The bar to entry and the bar to starting and trying is so much lower than it was even a year ago. It really comes down to willingness to try and see, which I recognize is hard: I have those thought patterns too of “I can’t do X”, but I’m trying to notice when I have those patterns; shift my thinking to “I used to not be able to do X; I wonder if it is possible to work with an LLM to do part of X or learn how to do Y so that I could try to do X”.

    A recent real example for me is power calculations and sample size estimates for future clinical trials. That’s something I can’t do; it requires a statistician and specialized software and expertise.

    Or…does it?

    I asked my LLM how power calculations are done. It explained. I asked if it was possible to do it using Python code in a Jupyter notebook. I asked what information would be needed to do so. It walked me through the decisions I needed to make about power and significance, and highlighted variables I needed to define/collect to put into the calculation. I had generated the data from a previous study so I had all the pieces (variables) I needed. I asked it to write code for me to run in a Jupyter notebook, and it did. I tweaked the code, input my variables, ran it..and got the result. I had run a power calculation! (Shocked face here). But then I got imposter syndrome again, reached out to a statistician who I had previously worked with on a research project. I shared my code and asked if that was the correct or an acceptable approach and if I was interpreting it correctly. His response? It was correct, and “I couldn’t have done it better myself”.

    (I’m still shocked about this).

    He also kindly took my variables and put it in the specialized software he uses and confirmed that the results output matched what my code did, then pointed out something that taught me something for future projects that might be different (where the data is/isn’t normally distributed) although it didn’t influence the output of my calculation for this project.

    What I learned from this was a) this statistician is amazing (which I already knew from working with him in the past) and kind to support my learning like this; b) I can do pieces of projects that I previously thought were far beyond my expertise; c) the blocker is truly in my head, and the more we break out of or identify the patterns stopping us from trying, the farther we will get.

    “Try it anyway” also refers to trying things over time. The LLMs are improving every few months and often have new capabilities that didn’t before. Much of my work is done with GPT-4 and the more nuanced, advanced technical tasks are way more efficient than when using GPT-3.5. That being said, some tasks can absolutely be done with GPT-3.5-level AI. Doing something now and not quite figuring it out could be something that you sort out in a few weeks/months (see above about my 3 month bug); it could be something that is easier to do once you advance your thinking ; or it could be more efficiently done with the next model of the LLM you’re working with.

  • Test whether custom instructions help. Be aware though that sometimes too many instructions can conflict and also take up some of your context window. Plus if you forget what instructions you gave it, you might get seemingly unexpected responses in future chats. (You can always change the custom instructions and/or turn it on and off.)

I’m hoping this helps give people confidence or context to try things with LLMs that they were not willing to try before; or to help get in the habit of remembering to try things with LLMs; and to get the best possible output for the project that they’re working on.

Remember:

  • Right-size the task by making a clear ask.
  • You can use different chat windows for different levels of the same project.
  • Use a list to help you, the human, keep track of all the pieces that contribute to the bigger picture of the project.
  • Try giving the LLM a persona for an ask; and test whether you also need to assign yourself a persona or not for a particular type of request.
  • Be specific, think of the LLM as a conversational partner that can’t read your mind.
  • Don’t be afraid to start over with a new context window/chat.
  • Things that were hard a year ago might be easier with an LLM; you should try again.
  • You can do more, partnering with an LLM, than you can on your own, and likely can do things you didn’t realize were possible for you to do!

Clear thinking + clear communication of ideas/request = effective prompting => effective code and other outputs

—

Have any tips to help others get more effective output from LLMs? I’d love to hear them, please comment below and share your tips as well!

Tips for prompting LLMs like ChatGPT, written by Dana M. Lewis and available from DIYPS.org

Accepted, Rejected, and Conflict of Interest in Gastroenterology (And Why This Is A Symptom Of A Bigger Problem)

by Dana Lewis

Recently, someone published a new clinical practice update on exocrine pancreatic insufficiency (known as EPI or PEI) in the journal called Gastroenterology, from the American Gastroenterology Association (AGA). Those of you who’ve read any of my blog posts in the last year know how much I’ve been working to raise awareness of EPI, which is very under-researched and under-treated clinically despite the prevalence rates in the general population and key sub-populations such as PWD. So when there was a new clinical practice update and another publication on EPI in general, I was jazzed and set out to read it immediately. Then frowned. Because, like so many articles about EPI, it’s not *quite* right about many things and it perpetuates a lot of the existing problems in the literature. So I did what I could, which was to check out the journal requirements for writing a letter to the editor (LTE) in response to this article and drafting and submitting a LTE article about it. To my delight, on October 17, 2023, I got an email indicating that my LTE was accepted.

You can find my LTE as a pre-print here.

See below why this pre-print version is important, and why you should read it, plus what it reminds us about what journal articles can or cannot tell us in healthcare.

Here’s an image of my acceptance email. I’ll call out a key part of the email:

A print of the acceptance email I received on October 17, 2023, indicating my letter would be sent to authors of the original articles for a chance to choose to respond (or not). Then my LTE would be published.

Letters to the Editor are sent to the authors of the original articles discussed in the letter so that they might have a chance to respond. Letters are not sent to the original article authors until the window of submission for letters responding to that article is closed (the last day of the issue month in which the article is published). Should the authors choose to respond to your letter, their response will appear alongside your letter in the journal.

Given the timeline described, I knew I wouldn’t hear more from the journal until the end of November. The article went online ahead of print in September, meaning likely officially published in October, so the letters wouldn’t be sent to authors until the end of October.

And then I did indeed hear back from the journal. On December 4, 2023, I got the following email:

A print of the email I received saying the LTE was now rejected
TLDR: just kidding, the committee – members of which published the article you’re responding to – and the editors have decided not to publish your article. 

I was surprised – and confused. The committee members, or at least 3 of them, wrote the article. They should have a chance to decide whether or not to write a response letter, which is standard. But telling the editors not to publish my LTE? That seems odd and in contrast to the initial acceptance email. What was going on?

I decided to write back and ask. “Hi (name redacted), this is very surprising. Could you please provide more detail on the decision making process for rescinding the already accepted LTE?”

The response?

Another email explaining that possible commercial affiliations influenced their choice to reject the article after accpeting it originally
“In terms of this decision, possible commercial affiliations, as well as other judgments of priority and relevance among other submissions, dampened enthusiasm for this particular manuscript. Ultimately, it was not judged to be competitive for acceptance in the journal. “

Huh? I don’t have any commercial affiliations. So I asked again, “Can you clarify what commercial affiliations were perceived? I have none (nor any financial conflict of interest; nor any funding related to my time spent on the article) and I wonder if there was a misunderstanding when reviewing this letter to the editor.”

The response was “There were concerns with the affiliation with OpenAPS; with the use of the term “guidelines,” which are distinct from this Clinical Practice Update; and with the overall focus being more fit for a cystic fibrosis or research audience rather than a GI audience.”

A final email saying the concern with my affiliation of OpenAPS, which is not a commercial organization nor related to the field of gastroenterology and EPI

Aha, I thought, there WAS a misunderstanding. (And the latter makes no sense in the context of my LTE – the point of it is that most research and clinical literature is a too-narrow focus, cystic fibrosis as one example – the very point is that a broad gastroenterology audience should pay attention to EPI).

I wrote back and explained how I, as a patient/independent researcher, struggle to submit articles to manuscript systems without a Ringgold-verified organization. (You can also listen to me describe the problem in a podcast, here, and I also talked about it in a peer-reviewed journal article about citizen science and health-related journal publishing here). So I use OpenAPS as an “affiliation” even though OpenAPS isn’t an organization. Let alone a commercial organization. I have no financial conflict of interest related to OpenAPS, and zero financial conflict of interest or commercial or any type of funding in gastroenterology at all, related to EPI or not. I actually go to such extremes to describe even perceived conflicts of interest, even non-financial ones, as you can see this in my disclosure statement publicly available from the New England Journal of Medicine here on our CREATE trial article (scroll to Supplemental Information and click on Disclosure Forms) where I articulate that I have no financial conflicts of interest but acknowledge openly that I created the algorithm used in the study. Yet, there’s no commercial or financial conflict of interest.

A screenshot from the publicly available disclosure form on NEJM's site, where I am so careful to indicate possible conflicts of interest that are not commercial or financial, such as the fact that I developed the algorithm that was used in that study. Again, that's a diabetes study and a diabetes example, the paper we are discussing here is on exocrine pancreatic insufficiency (EPI) and gastroenterology, which is unrelated. I have no COI in gastroenterology.

I sent this information back to the journal, explaining this, and asking if the editors would reconsider the situation, given that the authors (committee members?) have misconstrued my affiliation, and given that the LTE was originally accepted.

Sadly, there was no change. They are still declining to publish this article. And there is no change in my level of disappointment.

Interestingly, here is the article in which my LTE is in reply to, and the conflict of interest statement by the authors (committee members?) who possibly raised a flag about supposed concern about my (this is not true) commercial affiliation:

The conflict of interest statement for authors from the article "AGA Clinical Practice Update on the Epidemiology, Evaluation, and Management of Exocrine Pancreatic Insufficiency 2023"

The authors disclose the following: David C. Whitcomb: consultant for AbbVie, Nestlé, Regeneron; cofounder, consultant, board member, chief scientific officer, and equity holder for Ariel Precision Medicine. Anna M. Buchner: consultant for Olympus Corporation of America. Chris E. Forsmark: grant support from AbbVie; consultant for Nestlé; chair, National Pancreas Foundation Board of Directors.

As a side note, one of the companies with consulting and/or grant funding to two of the three authors is the biggest manufacturer of pancreatic enzyme replacement therapy (PERT), which is the treatment for EPI. I don’t think this conflict of interest makes these clinicians ineligible to write their article; nor do I think commercial interests should preclude anyone from publishing – but in my case, it is irrelevant, because I have none. But, it does seem weird given the stated COI for my (actually not a) COI then to be a reason to reject a LTE, of all things.

Here’s the point, though.

It’s not really about the fact that I had an accepted article rejected (although that is weird, to say the least…).

The point is that the presence of information in medical and research journals does not mean that they are correct. (See this post describing the incorrect facts presented about prevalence of EPI, for example.)

And similarly, the lack of presence of material in medical and research journals does not mean that something is not true or is not fact! 

There is a lot of gatekeeping in scientific and medical research. You can see it illustrated here in this accepted-rejected dance because of supposed COI (when there are zero commercial ties, let alone COI) and alluded to in terms of the priority of what gets published.

I see this often.

There is good research that goes unpublished because editors decide not to prioritize it (aka do not allow it to get published). There are many such factors in play affecting what gets published.

There are also systemic barriers.

  • Many journals require fees (called article processing charges or “APC”s) if your article is accepted for publication. If you don’t have funding, that means you can’t publish there unless you want to pay $2500 (or more) out of pocket. Some journals even have submission fees of hundreds of dollars, just to submit! (At least APCs are usually only levied if your article is accepted, but you won’t submit to these journals if you know you can’t pay the APC). That means the few journals in your field that don’t require APCs or fees are harder to get published in, because many more articles are submitted (thus, influencing the “prioritization” problem at the editor level) to the “free” journals.
  • Journals often require, as previously described, your organization to be part of a verified list (maintained by a third party org) in order for your article to be moved through the queue once submitted. Instead of n/a, I started listing “OpenAPS” as my affiliation and proactively writing to admin teams to let them know that my affiliation won’t be Ringgold-verified, explaining that it’s not an org/I’m not at any institution, and then my article can (usually) get moved through the queue ok. But as I wrote in this peer-reviewed article with a lot of other details about barriers to publishing citizen science and other patient-driven work, it’s one of many barriers involved in the publication process. It’s a little hard, every journal and submission system is a little different, and it’s a lot harder for us than it is for people who have staff/support to help them get articles published in journals.

I’ve seen grant funders say no to funding researchers who haven’t published yet; but editors also won’t prioritize them to publish on a topic in a field where they haven’t been funded yet or aren’t well known. Or they aren’t at a prestigious organization. Or they don’t have the “right” credentials. (Ahem, ahem, ahem). It can be a vicious cycle for even traditional (aka day job) researchers and clinicians. Now imagine that for people who are not inside those systems of academia or medical organizations.

Yet, think about where much of knowledge is captured, created, translated, studied – it’s not solely in these organizations.

Thus, the mismatch. What’s in journals isn’t always right, and the process of peer review can’t catch everything. It’s not a perfect system. But what I want you to take away, if you didn’t already have this context, is an understanding that what’s NOT in a journal is not because the information is not fact or does not exist. It may have not been studied yet; or it may have been blocked from publication by the systemic forces in play.

As I said at the end of my LTE:

“It is also critical to update the knowledge base of EPI beyond the sub-populations of cystic fibrosis and chronic pancreatitis that are currently over-represented in the EPI-related literature. Building upon this updated research base will enable future guidelines, including those like the AGA Clinical Practice Update on EPI, to be clearer, more evidence-based, and truly patient-centric ensuring that every individual living with exocrine pancreatic insufficiency receives optimal care. “

PS – want to read my LTE that was accepted then rejected, meaning it won’t be present in the journal? Here it is on a preprint server with a DOI, which means it’s still easily citable! Here’s an example citation:

Lewis, D. Navigating Ambiguities in Exocrine Pancreatic Insufficiency. OSF Preprints. 2023. DOI: 10.31219/osf.io/xcnf6

New Survey For Everyone (Including You – Yes, You!) To Help Us Learn More About Exocrine Pancreatic Insufficiency

by Dana Lewis

If you’ve ever wanted to help with some of my research, this is for you. Yes, you! I am asking people in the general public to take a survey (https://bit.ly/GI-Symptom-Survey-All) and share their experiences.

Why?

Many people have stomach or digestion problems occasionally. For some people, these symptoms happen more often. In some cases, the symptoms are related to exocrine pancreatic insufficiency (known as EPI or PEI). But to date, there have been few studies looking at the frequency of symptoms – or the level of their self-rated severity – in people with EPI or what symptoms may distinguish EPI from other GI-related conditions.

That’s where this survey comes in! We want to compare the experiences of people with EPI to people without EPI (like you!).

Will you help by taking this survey?

Your anonymous participation in this survey will help us understand the unique experiences individuals have with GI symptoms, including those with conditions like exocrine pancreatic insufficiency (EPI). In particular, data contributed by people without EPI will help us understand how the EPI experience is different (or not).

A note on privacy:

  • The survey is completely anonymous; no identifying information will be collected.
  • You can stop the survey at any point.

Who designed this survey:

Dana Lewis, an independent researcher, developed the survey and will manage the survey data. This survey design and the choice to run this survey is not influenced by funding from or affiliations with any organizations.

What happens to the data collected in this survey:

The aggregated data will be analyzed for patterns and shared through blog posts and academic publications. No individual data will be shared. This will help fill some of the documented gaps in the EPI-related medical knowledge and may influence the design of targeted research studies in the future.

Have Questions?
Feel free to reach out to Dana+GISymptomSurvey@OpenAPS.org.

How else can you help?
Remember, ANYONE can take this survey. So, feel free to share the link with your family and friends – they can take it, too!

Here’s a link to the survey that you can share (after taking it yourself, of course!): https://bit.ly/GI-Symptom-Survey-All

You (yes you!) can help us learn about exocrine pancreatic insufficiency by taking the survey linked on this page.

You’d Be Surprised: Common Causes of Exocrine Pancreatic Insufficiency

by Dana Lewis

Academic and medical literature often is like the game of “telephone”. You can find something commonly cited throughout the literature, but if you dig deep, you can watch the key points change throughout the literature going from a solid, evidence-backed statement to a weaker, more vague statement that is not factually correct but is widely propagated as “fact” as people cite and re-cite the new incorrect statements.

The most obvious one I have seen, after reading hundreds of papers on exocrine pancreatic insufficiency (known as EPI or PEI), is that “chronic pancreatitis is the most common cause of exocrine pancreatic insufficiency”. It’s stated here (“Although chronic pancreatitis is the most common cause of EPI“) and here (“The most frequent causes [of exocrine pancreatic insufficiency] are chronic pancreatitis in adults“) and here (“Besides cystic fibrosis and chronic pancreatitis, the most common etiologies of EPI“) and here (“Numerous conditions account for the etiology of EPI, with the most common being diseases of the pancreatic parenchyma including chronic pancreatitis, cystic fibrosis, and a history of extensive necrotizing acute pancreatitis“) and… you get the picture. I find this statement all over the place.

But guess what? This is not true.

First off, no one has done a study on the overall population of EPI and the breakdown of the most common co-conditions.

Secondly, I did research for my latest article on exocrine pancreatic insufficiency in Type 1 diabetes and Type 2 diabetes and was looking to contextualize the size of the populations. For example, I know overall that diabetes has a ~10% population prevalence, and this review found that there is a median prevalence of EPI of 33% in T1D and 29% in T2D. To put that in absolute numbers, this means that out of 100 people, it’s likely that 3 people have both diabetes and EPI.

How does this compare to the other “most common” causes of EPI?

First, let’s look at the prevalence of EPI in these other conditions:

  • In people with cystic fibrosis, 80-90% of people are estimated to also have EPI
  • In people with chronic pancreatitis, anywhere from 30-90% of people are estimated to also have EPI
  • In people with pancreatic cancer, anywhere from 20-60% of people are estimated to also have EPI

Now let’s look at how common these conditions are in the general population:

  • People with cystic fibrosis are estimated to be 0.04% of the general population.
    • This is 4 in every 10,000 people
  • People with chronic pancreatitis combined with all other types of pancreatitis are also estimated to be 0.04% of the general population, so another 4 out of 10,000.
  • People with pancreatic cancer are estimated to be 0.005% of the general population, or 1 in 20,000.

What happens if you add all of these up: cystic fibrosis, 0.04%, plus all types of pancreatitis, 0.04%, and pancreatic cancer, 0.005%? You get 0.085%, which is less than 1 in 1000 people.

This is quite a bit less than the 10% prevalence of diabetes (1 in 10 people!), or even the 3 in 100 people (3%) with both diabetes and EPI.

Let’s also look at the estimates for EPI prevalence in the general population:

  • General population prevalence of EPI is estimated to be 10-20%, and if we use 10%, that means that 1 in 10 people may have EPI.

Here’s a visual to illustrate the relative size of the populations of people with cystic fibrosis, chronic pancreatitis (visualized as all types of pancreatitis), and pancreatic cancer, relative to the sizes of the general population and the relative amount of people estimated to have EPI:

Gif showing the relative sizes of populations of people with cystic fibrosis, chronic pancreatitis, pancreatic cancer, and the % of those with EPI, contextualized against the prevalence of these in the general population and those with EPI. It's a small number of people because these conditions aren't common, therefore these conditions are not the most common cause of EPI!

What you should take away from this:

  • Yes, EPI is common within conditions such as cystic fibrosis, chronic pancreatitis (and other forms of pancreatitis), and pancreatic cancer
  • However, these conditions are not common: even combined, they add up to less than 1 in 1000!
  • Therefore, it is incorrect to conclude that any of these conditions, individually or even combined, are the most common causes of EPI.

You could say, as I do in this paper, that EPI is likely more common in people with diabetes than all of these conditions combined. You’ll notice that I don’t go so far as to say it’s the MOST common, because I haven’t seen studies to support such a statement, and as I started the post by pointing out, no one has done studies looking at huge populations of EPI and the breakdown of co-conditions at a population level; instead, studies tend to focus on the population of a co-condition and prevalence of EPI within, which is a very different thing than that co-condition’s EPI population as a percentage of the overall population of people with EPI. However, there are some great studies (and I have another systematic review accepted and forthcoming on this topic!) that support the overall prevalence estimates in the general population being in the ballpark of 10+%, so there might be other ‘more common’ causes of EPI that we are currently unaware of, or it may be that most cases of EPI are uncorrelated with any particular co-condition.

—

(Need a citation? This logic is found in the introduction paragraph of a systematic review found here, of which the DOI is 10.1089/dia.2023.0157. You can also access a full author copy of it and my other papers here.)

—
You can also contribute to a research study and help us learn more about EPI/PEI – take this anonymous survey to share your experiences with EPI-related symptoms!

—

New Systematic Review of Exocrine Pancreatic Insufficiency (EPI) In Type 1 Diabetes and Type 2 Diabetes – Focusing on Prevalence and Treatment

by Dana Lewis

I’m thrilled that the research I did evaluating the prevalence and treatment of EPI in both Type 1 diabetes and Type 2 diabetes (also presented as a poster at #ADA2023 – read a summary of the poster here) has now been published as a full systematic review in Diabetes Technology and Therapeutics.

Here is a pre-edited submitted version of my article that you can access if you don’t have journal access; and as a reminder, copies of ALL of my research articles are available on this page: DIYPS.org/research!

And if you don’t want to read the full paper, this is what I think you should take away from it as a person with diabetes or as a healthcare provider:

    1. What is EPI? 

      Exocrine pancreatic insufficiency (known as EPI in some places, and PEI or PI in other places) occurs when the pancreas no longer produces enough enzymes to digest food. People with EPI take pancreatic enzyme replacement therapy (PERT) whenever they eat (or drink anything with fat/protein).

    2. If I have diabetes, or treat people with diabetes, why should I be reading the rest of this about EPI?EPI often occurs in people with cystic fibrosis, pancreatitis, and pancreatic cancer. However, since these diseases are rare (think <0.1% of the general population even when these groups are added up all together), the total number of people with EPI from these causes is quite low. On the other hand, EPI is also common in people with diabetes, but this is less well-studied and understood. The research on other co-conditions is more frequent and often people confuse the prevalence WITHIN those groups with the % of those conditions occurring overall in the EPI community.This paper reviews every paper that includes data on EPI and people with type 1 diabetes or type 2 diabetes to help us better understand what % of people with diabetes are likely to face EPI in their lifetime.
    3. How many people with type 1 diabetes or type 2 diabetes (or diabetes overall) get EPI?TLDR of the paper: EPI prevalence in diabetes varies widely, reported between 5.4% and 77% when the type of diabetes isn’t specified. For Type 1 diabetes, the median EPI prevalence is 33% (range 14-77.5%), and for Type 2 diabetes, the median is 29% (range 16.8-49.2%). In contrast, in non-diabetes control groups, the EPI prevalence ranges from 4.4% to 18% (median 13%). The differences in ranges might be due to geographic variability and different exclusion criteria across studies.Diabetes itself is prevalent in about 10% of the general population. As such, I hypothesize that people with diabetes likely constitute one of the largest sub-groups of individuals with EPI, in contrast to what I described above might be more commonly assumed.
    4. Is pancreatic enzyme replacement therapy (PERT) safe for people with diabetes? 

      Yes. There have been safety and efficacy studies in people with diabetes with EPI, and PERT is effective just like in any other group of people with EPI.

    5. What is the effect of pancreatic enzyme replacement therapy (PERT) on glucose levels in people with diabetes?
      PERT itself does not affect glucose levels, but PERT *d0es* impact the digestion of food, which then changes glucose levels! So, most PERT labels warn to watch for hypoglycemia or hyperglycemia but the medicine itself doesn’t directly cause changes in glucose levels. You can read a previous study I did here using CGM data to show the effect of PERT actually causing improved glucose after meals in someone with Type 1 diabetes. But, in the systematic review, I found only 4 articles that even made note of glucose levels, and only 1 (the paper I linked above) actually included CGM data. Most of the studies are old, so there are no definitive conclusions on whether hypoglycemia or hyperglycemia is more common when a person with diabetes and EPI starts taking PERT. Instead, it’s likely very individual depending on what they’re eating, insulin dosing patterns before, and whether they’re taking enough PERT to match what they’re eating.TLDR here: more studies are needed because there’s no clear single directional effect on glucose levels from PERT in people with diabetes.Note: based on the n=1 study above, and subsequent conversations with other people with diabetes, I hypothesize that high variability and non-optimal post-meal glucose outcomes may be an early ‘symptom’ of EPI in people with diabetes. I’m hoping to eventually generate some studies to evaluate whether we could use this type of data as an input to help increase screening of EPI in people with diabetes.
    6. How common is EPI (PEI / PI) compared to celiac and gastroparesis in Type 1 diabetes and Type 2 diabetes? 

      As a person with (in my case, Type 1) diabetes, I feel like I hear celiac and gastroparesis talked about often in the diabetes community. I had NEVER heard of EPI prior to realizing I had it. Yet, EPI prevalence in Type 1 and Type 2 diabetes is much higher than that of celiac or gastroparesis!The prevalence of EPI is much higher in T1 and T2 than the prevalence of celiac and gastroparesis.Celiac disease is more common in people with diabetes (~5%) than in the general public (0.5-1%). Gastroparesis, when gastric emptying is delayed, is also more common in people with diabetes (5% in PWD).However, the  prevalence of EPI is 14-77.5% (median 33%) in Type 1 diabetes and 16.8-49.2% (median 29%) in Type 2 diabetes (and 5.4-77% prevalence when type of diabetes was not specified). This again is higher than general population prevalence of EPI.

      This data emphasizes that endocrinologists and other diabetes care providers should be more prone to initiate screening (using the non-invasive fecal elastase test) for individuals presenting with gastrointestinal symptoms, as the rates of EPI in diabetes are much higher in both Type 1 and Type 2 diabetes than the rates of celiac and gastroparesis.

    7. What should I do if I think I have EPI?
      Record your symptoms and talk to your doctor and ask for a fecal elastase (FE-1) screening test for EPI. It’s non-invasive. If your results are less than or equal to 200 (μg/g), this means you have EPI and should start on PERT. If you or your doctor feel that your sample may have influenced the results of your test, you can always re-do the test. But if you’re dealing with diarrhea, going on PERT may resolve or improve the diarrhea and improve the quality of the sample for the next test result. PERT doesn’t influence the test result, so you can start PERT and re-run the test any time.Symptoms of EPI can vary. Some people experience diarrhea, while others experience constipation. Steatorrhea or smelly, messy stools that stick to the side of the toilet are also common EPI symptoms, as is bloating, abdominal pain, and generally not feeling well after you eat.

      If you’ve been diagnosed with EPI, you may also want to check out some of my other posts (DIYPS.org/EPI) about my personal experiences with EPI and also this post about the amount of enzymes needed by most people with EPI. You may also want to check out PERT Pilot, a free iOS app, for recording and evaluating your PERT dosing.

—

If you want to read the full article, you can find copies of all of my research articles at DIYPS.org/research

If you’d like to cite this specific article in your future research, here’s an example citation:

Lewis, D. A Systematic Review of Exocrine Pancreatic Insufficiency Prevalence and Treatment in Type 1 and Type 2 Diabetes. Diabetes Technology & Therapeutics. http://doi.org/10.1089/dia.2023.0157

Why DIY AID in 2023? #ADA2023 Debate

by Dana Lewis

I was asked to participate in a ‘debate’ about AID at #ADA2023 (ADA Scientific Sessions), representing the perspective that DIY systems should be an option for people living with diabetes.

I present this perspective as a person with type 1 diabetes who has been using DIY AID for almost a decade (and as a developer/contributor to the open source AID systems used in DIY) – please note my constant reminder that I am not a medical doctor.

Dr. Gregory P. Forlenza, an Associate Professor from Barbara Davis Center, presented a viewpoint as a medical doctor practicing in the US.

FYI: here are my disclosures and Dr. Forlenza’s disclosures:

On the left is my slide (Dana M. Lewis) showing I have no commercial support or conflicts of interest. My research in the last 3 years has previously been funded by the New Zealand Health Research Council (for the CREATE Trial); JDRF; and DiabetesMine. Dr. Forlenza lists research support from NIH, JDRF, NSF, Helmsley Charitable Trust, Medtronic, Dexcom, Abbott, Insulet, Tandem, Beta Bionics, and Lilly. He also lists Consulting/Speaking/AdBoard: Medtronic, Dexcom, Abbott, Insulet, Tandem, Beta Bionics, and Lilly.

I opened the debate with my initial presentation. I talk about the history of DIY in diabetes going back to the 1970s, when people with diabetes had to “DIY” with blood glucose meters because initially healthcare providers did not want people to fingerstick at home because they might do something with the information. Similarly, even insulin pumps and CGMs have been used in different “DIY” ways over the years – notably, people with diabetes began dosing insulin using CGM data for years prior to them being approved for that purpose. It’s therefore less of a surprise in that context to think about DIY being done for AID. (If you’re reading this you probably also know that DIY AID was done years before commercial AID was even available; and that there are multiple DIY systems with multiple pump and CGM options, algorithms, and phone options).

And, for people with diabetes, using DIY is very similar to how a lot of doctors recommend or prescribe doing things off label. Diabetes has a LOT of these types of recommendations, whether it’s different types of insulins used in pumps that weren’t approved for that type of insulin; medications for Type 2 being used for Type 1 (and vice versa); and other things that aren’t regulatory approved at all but often recommended anyway. For example, GLP-1’s that are approved for weight management and not glycemic control, but are often prescribed for glycemic control reasons. Or things like Vitamin D, which are widely prescribed or recommended as a supplement even though it is not regulatory-approved as a pharmaceutical agent.

I always like to emphasize that although open source AID is not necessarily regulated (but can be: one open source system has received regulatory clearance recently), that’s not a synonym for ‘no evidence’. There’s plenty of high quality scientific evidence on DIY use and non-DIY use of open source AID. There’s even a recent RCT in the New England Journal of Medicine, not to mention several other RCTs (see here and here, plus another pending publication forthcoming). In addition to those gold-standard RCTs, there are also reviews of large-scale big data datasets from people with diabetes using AID, such as this one where we reviewed 122 people’s glucose data representing 46,070 days’ worth of data; or another forthcoming publication where we analyzed the n=75 unique (distinct from the previous dataset) DIY AID users with 36,827 days’ of data (average of 491 days per participant) and also found above goal TIR outcomes (e.g. mean TIR 70-180 mg/dL of 82.08%).

Yet, people often choose to DIY with AID not just for the glucose outcomes. Yes, commercial AID systems (especially now second-generation) can similarly reach the goal of 70+% TIR on average. DIY helps provide more choices about the type and amount of work that people with diabetes have to put IN to these systems in order to get these above-goal OUTcomes. They can choose, overall or situationally, whether to bolus, count carbs precisely, announce meals at all, or only announce relative meal size while still achieving >80% TIR, no or little hypoglycemia, and less hyperglycemia. Many people using DIY AID for years have been doing no-bolus and/or no meal announcements at all, bringing this closer to a full closed loop, or at least, an AID system with very, very little user input required on a daily basis if they so choose. I presented data back in 2018(!) showing how this was being done in DIY AID, and it was recently confirmed in a randomized control trial (hello, gold standard!) showing that between traditional use (with meal announcements and meal boluses); meal announcement only (no boluses); and no announcement nor bolusing, that they all got similar outcomes in terms of TIR (all above-goal). There was also no difference in those modes of total daily insulin dose (TDD) or amount of carb intake. There was a small difference in time below range being slightly higher in the first mode (where people were counting carbs and bolusing) as compared to the other two modes – which suggests that MORE user input may actually be limiting the capabilities of the system!

The TLDR here is that people with diabetes can do less work/provide less input into AID and still achieve the same level of ideal, above-goal outcomes – and ongoing studies are showing the increased QOL and other patient-reported outcomes that also improve as a result.

Again, people may be predisposed to think that the main difference between commercial and DIY is whether or not it is regulatory approved (and therefore prescribable by doctors and able to be supported by a company under warranty); the bigger differences are instead around interoperability across devices, data access, and transparency of how the system works.

There’s even an international consensus statement on open source AID, created by an international group of 48 medical and legal experts, endorsed by 9 national and international diabetes organizations, supporting that open source AID used in DIY AID is a safe and effective treatment option, confirming that the scientific evidence exists and it has the potential to help people with diabetes and reduce the burden of diabetes. They emphasize that doctors should support patient (and caregiver) autonomy and choice of DIY AID, and state that doctors have a responsibility to learn about all options that exist including DIY. The consensus statement is focused on open source AID but also, in my opinion, applies to all AID: they say that AID systems should fully disclose how they operate to enable informed decisions and that all users should have real-time and open access to their own data. Yes, please! (This is true of DIY but not true of all commercial systems.)

The elephant in the room that I always bring up is cost, insurance coverage, and therefore access and accessibility of AID. Many places have government or insurance that won’t cover AID. For example, the proposed NICE guidelines in the UK wouldn’t provide AID to everyone who wants one. In other places, some people can get their pump covered but not CGM, or vice versa, and must pay out of pocket. Therefore in some cases, DIY has out of pocket costs (because it’s not covered by insurance), but is still cheaper than AID with insurance coverage (if it’s even covered).

I also want to remind everyone that choosing to DIY – or not – is not a once-in-a-lifetime decision. People who use DIY choose every day to use it and continue to use it; at any time, they could and some do choose to switch to a commercial system. Others try commercial, switch back to DIY, and switch back and forth over time for various reasons. It’s not a single or permanent decision to DIY!

The key point is: DIY AID provides safety and efficacy *and* user choice for people with diabetes.

—

Dr. Forlenza followed my presentation, talking about commercial AID systems and how they’ve moved through development more quickly recently. He points to the RCTs for each approved commercial system that exist, saying commercial AID systems work, and describing different feature sets and variety across commercial systems. He shared his thoughts on advantages of commercial systems including integration between components by the companies; regulatory approval meaning these systems can be prescribed by healthcare providers; company-provided warranties; and company provided training and support of healthcare providers and patients.

He makes a big point about a perceived reporting bias in social media, which is a valid point, and talks about people who cherry pick (my words) data to share online about their TIR.

He puts an observational study and the CREATE Trial RCT data up next to the commercial AID systems RCT data, showing how the second generation commercial AID reach similar TIR outcomes.

He then says “what are you #notwaiting for?”, pointing out in the US that there are 4 commercial systems FDA approved for type 1 diabetes. He says “Data from the DIY trials themselves demonstrate that DIY users, even with extreme selection bias, do not achieve better glycemic control than is seen with commercial systems.” He concludes that commercial AID has a wide variety of options; commercial systems achieve target-level outcomes; a perception that both glucose outcomes and QOL are being addressed by the commercial market, and that “we do not need Unapproved DIY solutions in this space”.

—

After Dr. Forlenza’s presentation, I began my rebuttal, starting with pointing out that he is incorrectly conflating perceived biases/self-reporting of social media posts with gold-standard, rigorously performed scientific trials evaluating DIY. Data from DIY AID trials do not suffer from ‘selection bias’ any more than commercial AID trials do. (In fact, all clinical trials have their own aspects of selection bias, although that isn’t the point here.) I reminded the audience of the not one but multiple RCTs available as well as dozens of other prospective and retrospective clinical trials. Plus, we have 82,000+ data points analyzed showing above-goal outcomes, and many studies that evaluate this data and adjust for starting outcomes still show that people with diabetes who use DIY AID benefit from doing so, regardless of their starting A1c/TIR or demographics. This isn’t cherry-picked social media anecdata.

When studies are done rigorously, as they have been done in DIY, we agree that now second-generation commercial AID systems reach (or exceed, depending on the system) ADA standard of care outcomes. For example, Dr. Forlenza cited the OP5 study with 73.9% TIR which is similar to the CREATE Trial 74.5% TIR.

My point is not that commercial systems don’t work; my point is that DIY systems *do* work and that the fact that commercial systems work doesn’t then override the fact that DIY systems have been shown to work, also! It’s a “yes, and”! Yes, commercial AID systems work; and yes, DIY AID systems work.

The bigger point, which Dr. Forlenza does not address, is that the person with diabetes should get to CHOOSE what is best for them, which is not ONLY about glucose outcomes. Yes, a commercial system- like DIY AID – may help someone get to goal TIR (or above goal), but DIY provides more choice in terms of the input behaviors required to achieve those outcomes! There’s also possible choice of systems with different pumps or CGMs, different (often lower) cost, increased data access and interoperability of data displays, different mobile device options, and more.

Also, supporting user choice of DIY is in fact A STANDARD OF CARE!

It’s in the ADA’s Standards of Care, in fact, as I wrote about here when observing that it’s in the 2023 Standards of Care…as well as in 2022, 2021, 2020, and 2019!

I wouldn’t be surprised if there are people attending the debate who think they don’t have any – or many – patients using DIY AID. For those who think that (or are reading this thinking the same), I ask a question: how many patients have you asked if they are using DIY AID?

There’s a bunch of reasons why it may not come up, if you haven’t asked:

  • They may use the same consumables (sites, reservoirs) with a different or previous pump in a DIY AID system.
  • Their prescribed pump (particularly in Europe and non-US places that have Bluetooth-enabled pumps) may be usable in a DIY AID.
  • They may not be getting their supplies through insurance, so their prescription doesn’t match what they are currently using.
  • Or, they have more urgent priorities to discuss at appointments, so it doesn’t come up.
  • Or, it’s also possible that it hasn’t come up because they don’t need any assistance or support from their healthcare provider.

Speaking of learning and support, it’s worth noting that in DIY AID, because it is open source and the documentation is freely available, users typically begin learning more about the system prior to initiating their start of closed loop (automated insulin delivery). As a result, the process of understanding and developing trust in the system begins prior to closed loop start as well. In contrast, much of the time there is limited available education prior to receiving the prescription for a commercial AID; it often aligns more closely with the timeline of starting the device. Additionally, because it is a “black box” with fewer available details about exactly how it works (and why), the process of developing trust can be a slower process that occurs only after a user begins to use a commercial device.

With DIY AID, because it is open source and the documentation is freely available, users typically begin learning more about the system prior to initiating their start of closed loop (automated insulin delivery). As a result, the process of understanding and developing trust in the system begins prior to closed loop start as well. In contrast, much of the time there is limited available education prior to receiving the prescription for a commercial AID; it often aligns more closely with the timeline of starting the device. Additionally, because it is a black box with less available details about exactly how it works (and why), the process of developing trust can be a slower process that occurs only after a user begins to use a commercial device. The learning & trust in AID timelines is something that needs more attention in commercial AID moving forward.

I closed my rebuttal section by asking a few questions out loud:

I wonder how healthcare providers feel when patients learn something before they do – which is often what happens with DIY AID. Does it make you uncomfortable, excited, curious, or some other feeling? Why?

I encouraged healthcare providers to consider when they are comfortable with off-label prescriptions (or recommending things that aren’t approved, such as Vitamin D), and reflect on how that differs from understanding patients’ choices to DIY.

I also prompted everyone to consider whether they’ve actually evaluated (all of) the safety and efficacy data, of which many studies exist. And to consider who benefits from each type of system, not only commercial/DIY but individual systems within those buckets. And to consider who gets offered/prescribed AID systems (of any sort) and whether subconscious biases around tech literacy, previous glucose outcomes, and other factors (race, gender, other demographic variables) result in particular groups of people being excluded from accessing AID. I also remind everyone to think about what financial incentives influence access and available of AID education, and where the education comes from.

—

Although Dr. Forlenza’s  rebuttal followed mine, I’ll summarize it here before finishing a recap of my rebuttal: he talks about individual selection bias/cherry picked data, acknowledging it can occur in anecdotes with commercial systems as well; talks about the distinction of regulatory approval vs. off label and unapproved; legal concerns for healthcare providers; and closes pointing out that many PWD see primary care providers, he doesn’t believe it is reasonable to expect PCPs to become familiar with DIY since there are no paid device representatives to support their learning, and that growth of AID requires industry support.

—

People probably wanted to walk out of this debate with a black and white, clear answer on what is the ‘right’ type of AID system: DIY or commercial. The answer to that question isn’t straightforward, because it depends.

It depends on whether a system is even AVAILABLE. Not all countries have regulatory-approved systems available, meaning commercial AID is not available everywhere. Some places and people are also limited by ACCESSIBILITY, because their healthcare providers won’t prescribe an AID system to them; or insurance won’t cover it. AFFORDABILITY, even with insurance coverage, also plays a role: commercial AID systems (and even pump and CGM components without AID) are expensive and not everyone can afford them. Finally, ADAPTABILITY matters for some people, and not all systems work well for everyone.

When these factors align – they are available, accessible, affordable, and adaptable – that means for some people in some places in some situations, there are commercial systems that meet those needs. But for other people in other places in other situations, DIY systems instead or also can meet that need.

The point is, though, that we need a bigger overlap of these criteria! We need MORE AID systems to be available, accessible, affordable, and adaptable. Those can either be commercial or DIY AID systems.

The point that Dr. Forlenza and I readily agree on is that we need MORE AID – not less.

This is why I support user choice for people with diabetes and for people who want – for any variety of reasons – to use a DIY system to be able to do so.

People probably want a black and white, clear answer on what is the ‘right’ type of AID system: DIY or commercial. It depends on whether a system is even AVAILABLE. Not all countries have regulatory-approved systems available, meaning commercial AID is not available everywhere. Some places and people are also limited by ACCESSIBILITY, because their healthcare providers won’t prescribe an AID system to them; or insurance won’t cover it. AFFORDABILITY, even if insurance coverage, also plays a role: commercial AID systems (and even pump and CGM components without AID) are expensive and not everyone can afford them. Finally, ADAPTABILITY matters for some people, and not all systems work well for everyone. The point is that we need a bigger overlap of these criteria! We need more alignment of these factors - more AID (DIY and commercial) available, accessible, affordable, and adaptable for people with diabetes. I support user choice for people with diabetes, which includes DIY AID systems

—

PS – I also presented a poster at #ADA2023 about the high prevalence rates of exocrine pancreatic insufficiency (EPI / PEI / PI) in Type 1 and Type 2 diabetes – you can find the poster and a summary of it here.

Exocrine Pancreatic Insufficiency (EPI/PEI) In Type 1 and Type 2 Diabetes – Poster at #ADA2023

by Dana Lewis

When I was invited to contribute to a debate on AID at #ADA2023 (read my debate recap here), I decided to also submit an abstract related to some of my recent work in researching and understanding the prevalence and treatment of exocrine pancreatic insufficiency (known as EPI or PEI or PI) in people with diabetes.

I have a personal interest in this topic, for those who aren’t aware – I was diagnosed with EPI last year (read more about my experience here) and now take pancreatic enzyme replacement therapy (PERT) pills with everything that I eat.

I was surprised that it took personal advocacy to get a diagnosis, and despite having 2+ known risk factors for EPI (diabetes, celiac disease), that when I presented to a gastroenterologist with GI symptoms, EPI never came up as a possibility. I looked deeper into the research to try to understand what the correlation was in diabetes and EPI and perhaps understand why awareness is low compared to gastroparesis and celiac.

Here’s what I found, and what my poster (and a forthcoming full publication in a peer-reviewed journal!) is about (you can view my poster as a PDF here):

1304-P at #ADA2023, “Exocrine Pancreatic Insufficiency (EPI / PEI)  Likely Overlooked in Diabetes as Common Cause of Gastrointestinal-Related Symptoms”

Exocrine Pancreatic Insufficiency (EPI / PEI / PI) occurs when the pancreas no longer makes enough enzymes to support digestion, and is treated with pancreatic enzyme replacement therapy (PERT). Awareness among diabetes care providers of EPI does not seem to match the likely rates of prevalence and contributes to underscreening, underdiagnosis, and undertreatment of EPI among people with diabetes.

Methods:

I performed a broader systematic review on EPI, classifying all articles based on co-condition. I then did a second specific diabetes-specific EPI search, and de-duplicated and combined the results. (See PRISMA figure).

A PRISMA diagram showing that I performed two separate literature searches - one broadly on EPI before classifying and filtering for diabetes, and one just on EPI and diabetes. After filtering out irrelevant, animal, and off topic papers, I ended up with 41

I ended up with 41 articles specifically about EPI and diabetes, and screened them for diabetes type, prevalence rates (by type of diabetes, if it was segmented), and whether there were any analyses related to glycemic outcomes. I also performed an additional literature review on gastrointestinal conditions in diabetes.

Results:

From the broader systematic review on EPI in general, I found 9.6% of the articles on specific co-conditions to be about diabetes. Most of the articles on diabetes and EPI are simply about prevalence and/or diagnostic methods. Very few (4/41) specified any glycemic metrics or outcomes for people with diabetes and EPI. Only one recent paper (disclosure – I’m a co-author, and you can see the full paper here) evaluated glycemic variability and glycemic outcomes before and after PERT using CGM.

There is a LOT of work to be done in the future to do studies with properly recording type of diabetes; using CGM and modern insulin delivery therapies; and evaluating glycemic outcomes and variabilities to actually understand the impact of PERT on glucose levels in people with diabetes.

In terms of other gastrointestinal conditions, healthcare providers typically perceive the prevalence of celiac disease and gastroparesis to be high in people with diabetes. Reviewing the data, I found that celiac has around ~5% prevalence (range 3-16%) in people with type 1 diabetes and ~1.6% prevalence in Type 2 diabetes, in contrast to the general population prevalence of 0.5-1%. For gastroparesis, the rates in Type 1 diabetes were around ~5% and in Type 2 diabetes around 1.3%, in contrast to the general population prevalence of 0.2-0.9%.

Speaking of contrasts, let’s compare this to the prevalence of EPI in Type 1 and Type 2 diabetes.

  • The prevalence of EPI in Type 1 diabetes in the studies I reviewed had a median of 33% (range 14-77.5%).
  • The prevalence of EPI in Type 2 diabetes in the studies I reviewed had a median of 29% (16.8-49.2%).

You can see this relative prevalence difference in this chart I used on my poster:

The prevalence of EPI is much higher in T1 and T2 than the prevalence of celiac and gastroparesis.

Key Findings and Takeaways:

Gastroparesis and celiac are often top of mind for diabetes care providers, yet EPI may be up to 10 times more common among people with diabetes! EPI is likely significantly underdiagnosed in people with diabetes.

Healthcare providers who see people with diabetes should increase the screening of fecal elastase (FE-1/FEL-1) for people with diabetes who mention gastrointestinal symptoms.

With FE-1 testing, results <=200 μg/g are indicative of EPI and people with diabetes should be prescribed PERT. The quality-of-life burden and long-term clinical implications of undiagnosed EPI are significant enough, and the risks are low enough (aside from cost) that PERT should be initiated more frequently for people with diabetes who present with EPI-related symptoms.

EPI symptoms aren’t just diarrhea and/or weight loss: they can include painful bloating, excessive gas, changed stools (“messy”, “oily”, “sticking to the toilet bowl”), or increased bowel movements. People with diabetes may subconsciously adjust their food choices in response to symptoms for years prior to diagnosis.

Many people with diabetes and existing EPI diagnoses may be undertreated, even years after diagnosis. Diabetes providers should periodically discuss PERT dosing and encourage self-adjustment of dosing (similar to insulin, matching food intake) for people with diabetes and EPI who have ongoing GI symptoms. This also means aiding in updating prescriptions as needed. (PERT has been studied and found to be safe and effective for people with diabetes.)

Non-optimal PERT dosing may result in seemingly unpredictable post-meal glucose outcomes. Non-optimal postprandial glycemic excursions may be a ‘symptom’ of EPI because poor digestion of fat/protein may mean carbs are digested more quickly even in a ’mixed meal’ and result in larger post-meal glucose spikes.

—

As I mentioned, I have a full publication with this systematic review undergoing peer review and I’ll share it once it’s published. In the meantime, if you’re looking for more personal experiences about living with EPI, check out DIYPS.org/EPI, and also for people with EPI looking to improve their dosing with pancreatic enzyme replacement therapy – you may want to check out PERT Pilot (a free iOS app to record enzyme dosing).

Researchers, if you’re interested in collaborating on studies in EPI (in diabetes, or more broadly on EPI), please reach out! My email is Dana@OpenAPS.org

Air Quality, CO2 monitoring, and Situational Masking

by Dana Lewis

I do a lot of things most people don’t want to do themselves – and I get that. (For example, recording macronutrients while running? Running for up to 16 or 25 hours? Let alone other choices like building DIY and making open source automated insulin delivery systems not only for myself but more widely available for other people.) I’ve also talked before about functional self-tracking and how I don’t track things for the sake of tracking, I track when the data/information is actionable either retrospectively or in real-time.

I’ve spent enough time now collecting real-time data on air quality (via a proxy of CO2 levels) that I think it would be useful to share for other people to consider the retrospective data for THEIR decision making.

You may not want (or be able to afford) a CO2 monitor, and you may not want to mask inside all the time, but the below outlines the general scenarios in which air quality tends to be better or worse and when you would get the most benefit from situational masking in response to those situations.

(Think about situational masking indoors like you think about situational masking for smoke and poor air quality outside. Most of the time, you likely don’t mask outside. But if you’re on the east coast right now or have lived through a previous west coast US summer with a “smoke season”, you’ve probably experienced multi-day air quality outside that was so poor that you considered or did wear a high-quality (N95/K95) mask outside or limit your time exposed to that outdoor air.)

Air quality assessment via CO2 monitoring

In the last few years, Scott and I acquired two different CO2 monitors. The first was cheap, required to be plugged into a battery pack to run it, and was simply viewable on the device display. It was useful to start to get a sense for what the CO2 levels were in indoor spaces as compared to outdoor air.

Later, we decided to invest in an Aranet CO2 monitor, which runs on two AA batteries and lasts months on a single pair of batteries. You can view the data on the device display AND see a retrospective and realtime graph of the data in your phone, because it connects via Bluetooth. You can see not only CO2 but also temperature, humidity, and air pressure.

We have found this useful because CO2 is something that we all produce when we breathe out. The more we breathe out, and the more people that are breathing out, the higher the CO2 levels. The more of that air that is replaced with low-CO2 outside air, the lower the CO2 levels. Measuring the CO2 then helps us understand the ventilation (how much air is flowing through the space and how often it is being cleared out) and the risks of being in that space. A higher CO2 level means more people and/or less air being cleared out of the space, meaning  you are more likely to be breathing in what someone else is breathing out.

How we evaluate CO2 levels

An outdoor CO2 level would be around ~450 ppm in urban areas, or as low as 400 ppm out in nature. Since a perfectly-ventilated space would be 100% outside air, we want to compare any indoor air CO2 reading to outdoor air.

For example, at home in our enclosed apartment with 2 people (and 2 cats), we typically run around 700 ppm, which means ~250 ppm above outdoor air levels. When we open our door or a window, it drops to ~500 ppm, or only ~50 ppm above outdoor air levels. Given that we have confirmed our air intake into our HVAC system for our apartment is outdoor air, this means the ~250 ppm we are sharing between the two of us is just our (and the cats) exhalations, rather than anyone outside our household. So those levels are acceptable to us, but our choice of interventions would change if we were sharing air with other people, especially random strangers. (Stranger danger is one way to think about air, further contextualized below with data.)

In a shared space with random strangers, your risk of COVID aerosol-based transmission is proportional to how elevated the CO2 level is above that of outside air, and the amount of time spent in that space. So a CO2 reading of 650 ppm, which is ~200 ppm over outside air, would be half as risky as a reading of 850 ppm, or ~400 ppm over outside air. And timing matters, so a 1 hour bus ride or the hour you spend boarding and waiting for takeoff on your plane when CO2 levels are highest and the air filtration (see below) is off will be of greater risk than short exposure to the same levels.

Now, we’ve also used our CO2 monitors in many other places, such as in airports and on planes and other public transportation, and other indoor shared spaces like grocery stores etc.

Here’s what we’ve learned about where CO2 levels trend (based on our repeated but n=1 testing).

Trains, buses, and rideshare (e.g. Uber, Lyft, etc) = BAD NEWS BEAR AIR

Public transportation, in every location and every country we have been in, has much higher CO2 levels.

What do I mean by much higher? Often 1000-1500 ppm easily (and sometimes >2000 ppm), which is anywhere from 500-1500 ppm above outdoor air quality.

Trains/metros/light rail where the doors are constantly opening and closing to outdoor air would seem like they would be better, but sometimes they still have (due to the density of riders) >1500 ppm.

Buses where you can’t open the window can be as high of CO2 levels as planes, without the benefit of air exchange or HEPA filtration of the air. Our recent 20 minute bus ride was up to >2500 ppm on a full bus.

Watch out for rideshares, too. Often times we get in a rideshare and the driver intentionally or accidentally has “recirc” or “recirculating air” on, meaning the air isn’t exchanged outside and the driver and riders are re-breathing each other’s air over and over and over and over again..yikes. Specifically looking at the console when you get in the car is useful: if you see the recirc button lit up, ask the driver to turn it off. If they don’t understand or refuse, or you don’t want to try to explain it, opening a window helps immensely to reduce the CO2 levels and the amount of re-breathing air. (The recirc icon usually looks like a car with a U-shaped arrow on it).

Planes (including airports, during boarding, in flight, and during landing/deplaning) = ALSO BAD NEWS BEAR AIR

Airports sometimes have better-ventilated spaces: you can often find less crowded corners of a terminal and see CO2 readings of <900 ppm. However, it’s still pretty common to be in the airport and see >1000 ppm, meaning that the CO2 is >500 ppm above outdoor air quality, and it is air from a whole assortment of random strangers coming and going, so it’s less safe than the air you’d be breathing in at home or in private spaces.

When boarding, both standing close in line with other people but also on the jet bridge and while you are on the plane, is usually even HIGHER CO2 levels than the airport. The typical air for a plane (that they tout with HEPA filters and high air exchange rates) is not turned on until you start to take off, and then it takes some time to exchange all of the air. This means there is a MUCH higher rate of re-breathing other people’s air while boarding and until you are in the air.

Now, we have measured CO2 levels during all of these times. If indoor airport air is around 900 ppm, it usually jumps to 1100-1300 on the jetbridge (if you’ve got a backed up line) and when you’re sitting on the plane watching other people board, it can go up to 2500+. And then it continues to go up as you have a full flight of people breathing in this enclosed space. During flight, we’ve seen CO2 levels hover between 1700-3000 ppm, and in some cases have gone up to ~4000 ppm. This is a lot of CO2! However, there are HEPA filters cleaning the ~half of the air that is recirculated instead of replaced. So, it’s harder to say when the airplane air systems ARE running (during most of the flight) whether the risk is as high (for infectious disease transmission) as it is in other environments that aren’t studiously exchanging and HEPA-filtering any recirculated air.

Note that when they spin down the engines after landing and all the way through taxiing, deplaning, and getting back into the airport – the CO2 level again tends to rise because they again change the air flow when they’re on the ground. So like standing in line to get on or waiting for other people to board, standing in line to get off/waiting for everyone to get off produces high CO2 levels *without the benefit of in-flight air exchange*, so it’s likely higher risk during those times than in the air during the middle of the flight, even if CO2 levels are equally high during flight.

Indoor spaces like grocery stores or conference rooms/meeting halls

Indoor spaces can vary quite a bit, and often by country or venue.

For example, most indoor spaces in the US we’ve found to often have a fairly high (e.g. 900+ ppm) indoor CO2 level, even without a huge density of people. For example, we quickly went into a grocery store the other day and the CO2 was high-800s without being around many people in the aisles, across the entire store. For not having people actively occupying the space, this is fairly high and less optimal.

In contrast, we recently were in Sweden for a conference and were honestly gobsmacked when we got off the plane and found the CO2 levels to be <600 ppm in the airport! And in the hotel lobby! And in the hotel elevator! And at the local grocery store!

(Seriously, it shocked us, because we’ve also recently been in the UK with our CO2 monitor and found US-like CO2 levels typically around 900-1000 ppm or higher, and also in Spain last year where we similarly found it to be >900 ppm even when not densely occupied. The exception to optimal air quality in Sweden was our ~20 minute bus ride where CO2 levels were >2500 ppm).

So, the CO2 levels may vary quite a bit and this is why measuring is helpful. Because you can’t assume that one country/one room means that all of the rooms in that country or even that venue will be the same.

Case in point? Conference rooms/halls or meeting rooms.

In Barcelona, Spain in April 2022, I spoke at a conference. The CO2 levels in the hallways and in the meeting room before the session started were around 800-900 ppm when not occupied. Again, a little high for not having people actively in the spaces. Then, when the conference started, Scott watched the CO2 monitor and saw it rise..and rise…and rise. Within 45 minutes, the CO2 levels were around 2000 ppm (>1500 ppm over outdoor air quality)! He went to the back of the room and opened the doors to try to get some air circulating in the room, although it didn’t make a big difference. That room did not have a high number of air exchanges per hour and was not successfully clearing out the air people were breathing out.

In Sweden (May 2023, where the CO2 was <600 in a lot of public indoor spaces), we found the same challenge in a high ceiling, large meeting hall. With 300 people, the start of the session had about 950 ppm (as opposed to the <600 ppm of less occupied hallways). Not too bad given 300 people in the space. However, by the end of the session, the CO2 level had risen to ~1350! And it continued to rise even as people had exited the room; we didn’t see a drop in CO2 levels until we went out in the hallway to continue talking to people, and it took another ~25 minutes before CO2 levels in the hallway were back down <600 ppm.

Again, we were surprised, because this venue (the hallways, lobby, elevator, etc) all had really great otherwise indoor air quality with CO2 <600 ppm!

But the challenge is the space (and the infrastructure for filtration and air exchanges); the number of people filling the space; and the amount of time, in terms of what happens to the CO2 levels.

The takeaway from this? Conference halls, meeting rooms, and anywhere where you are sitting with a group of people over a period of time is going to have a much higher CO2 level and it will increase in proportion to the time that you are occupying that space (e.g. a 30 minute or 1 hour session is going to have a much higher CO2 buildup than a 10 minute talk where the audience is turning over and leaving the room and it clears out some before the next session).

So what should you do about this information? Consider situational masking.

I really have found a CO2 monitor helpful, because even my best guesses about air quality (e.g. thinking Sweden’s conference hall would have good air quality given the size of the room and ceilings) aren’t always accurate. But if you don’t want to invest in a CO2 monitor, here’s where you can get the biggest bang for your buck with situational masking.

What do I mean by situational masking? Maybe you don’t think you’re at very high risk for COVID or other infectious illnesses, but you are interested in reducing the likelihood that you spread anything you get to other people (thanks!). But you don’t want to have to think about it, and maybe you’ve chosen previously to drop masking so you don’t have to think about it. Here’s a set of easy rules/situations in which, like learning to dump your liquids out before going through airport security, you can get into a habit of doing and not have to think about it much.

  • Public/shared transportation.

    Riding a bus, train, metro, or a car with a stranger and especially with multiple strangers – these have high CO2 levels.

  • Airports, boarding a plane and during takeoff, and during descent/landing/deboarding the plane.

    This is when the CO2 levels are highest and the air exchanges/HEPA filtration is not running.

    Think of it like the seatbelt sign. You board the plane and put your seatbelt on, then eventually once you’ve reached cruising altitude the seatbelt sign goes off. If you’re standing in a line of people (to board or deplane) OR if the seatbelt sign is ON, that’s a huge ROI for wearing a high-quality (N95 or KN95) mask. When the seatbelt sign first turns off during the flight (or you hear the 10k-feet chime) and you want to take and leave it off, or take it off a while to eat or drink – that’s less risky during those times due to the HEPA filtration and air exchanges during flight. But when the seatbelt sign goes on for the plane’s final descent? The air quality is going down, too, so putting your seatbelt AND your mask back on is a higher ROI thing to do.

    (You do you inside the airport, too, but see below about density of people and temperature as a guide for whether you might want to consider situational masking in airports when you’re not eating/drinking.)

 

  • Conferences or meetings where you are sitting for more than a few minutes and there are many people in the room.

    Even with super big rooms and super high ceilings, so far every conference space I’ve presented in during the last several years has high CO2 levels even before the talk starts, and is even higher (>500-1000 ppm added) by the end of the session). If you’re not presenting or eating and drinking and are just sitting there listening and engaging in the session…it’s a low hassle opportunity to pop a high-quality mask on so you’re not breathing so much of the air around you from everyone else. When you’re done with the session and head out and want to socialize? Like leaving the plane, you’ll be around fewer people, and the CO2 levels (and risk) goes down. But sitting there quietly is a great time to wear a good mask and reduce your intake of other people’s exhalations.

 

You might find yourself in situations where the room feels hot and stuffy, or in the case of conferences and meetings, the air feels FREEZING cold. It runs freezing cold because the room gets hot and stuffy with so many people, indicating this space is not well ventilated, so they pump the AC to change the temperature. But that is a compensation for a too-low rate of air exchanges, and pre-cooling doesn’t prevent CO2 and aerosol buildup, so a room that either gets freezing cold or hot and stuffy should be a signal that the air quality likely isn’t ideal.

So a good rule of thumb is, if you’re in a space that feels hot and stuffy OR freezing cold, that’s an indicator that the air quality might be non-optimal. Consider masking in those situations even if you don’t have a CO2 monitor to evaluate the air.

It would be great if we could get 10x people to consider situational masking like this. Avoid the worst of the bad-news-bear-air of public and shared transportation and indoor spaces, which would cut down on a lot of transmission, even if people otherwise are still socializing and eating in indoor spaces and doing whatever it is they want to do. The choice to situationally mask might occasionally protect them but would also protect everyone around them in those situations when their exhalations have the greatest risk of doing the most damage.

A good way to think about it is at a conference. You might be willing to go to bars and socialize, but someone who is higher risk may be choosing not to attend those indoor dining scenarios. That’s fine: you each get to make your own choices! But when you go and sit down next to that person in a conference session, your choices then influence that person by every breath you take in that conference session.

That’s why situational masking – knowing that a situation is low-hassle to wear a high-quality mask (sitting quietly in a session) but high-risk (due to the poor air quality) means you have a high ROI to pull a mask out of your pocket/bag and throw it on – can help the people around you very effectively with little hassle and thought on your part.

You can get in the habit of masking in the bad-news-bear-air situations/locations, and you don’t have to think much about it. You’ll make things a bit safer for yourself and for the people around you, for far less hassle than avoiding buying a drink before you go through airport security because you know you need to dump liquids out.

Data-driven situational masking based on indoor air quality

How To Talk To Your Doctor About Your Enzyme (PERT) Dosing If You Have Exocrine Pancreatic Insufficiency (EPI or PEI or PI)

by Dana Lewis

In exocrine pancreatic insufficiency (EPI/PEI/PI), people are responsible for self-dosing their medication every time they eat something.

Doctors prescribe a starting dose, but a person with EPI determines each and every time they eat or drink something how many enzyme pills (of pancreatic enzyme replacement therapy, known as PERT) to take. Doctors often prescribe a low starting dose, and people have to try experimenting with multiple pills of the small size, and eventually work with their doctors to change their prescription to get a bigger pill size (so they can take fewer pills per meal) and the correct number of pills per day to match their needs.

For example, often people are prescribed one 10,000 unit pill per meal. The 10,000 units represents the amount of lipase (to help digest fat). There are also two other enzymes (protease, for protein digestion, and amylase, for carbohydrate digestion). They may be prescribed 1 pill per meal, which means 10,000 units of lipase per meal. But most dosing guidelines recommend starting at a dose of 40,000-50,000 units of lipase per meal (and people often need more), so it wouldn’t be surprising that someone prescribed one 10,000 pill per meal would need 4-5 pills of the 10,000 size pill PER MEAL, and times three meals per day (let alone any snacks), to get acceptable GI outcomes.

Mathematically, this means the initial prescription wouldn’t last long. The initial prescription for 1 pill per meal, with 3 meals a day, means 3 pills per day. 3 pills per day across a 30 day month is 90 pills. But when the pills per meal increase, that means the prescription won’t cover the entire month.

In fact, it would last a lot less than a month; closer to one week!

Showing that based on the number of pills and 3 meals per day, an intitial RX of 10,000 size pills may last more like a week rather than a full 30 days when the doctor is unaware of prescribing guidlines that typically suggest 40,000-50,000 per meal is needed as the starting meal dose.

Let’s repeat: with a too-small prescription pill size (e.g. 10,000 starting dose size) and count (e.g. 3 pills per day to cover 1 per meal) and with a person with EPI titrating themselves up to the starting dose guidelines in all of the medical literature, they would run out of their prescription WITHIN ONE WEEK. 

So. If you have EPI, you need to be prepared to adjust your dosing yourself; but you also need to be ready to reach out to your doctor and talk about your need for more enzymes and a changed prescription.

PERT (enzymes) come in different sizes, so one option is to ask for a bigger pill size and/or a different amount (count) per meal/day. Depending on the brand and the number of pills you need per meal, it could be simply going up to a bigger pill size. For example, if you need 3 pills of the 10,000 PERT size, you could move to a 36,000 pill size and take one per meal. If you find yourself taking 5 pills of the 10,000 PERT size, that might mean 2 pills of the 25,000 size. (Brands differ slightly, e.g. one might be 24,000 instead of 25,000, so the math may work out slightly differently depending on which brand you’re taking.)

Don’t be surprised if you need to do this within a week or two of starting PERT. In fact, based on the math above, especially if you’re on a much lower dose than starting guidelines (e.g. 40,000-50,000 units of lipase per meal), you should expect within a few days to need an updated prescription to make sure that you don’t run out of PERT.

If you do find yourself running out of PERT before you can get your prescription updated, there is an alternative you can consider: either substituting or adding on over the counter enzymes. The downsides include the fact that insurance doesn’t cover them so you would be paying out of pocket; plus there are no studies with these so you can’t (shouldn’t) rely on these as full 1:1 substitutes for prescription PERT without careful personal testing that you can do so. That being said, there is anecdotal evidence (from me, as well as hundreds of other people I’ve seen in community groups) that it is possible to use OTC enzymes if you can’t afford or can’t get a PERT prescription; or if you need to “top off”/supplement/add to your PERT because your prescription won’t last a full month and you can’t get a hold of your doctor or they won’t update your prescription.

For me, I generally evaluate the units of lipase (e.g. this kind is 17,000 units of lipase per pill) but then factor in for the lack of reliability for OTC and really treat it like it contains 13-15,000 units of lipase when choosing to take it. Similarly for another lipase-only OTC option (that has ~6,000 units per pill), I assume it acts like it only has ~5,000 units. Unlike insulin, there is little downside to taking a little too much of enzymes; but there is a LOT of downside to not taking enough, so my personal approach is that if in doubt, or on the fence, to round up (especially with OTC pills, which cost somewhere between $0.08/pill (lipase-only) to $0.34/pill (for the larger and multiple enzyme pill)).

So how do you talk to your doctor about needing more PERT?

It helps if you bring data and evidence to the conversation, especially if your doctor thinks by default that you don’t need more than what they initially prescribed. You can bring your personal data (more on that below and how to collect and present that), but you can also cite relevant medical literature to show if your dose is below standard starting guidelines.

Below I’ve shared a series of citations that show that the typical starting dose for people with EPI should be around 40,000-50,000 units of lipase per meal.

Important note that this is the STARTING DOSE SIZE, and most of these recommend further increasing of dose to 2-3 times this amount as needed. Depending on the starting dose size, you can see the chart I built below that illustrates with examples exactly how much this means one might need to increase to. Not everyone will need the upper end of the numbers, but if a doctor starts someone on 10,000 and doesn’t want to get them up to 40,000 (the lower end of starting doses) or go beyond 40,000 because it’s the starting dose, I’ve found this chart useful to show that numerically the range is a lot larger than we might assume.

Example of Titrating According to Common Dose Guidelines, Before Adding PPI

Examples of PERT starting doses of 25,000, 40,000, and 50,000 (plus half that for snacks) and what the dose would be if increased according to guidelines to 2x and 3x, plus the sum of the total daily dose needed at those levels.

Here are some citations that back up my point about 40,000-50,000 units of lipase being the typically recommended starting dose, including across different conditions (e.g. regardless of whether you have EPI + any of (chronic pancreatitis, diabetes, celiac, etc)).

  • Shandro et al, 2020, the median starting dose of 50,000 units per lipase “is an appropriate starting dose”, also citing UEG 2017 guidelines.
  • Forsmark et al, 2020, defined appropriate dose of PERT as >=120,000 units of lipase per day (e.g. 40,000 units of lipase per meal).
  • Whitcomb et al, 2022, in a joint American Gastroenterology Association and PancreasFest symposium paper, concur on 40,000 units as a starting dose and that “This dose should be titrated up as needed to reduce steatorrhea or gastrointestinal symptoms of maldigestion “
  • 2021 UK guidelines for EPI management suggest 50,000 units as the starting dose and emphasize that “all guidelines endorse dose escalation if the initial dose is not effective”

There are also many guidelines and research specific for EPI and different co-conditions supporting the ballpark of 40-50,000 units of lipase starting dose:

It is also worth noting that these guidelines also point out that after titrating 2-3x above the starting dose, PPI (proton pump inhibitors, to suppress acid) should be added if gastrointestinal symptoms are still not resolved. Anecdotally, it seems a lot of doctors are not aware that PPIs should be added if 3x the starting dose is not effective, so make sure to bring this up as well.

How to Share Your Personal PERT Data To Show How Much You Need

In addition to pointing out the guidelines (based on the above), it’s useful to share your data to show what you’ve been taking (dosing) and how it’s been working. I’ve written a lot about how you can do this manually, but I also recently created an iOS based app to make it easier to track what you’re eating, what you’re dosing in terms of PERT/enzymes, and what the outcome is. This app, PERT Pilot, is free to use, and it also enables you to visualize on a graph the relationship between what you’re eating and dosing.

PERT Pilot lets you track how many grams of fat each pill of your current prescription has been used for, so you can see with red and green coloring the relationship between meals that you’ve had symptoms after (in red) vs. when you recorded no symptoms (green). If you have a “convergence zone” of green and red in the same area, that may help you decide to change your ratio (e.g. dose more) around that amount, until you can comfortably and repeatedly get green results (no symptoms when you eat).

How you might use this to talk to your doctor

You can take a screenshot of your PERT Pilot graph and share it with your doctor to show them how many grams of fat your prescription size (e.g. pill size) effectively “covers” for you, and how many meals that you’ve tested it with.

Meals based on the ratio of fat:lipase and protein:protease mapped with color coded dots where green means no symptoms, orange means not sure if symptoms, and red means symptoms occurred and the dose likely didn't work at that ratio.For example, I was initially prescribed an enzyme dose that was one pill per meal (and no snacks), so I had 3 pills per day. But I quickly found myself needing two pills per meal, based on what I was typically eating. I summarized my data to my doctor, saying that I found one pill typically covered up to ~30 grams of fat per meal, but most of my meals were >30 grams of fat, so that I wanted to update my prescription to have an average of 2 pills per meal of this prescription size. I also wanted to be able to eat snacks, so I asked for 2 pills per meal, 1 per snack, which meant that my prescription increased to 8 pills per day (of the same size), to cover 2 pills x 3 meals a day (=6) plus up to 2 snacks (=2). I also had weeks of data to show that my average meal was >30 grams of fat to confirm that I need more than the amount of lipase I was originally prescribed. My doctor was happy to increase my prescription as a result, and this is what I’ve been using successfully for over a year ever since.

So in summary, the data that would be useful to share is:

  • How much one pill ‘covers’ (which is where the PERT Pilot graph can be used)
  • How many pills per meal you’ve been taking and how big your meals typically are
  • Whether you are struggling with the number of pills per meal: if so, ask whether there’s a larger pill size in your current brand that you could increase to, in order to reduce the number of pills per meal (and/or snack) you need to take every time

If you are told that you shouldn’t need “that much”, remember the above section and have those resources ready to discuss that the starting dose is often 40,000-50,000 per meal and that the guidelines say to titrate up to 3x that before adding PPI. Therefore, it would be expected for some people to need upwards of 600,000 units of lipase per day (50,000 starting dose, increased 3x per meal and half of the dose used per snack). Depending on what people eat, this could be even higher (because not everyone eats the same size meal and snack and many of us adjust dose based on what we eat).

Also, it is worth noting that the dosing guidelines never mention the elastase levels or severity of EPI: so PERT prescriptions should not be based on whether you have “moderate” or “severe” EPI and what your elastase level is (e.g. whether it’s 45 or 102 or 146 or even 200, right on the line of EPI – all of those elastase levels would still get the same starting dose of PERT, based on the clinical guidelines for EPI).

It is common and you are not alone if you’ve not been giving the starting dose of PERT that the guidelines recommend.

There are numerous studies showing most people with EPI are initially underdosed/underprescribed PERT. For example, in 2020 Forsmark et al reported that only 8.5% of people with chronic pancreatitis and EPI received an adequate prescription for PERT: and only 5.5% of people with pancreatic cancer and EPI received an adequate prescription dose of PERT. Other studies in chronic pancreatitis and EPI from 2014, 2016, and 2020 report that undertreatment often occurs in EPI and CP; and I’ve found studies in other conditions as well showing undertreatment compared to guidelines, although it’s most studied in CP and cancer (which is true of all types of EPI-related research, despite the prevalence in many other conditions like diabetes, celiac, etc.).

You may need to advocate for yourself, but know that you’re not alone. Again, feel free to comment or email privately (Dana@OpenAPS.org) if you need help finding research for another co-condition and EPI that I haven’t mentioned here.

—

PS – if you haven’t seen it, I have other posts about EPI at DIYPS.org/EPI

—
You can also contribute to a research study and help us learn more about EPI/PEI – take this anonymous survey to share your experiences with EPI-related symptoms!

—

How I Use LLMs like ChatGPT And Tips For Getting Started

by Dana Lewis

You’ve probably heard about new AI (artificial intelligence) tools like ChatGPT, Bard, Midjourney, DALL-E and others. But, what are they good for?

Last fall I started experimenting with them. I looked at AI art tools and found them to be challenging, at the time, for one of my purposes, which was creating characters and illustrating a storyline with consistent characters for some of my children’s books. I also tested GPT-3 (meaning version 3.0 of GPT). It wasn’t that great, to be honest. But later, GPT-3.5 was released, along with the ChatGPT chat interface to it, which WAS a big improvement for a lot of my use cases. (And now, GPT-4 is out and is an even bigger improvement, although it costs more to use. More on the cost differences below)

So what am I using these AI tools for? And how might YOU use some of these AI tools? And what are the limitations? This is what I’ve learned:

  1. The most frequent way I use these AI tools is for getting started on a project, especially those related to writing.

You know the feeling of staring at a blank page and not knowing where to start? Maybe it’s the blank page of a cold email; the blank page of an essay or paper you need to write; the blank page of the outline for a presentation. Starting is hard!

Even for this blog post, I had a list of bulleted notes of things I wanted to remember to include. But I wasn’t sure how I wanted to start the blog post or incorporate them. I stuck the notes in ChatGPT and asked it to expand the notes.

What did it do? It wrote a few paragraph summary. Which isn’t what I wanted, so I asked it again to use the notes and this time “expand each bullet into a few sentences, rather than summarizing”. With these clear directions, it did, and I was able to look at this content and decide what I wanted to edit, include, or remove.

Sometimes I’m stuck on a particular writing task, and I use ChatGPT to break it down. In addition to kick-starting any type of writing overall, I’ve asked it to:

  • Take an outline of notes and summarize them into an introduction; limitations section; discussion section; conclusion; one paragraph summary; etc.
  • Take a bullet point list of notes and write full, complete sentences.
  • Take a long list of notes I’ve written about data I’ve extracted from a systematic review I was working on, and ask it about recurring themes or outlier concepts. Especially when I had 20 pages (!) of hand-written notes in bullets with some loose organization by section, I could feed in chunks of content and get help getting the big picture from that 20 pages of content I had created. It can highlight themes in the data based on the written narratives around the data.

A lot of times, the best thing it does is it prompts my brain to say “that’s not correct! It should be talking about…” and I’m able to more easily write the content that was in the back of my brain all along. I probably use 5% of what it’s written, and more frequently use it as a springboard for my writing. That might be unique to how I’m using it, though, and other simple use cases such as writing an email to someone or other simplistic content tasks may mean you can keep 90% or more of the content to use.

2. It can also help analyze data (caution alert!) if you understand how the tools work.

Huge learning moment here: these tools are called LLMs (large language models). They are trained on large amounts of language. They’re essentially designed so that, based on all of those words (language) it’s taken in previously, to predict content that “sounds” like what would come after a given prompt. So if you ask it to write a song or a haiku, it “knows” what a song or a haiku “looks” like, and can generate words to match those patterns.

It’s essentially a PATTERN MATCHER on WORDS. Yeah, I’m yelling in all caps here because this is the biggest confusion I see. ChatGPT or most of these LLMs don’t have access to the internet; they’re not looking up in a search engine for an answer. If you ask it a question about a person, it’s going to give you an answer (because it knows what this type of answer “sounds” like), but depending on the amount of information it “remembers”, some may be accurate and some may be 100% made up.

Why am I explaining this? Remember the above section where I highlighted how it can start to sense themes in the data? It’s not answering solely based on the raw data; it’s not doing analysis of the data, but mostly of the words surrounding the data. For example, you can paste in data (from a spreadsheet) and ask it questions. I did that once, pasting in some data from a pivot table and asking it the same question I had asked myself in analyzing the data. It gave me the same sense of the data that I had based on my own analysis, then pointed out it was only qualitative analysis and that I should also do quantitative statistical analysis. So I asked it if it could do quantitative statistical analysis. It said yes, it could, and spit out some numbers and described the methods of quantitative statistical analysis.

But here’s the thing: those numbers were completely made up!

It can’t actually use (in its current design) the methods it was describing verbally, and instead made up numbers that ‘sounded’ right.

So I asked it to describe how to do that statistical method in Google Sheets. It provided the formula and instructions; I did that analysis myself; and confirmed that the numbers it had given me were 100% made up.

The takeaway here is: it outright said it could do a thing (quantitative statistical analysis) that it can’t do. It’s like a human in some regards: some humans will lie or fudge and make stuff up when you talk to them. It’s helpful to be aware and query whether someone has relevant expertise, what their motivations are, etc. in determining whether or not to use their advice/input on something. The same should go for these AI tools! Knowing this is an LLM and it’s going to pattern match on language helps you pinpoint when it’s going to be prone to making stuff up. Humans are especially likely to make something up that sounds plausible in situations where they’re “expected” to know the answer. LLMs are in that situation all the time: sometimes they actually do know an answer, sometimes they have a good guess, and sometimes they’re just pattern matching and coming up with something that sounds plausible.

In short:

  • LLM’s can expand general concepts and write language about what is generally well known based on its training data.
  • Try to ask it a particular fact, though, and it’s probably going to make stuff up, whether that’s about a person or a concept – you need to fact check it elsewhere.
  • It can’t do math!

But what it can do is teach you or show you how to do the math, the coding, or whatever thing you wish it would do for you. And this gets into one of my favorite use cases for it.

3. You can get an LLM to teach you how to use new tools, solve problems, and lower the barrier to entry (and friction) on using new tools, languages, and software.

One of the first things I did was ask ChatGPT to help me write a script. In fact, that’s what I did to expedite the process of finding tweets where I had used an image in order to get a screenshot to embed on my blog, rather than embedding the tweet.

It’s now so easy to generate code for scripts, regardless of which language you have previous experience with. I used to write all of my code as bash scripts, because that’s the format I was most familiar with. But ChatGPT likes to do things as Python scripts, so I asked it simple questions like “how do I call a python script from the command line” after I asked it to write a script and it generated a python script. Sure, you could search in a search engine or Stack Overflow for similar questions and get the same information. But one nice thing is that if you have it generate a script and then ask it step by step how to run a script, it gives you step by step instructions in context of what you were doing. So instead of saying “to run a script, type `python script.py’”, using placeholder names, it’ll say “to run the script, use ‘python actual-name-of-the-script-it-built-you.py’ “ and you can click the button to copy that, paste it in, and hit enter. It saves a lot of time for figuring out how to take placeholder information (which you would get from a traditional search engine result or Stack Overflow, where people are fond of things like saying FOOBAR and you have no idea if that means something or is meant to be a placeholder). Careful observers will notice that the latest scripts I’ve added to my Open Humans Data Tools repository (which is packed with a bunch of scripts to help work with big datasets!) are now in Python rather than bash; such as when I was adding new scripts for fellow researchers looking to check for updates in big datasets (such as the OpenAPS Data Commons). This is because I used GPT to help with those scripts!

It’s really easy now to go from an idea to a script. If you’re able to describe it logically, you can ask it to write a script, tell you how to run it, and help you debug it. Sometimes you can start by asking it a question, such as “Is it possible to do Y?” and it describes a method. You need to test the method or check for it elsewhere, but things like uploading a list of DOIs to Mendeley to save me hundreds of clicks? I didn’t realize Mendeley had an API or that I could write a script that would do that! ChatGPT helped me write the script, figure out how to create a developer account and app access information for Mendeley, and debug along the way so I ended up within an hour and a half of having a tool that easily saved me 3 hours on the very first project that I used it with.

I’m gushing about this because there’s probably a lot of ideas you have that you immediately throw out as being too hard, or you don’t know how to do it. It takes time, but I’m learning to remember to think “I should ask the LLM this” and ask it questions such as:

  • Is it possible to do X?
  • Write a script to do X.
  • I have X data. Pretend I am someone who doesn’t know how to use Y software and explain how I should do Z.

Another thing I’ve done frequently is ask it to help me quickly write a complex formula to use in a spreadsheet. Such as “write a formula that can be used in Google Sheets to take an average of the values in M3:M84 if they are greater than zero”.

It gives me the formula, and also describes it, and in some cases, gives alternative options.

Other things I’ve done with spreadsheets include:

  • Ask it to write a conditional formatting custom formula, then give me instructions for expanding the conditional formatting to apply to a certain cell range.
  • Asking it to check if a cell is filled with a particular value and then repeating the value in the new cell, in order to create new data series to use in particular charts and graphs I wanted to create from my data.
  • Help me transform my data so I could generate a box and whisker plot.
  • Ask it for other visuals that might be effective ways to illustrate and visualize the same dataset.
  • Explain the difference between two similar formulas (e.g. COUNT and COUNTA or when to use IF and IFS).

This has been incredibly helpful especially with some of my self-tracked datasets (particularly around thyroid-related symptom data) where I’m still trying to figure out the relationship between thyroid levels, thyroid antibody levels, and symptom data (and things like menstrual cycle timing). I’ve used it for creating the formulas and solutions I’ve talked about in projects such as the one where I created a “today” line that dynamically updates in a chart.

It’s also helped me get past the friction of setting up new tools. Case in point, Jupyter notebooks. I’ve used them in the web browser version before, but often had issues running the notebooks people gave me. I debugged and did all kinds of troubleshooting, but have not for years been able to get it successfully installed locally on (multiple of) my computers. I had finally given up on effectively using notebooks and definitely given up on running it locally on my machine.

However, I decided to see if I could get ChatGPT to coax me through the install process.

I told it:

“I have this table with data. Pretend I am someone who has never used R before. Tell me, step by step, how to use a Jupyter notebook to generate a box and whisker plot using this data”

(and I pasted my data that I had copied from a spreadsheet, then hit enter).

It outlined exactly what I needed to do, saying to install Jupyter Notebook locally if I hadn’t, gave me code to do that, installing the R kernel, told me how to do that, then how to start a notebook all the way down to what code to put in the notebook, the data transformed that I could copy/paste, and all the code that generated the plot.

However, remember I have never been able to successfully get Jupyter Notebooks running! For years! I was stuck on step 2, installing R. I said:

“Step 2, explain to me how I enter those commands in R? Do I do this in Terminal?”

It said “Oh apologies, no, you run those commands elsewhere, preferably in Rstudio. Here is how to download RStudio and run the commands”.

So, like humans often do, it glossed over a crucial step. But it went back and explained it to me and kept giving more detailed instructions and helping me debug various errors. After 5-6 more troubleshooting steps, it worked! And I was able to open Jupyter Notebooks locally and get it working!

All along, most of the tutorials I had been reading had skipped or glossed over that I needed to do something with R, and where that was. Probably because most people writing the tutorials are already data scientists who have worked with R and RStudio etc, so they didn’t know those dependencies were baked in! Using ChatGPT helped me be able to put in every error message or every place I got stuck, and it coached me through each spot (with no judgment or impatience). It was great!

I was then able to continue with the other steps of getting my data transformed, into the notebook, running the code, and generating my first ever box and whisker plot with R!

A box and whisker plot, illustrated simply to show that I used R and Jupyter finally successfully!

This is where I really saw the power of these tools, reducing the friction of trying something new (a tool, a piece of software, a new method, a new language, etc.) and helping you troubleshoot patiently step by step.

Does it sometimes skip steps or give you solutions that don’t work? Yes. But it’s still a LOT faster than manually debugging, trying to find someone to help, or spending hours in a search engine or Stack Overflow trying to translate generic code/advice/solutions into something that works on your setup. The beauty of these tools is you can simply paste in the error message and it goes “oh, sorry, try this to solve that error”.

Because the barrier to entry is so low (compared to before), I’ve also asked it to help me with other project ideas where I previously didn’t want to spend the time needed to learn new software and languages and all the nuances of getting from start to end of a project.

Such as, building an iOS app by myself.

I have a ton of projects where I want to temporarily track certain types of data for a short period of time. My fall back is usually a spreadsheet on my phone, but it’s not always easy to quickly enter data on a spreadsheet on your phone, even if you set up a template with a drop down menu like I’ve done in the past (for my DIY macronutrient tool, for example). For example, I want to see if there’s a correlation in my blood pressure at different times and patterns of inflammation in my eyelid and heart rate symptoms (which are symptoms, for me, of thyroid antibodies being out of range, due to Graves’ disease). That means I need to track my symptom data, but also now some blood pressure data. I want to be able to put these datasets together easily, which I can, but the hardest part (so to speak) is finding a way that I am willing to record my blood pressure data. I don’t want to use an existing BP tracking app, and I don’t want a connected BP monitor, and I don’t want to use Apple Health. (Yes, I’m picky!)

I decided to ask ChatGPT to help me accomplish this. I told it:

“You’re an AI programming assistant. Help me write a basic iOS app using Swift UI. The goal is a simple blood pressure tracking app. I want the user interface to default to the data entry screen where there should be three boxes to take the systolic, diastolic blood pressure numbers and also the pulse. There should also be selection boxes to indicate whether the BP was taken sitting up or laying down. Also, enable the selection of a section of symptom check boxes that include “HR feeling” and “Eyes”. Once entered on this screen, the data should save to a google spreadsheet.” 

This is a completely custom, DIY, n of 1 app. I don’t care about it working for anyone else, I simply want to be able to enter my blood pressure, pulse, whether I’m sitting or laying down, and the two specific, unique to me symptoms I’m trying to analyze alongside the BP data.

And it helped me build this! It taught me how to set up a new SwiftUI project in XCode, gave me code for the user interface, how to set up an API with Google Sheets, write code to save the data to Sheets, and get the app to run.

(I am still debugging the connection to Google Sheets, so in the interim I changed my mind and had it create another screen to display the stored data then enable it to email me a CSV file, because it’s so easy to write scripts or formulas to take data from two sources and append it together!)

Is it fancy? No. Am I going to try to distribute it? No. It’s meeting a custom need to enable me to collect specific data super easily over a short period of time in a way that my previous tools did not enable.

Here’s a preview of my custom app running in a simulator phone:

Simulator iphone with a basic iOS app that intakes BP, pulse, buttons for indicating whether BP was taken sitting or laying down; and toggles for key symptoms (in my case HR feeling or eyes), and a purple save button.

I did this in a few hours, rather than taking days or weeks. And now, the barrier to entry to creating more custom iOS is reduced, because now I’m more comfortable working with XCode and the file structures and what it takes to build and deploy an app! Sure, again, I could have learned to do this in other ways, but the learning curve is drastically shortened and it takes away most of the ‘getting started’ friction.

That’s the theme across all of these projects:

  • Barriers to entry are lower and it’s easier to get started
  • It’s easier to try things, even if they flop
  • There’s a quicker learning curve on new tools, technologies and languages
  • You get customized support and troubleshooting without having to translate through as many generic placeholders

PS – speaking of iOS apps, based on building this one simple app I had the confidence to try building a really complex, novel app that has never existed in the world before! It’s for people with exocrine pancreatic insufficiency like me who want to log pancreatic enzyme replacement therapy (PERT) dosing and improve their outcomes – check out PERT Pilot and how I built it here.

4. Notes about what these tools cost

I found ChatGPT useful for writing projects in terms of getting started, even though the content wasn’t that great (on GPT-3.5, too). Then they came out with GPT-4 and made a ChatGPT Pro option for $20/month. I didn’t think it was worth it and resisted it. Then I finally decided to try it, because some of the more sophisticated use cases I wanted to use it for required a longer context window, and in addition to a better model it also gave you a longer context window. I paid the first $20 assuming I’d want to cancel it by the end of the month.

Nope.

The $20 has been worth it on every single project that I’ve used it for. I’ve easily saved 5x that on most projects in terms of reducing the energy needed to start a project, whether it was writing or developing code. It has saved 10x that in time cost recouped from debugging new code and tools.

GPT-4 does have caps, though, so even with the $20/month, you can only do 25 messages every 3 hours. I try to be cognizant of which projects I default to using GPT-3.5 on (unlimited) versus saving the more sophisticated projects for my GPT-4 quota.

For example, I saw a new tool someone had built called “AutoResearcher”, downloaded it, and tried to use it. I ran into a bug and pasted the error into GPT-3.5 and got help figuring out where the problem was. Then I decided I wanted to add a feature to output to a text file, and it helped me quickly edit the code to do that, and I PR’ed it back in and it was accepted (woohoo) and now everyone using that tool can use that feature. That was pretty simple and I was able to use GPT-3.5 for that. But sometimes, when I need a larger context window for a more sophisticated or content-heavy project, I start with GPT-4. When I run into the cap, it tells me when my next window opens up (3 hours after I started using it), and I usually have an hour or two until then. I can open a new chat on GPT-3.5 (without the same context) and try to do things there; switch to another project; or come back at the time it says to continue using GPT-4 on that context/setup.

Why the limit? Because it’s a more expensive model. So you have a tradeoff between paying more and having a limit on how much you can use it, because of the cost to the company.

—–

TLDR:

Most important note: LLMs don’t “think” or “know” things the way humans do. They output language they predict you want to see, based on its training and the inputs you give it. It’s like the autocomplete of a sentence in your email, but more words on a wider range of topics!

Also, the LLM can’t do math. But they can write code. Including code to do math.

(Some, but not all, LLMs have access to the internet to look up or incorporate facts; make sure you know which LLM you are using and whether it has this feature or not.)

Ways to get started:

    1. The most frequent way I use these AI tools is for getting started on a project, especially those related to writing.
      • Ask it to help you expand on notes; write summaries of existing content; or write sections of content based on instructions you give it
    2.  It can also help analyze data (caution alert!) if you understand the limitations of the LLM.
      • The most effective way to work with data is to have it tell you how to run things in analytical software, whether that’s how to use R or a spreadsheet or other software for data analysis. Remember the LLM can’t do math, but it can write code so you can then do the math!
    3.  You can get an LLM to teach you how to use new tools, solve problems, and lower the barrier to entry (and friction) on using new tools, languages, and software.
      • Build a new habit of asking it “Can I do X” or “Is it possible to do Y” and when it says it’s possible, give it a try! Tell it to give you step-by-step instructions. Tell it where you get stuck. Give it your error messages or where you get lost and have it coach you through the process. 

—

What’s been your favorite way to use an LLM? I’d love to know other ways I should be using them, so please drop a comment with your favorite projects/ways of using them!

Personally, the latest project that I built with an LLM has been PERT Pilot!

How I use LLMs (like ChatGPT) and tips for getting started