What you should know about closed looping (DIY like #OpenAPS or otherwise)

I’ve been wearing a DIY closed loop for something like 979 days..which means something like ~20,000 hours with this technology. Additionally, I’m not the only one. There are (n=1)*369+ (and that’s an undercount just based on who’s told us they’re looping) other DIYers out there, so the community has an estimated 1,800,000+ hours of cumulative experience, too.

Suffice to say, we’ve all learned a lot about this technology and how hybrid closed loop makes a difference in life with diabetes.

I previously gave a talk almost two years ago to the Sports & Diabetes Group Northwest here in Seattle, talking about #DIYPS, how we closed the loop, and #OpenAPS. (And you can see a recent TEDX talk I gave on OpenAPS here.) That was a springboard for meeting some awesome individuals who became very early DIY loopers in the Seattle area. And one of them (who also wore a pancreas at HIS wedding :)) had suggested we do another talk for SDGNW to update on some of what we have learned since then. But unfortunately, he got called out of town for work and couldn’t join me for presenting, so I went solo (ish, because Scott also came and contributed). I used a new analogy, because I think there’s a lot to think about before choosing and using closed loop technology, whether it’s DIY or commercial, and wanted to write it up for sharing here.


First, some reminders for those familiar and some context for those who are not close to this technology. We’re talking about a hybrid closed loop, which is what I’m referring to when I say “artificial pancreas” or “AP” here. This type of technology makes small adjustments every few minutes to provide more or less insulin with the goal of keeping blood glucose (BG) levels in range. It’s complicated by the fact that insulin often peaks at 60-90 minutes…but food hits in ~15 minutes. So there’s often “catch up” being done with insulin to deal with food eaten previously, and also with hormones and other things that impact BGs that aren’t measurable. (This is also why it’s called hybrid, because for best outcomes people will still be doing some kind of meal announcement/bolus to deal with insulin timing.) As a result, even with pumps and CGMs, diabetes is still hard. A closed loop can do the needed math every five minutes, doesn’t go to sleep, and is very precise. It can respond more quickly (because it’s paying attention) than a human will in most situations, because we’re out living our lives/working/sleeping and not paying attention ONLY to diabetes. It’s not a cure, but it helps make living with diabetes better than it used to be.

However, I equate it to being a pilot who has seen technology on planes evolve to include “autopilot”. Even with hybrid closed loop technology, we’re still flying the “plane”.


Here’s what I mean. There are stages for picking out and deciding to use the technology; preparing to use it/getting in the mode where you CAN use it; using it successfully; getting ready for the times when you can’t use it; and smoothing the way for the next time you use it.

It’s not perfect 24/7, you see, because we’re still using pump sites and continuous glucose monitor (CGM) sensors. The CGM sensor may last for 7 days, but then you have to change it out (or cough restart it cough), and you have a gap in data, which means you can’t loop. So you have this type of cycle regularly, and here’s what you need to know about each of these stages, regardless of whether we’re talking about DIY (like OpenAPS) or a commercial closed loop solution.

Preparing for takeoff

prepare_for_looping_danamlewisWhen you’re getting into the plane, you have a flight plan. You know when you will and won’t use the technology on board. Same for diabetes & closed looping. Make sure to think about the following for your tech of choice:

When will your loop work? When does it not? What happens if it breaks? What are your back up tools? How do you operate it: what happens if your sensor loses data, or you don’t calibrate? How does the algorithm work? What will it target your BG to be? What behaviors will you have to do (meal bolus or announcement, etc.) and how can you alter those to optimize performance? Also, what are the warning signs of failure to let you know when you need to take additional action with corrective insulin or eating carbs?

Taking off and the new technology learning curve

taking_off_learning_curve_danamlewisJust like switching from MDI pump (or even iPhone to Android and vice versa), you have a learning curve. When you go into looping or automated insulin delivery mode, you have to figure things out. You need to be able to figure out what’s happening and why it’s doing what it’s doing, so if you’re not happy with what’s happening, you can make a change. Why are you running high? Why are you running low? Knowing why it’s doing what it’s doing is critical for adjusting – either tweaking the closed loop settings, if you can, or adjusting your own behavior. Especially in the first few cycles of new tech, you’ll have a lot of learning around “I used to do things like X, but now I need to do them like Y.”

Why you might not be taking off and able to loop

blocking_takeoff_danamlewisYou also need to know why you can’t loop. There are three major categories of things that will prevent you from looping:

  1. No sensor, no looping.
  2. In some systems, wonky or missing data, no looping
  3. Communication errors between pieces of a system.

Some of these are obvious fixes (put in a new sensor if one fell out, or decide to put in a new sensor if the old one is bad), but depending on the system may involve some troubleshooting to get things going again.

Also, some of the commercial systems will kick you out of looping for various reasons (including lack of calibration), in addition to preventing you from looping in the first place without them, so knowing what these basic things are required for looping is useful to make sure you CAN automate.

Flying high: maintenance when you’re actually looping

maintenance_when_looping_danamlewisThere are some critical behaviors required for looping. (After all, when flying, there’s always a pilot present in the cockpit..right?!)

Some of these are basic behaviors you’ll be used to if you’ve been wearing a pump and CGM previously: keeping pump sites changed so the insulin works, and changing and calibrating CGM sensors.

HOWEVER – many people who “stretch” their CGM sensors find that they don’t want to stretch their sensors as far, as the data degrades over time. You do you, but keep in mind this might change when you’re looping vs. not, because you’re relying on good data to operate the system.

That being said, in addition to good sensor life, calibration hygiene is critical. You don’t want to loop off of wonky data, but also some commercial systems will kick you out if your calibration is way off and/or if you miss a calibration. (Personal opinion on this is a big ugh, which is why no DIY system that I know of does this.)

But if you keep your sites and sensors in good condition, this is where life is good. You’re looping! It’s microadjusting and helping keep things in range. Yay! This means better sleep, more time in range, and feeling better all around.

However, you still have diabetes, you’re still in the plane, so you still need to keep an eye on things. Monitoring the system is important (to make sure you’re still in autopilot and don’t need to actually fly the plane manually), so make sure you know how you (and your loved ones) can monitor the system’s operation, and know what your backup alarms are in case of system failures.

Note: there are approximately eleventy bajillion ways to remote monitor in DIY systems, but even if you have a commercial system that comes pre-baked without remote monitoring… you can add a DIY solution for that. So don’t feel like if you have a commercial AP that you can never use anything DIY – you can totally mix and match!

Dealing with turbulence

turbulence_danamlewisWhat kind of airplane/flight analogy would this be without including turbulence? :)

Like the things that can prevent looping in the first place, there are things that can throw off your looping. I already mentioned wonky sensor data that may mean either a blip in your looping time, or may kick you off looping. Again, your sensor life and your calibration practices will likely change.

But the other big disturbance, so to speak, is around body sensitivity changes. You know all the ways it can happen: you’re getting sick, recovering from getting sick, getting ready for/or are on/or are right after your period, or have an adrenaline spike, or have hormones surging, or have a growth spurt, or just exercised, etc.

This is what makes diabetes oh so hard so often. But this is where different closed loop systems can help, so this is one area you should ask about when picking a system: how does it adjust and adapt to sensitivity changes, and on what time frame? (In the DIY world, we use a number of techniques with this, ranging from autosensitivity to adapt on a 24 hour rolling scale of sensitivity changes, as well as using autotune to track bigger picture trends and changes needed to underlying settings. Reminder – anyone can use autotune if they’re willing to log bolus & carb data in Nightscout, not just closed loopers, so check that out if you’re interested! All DIY closed loop systems also use dynamic carbohydrate absorption in their respective algorithms, so that if you have slowed digestion for ANY reason, ranging from gastroparesis to getting glutened if you have celiac to merely walking after a meal, the system takes that into account and adjusts accordingly.)

The other things that can help you tough out some turbulence? Setting different modes, like an activity mode for exercise. The two things to know about exercise are:

  1. You don’t want to go into exercise with a bucket of IOB, so set activity mode WELL BEFORE you go out for activity. Depending on how much netIOB you have, that time may vary, but planning ahead with an activity mode makes a big difference for not going low during activity – even with a closed loop.
  2. Your sensitivity may be impacted for hours afterward, into the next day. See above about having a system that can respond to sensitivity changes like that, but also think about having multiple targets you can use temporarily (if your system allows it) so you can give the system a bigger buffer while it sorts out your body’s sensitivity changes.

Preparing for landing and making time between loops more smooth

prepare_for_landing_danamlewisJust like you’ll want to plan to go on the closed loop, you’ll want to plan for how to cycle off and then back on again. Depending on your system, there may be things you can do to smooth things out. One of the things I do is pre-soak a CGM sensor to skip the first day jumpy numbers. That makes a big difference for the first hours back on a “new” looping session. The other thing I do is stagger receiver start times (where I have two receivers that I stop/start at different times, so I’m not stuck for two hours without BG data to loop on).

But even if you can’t do that, you can do some other general planning ahead – like making sure your looping session doesn’t end in the middle of a big meal that’s being digested, or overnight. Those are the times when you’ll want to be looping the most.

Landing and preparing for the next looping session

Landing_danamlewisJust like learning to fly where you take a lot of training flights and review how the flight went, you’ll want to think about how things went and what you might change behavior-wise for your next looping session. Some of the things that may change over time as you learn more about your tech of choice:

  • Timing of meal announcement or boluses
  • Precision (if needed, or otherwise lack thereof) around carb counting
  • Using things like “eating soon” mode to optimize meal-time insulin effectiveness and reduce post-meal spikes
  • Using different activity patterns and targets to get ideal outcomes around exercise
  • Tweaking underlying settings (if you can)

General thoughts on looping

general_looping_reminders_danamlewisSome last thoughts about closed looping in general, regardless of the tech you might choose now or in the future:

  1. Picking one kind of technology does NOT lock you into it forever. If you’re DIYing now, you can choose commercial later. If you start on a commercial system, you can still try a DIY system.
  2. Don’t compare the original iPhone with an iPhone 6. Let’s be blunt: the Dexcom 7plus is a different beast than the Dexcom G4/G5. Similarly, Medtronic’s original “harpoon” sensor is different than their newest sensor tech. The Abbott Navigator is different than their Libre. Don’t be held up by perceptions of the old tech – make sure to check out the new stuff with a somewhat open mind.
  3. (Similarly, hopefully, in the future we’ll get to say the same about first-generation devices and algorithms. These things in commercial systems should change over time in terms of algorithm capabilities, targets, features, and usability. They certainly have in DIY – we’ve gotten smaller pancreases, algorithm improvements, all kinds of interoperability integration, etc.)
  4. All systems (both DIY and commercial) have pros and cons. They also each will have their own learning curves. Some of that learning is generalized, and will translate between systems. But again, iPhone to Android or vice versa – your cheese gets moved and there will be learning to do if you switch systems.
  5. Remember, everyone learns differently – and everyone’s diabetes is different. Figure out what works well for you, and rock it!


What I wish CDEs (diabetes educators) and other HCPs knew about DIY and other diabetes tech (#OpenAPS or otherwise)

I had the awesome opportunity to present at #AADE17, the annual education meeting for the American Association of Diabetes Educators, this past weekend. My topic was about OpenAPS and DIY diabetes… which really translates to some broader things I want all educators and HCPs to know about patients and technology, whether it’s DIY or just unknown to them. Unfortunately AADE didn’t record or livestream my session, so I wanted to write up a summary of the content here.

(If you’re new to this blog/me/OpenAPS, you can also watch this June 2017 TEDX talk where I share some of the story of how I ended up with a DIY artificial pancreas and how the OpenAPS community came to be; or this older talk from OSCON 2016 as well. As always, if you’re curious to learn more about OpenAPS or wondering how to build your own DIY artificial pancreas, OpenAPS.org is the first place to learn more!)

Diabetes is hard. Even if you are privileged to have access to insulin, education, and technology – it can still be so incredibly hard to get it right. And even if you do everything “right”, the outcomes will still vary. And after all, the devices themselves are not perfect, and we still have diabetes.

The lack of varying alarms and the unchangeable volume is what led me to create DIYPS (my open loop and louder alarm system), and the same frustration with lack of data access and visualization led John Costik, Lane Desborough, Ben West, and so many others to explore creating other DIY tools, such as Nightscout. And thanks to social media, we all didn’t have to create in a vacuum: we can share code (this is what open source means) and insight through social media, and build upon each other’s work. As a result, these collaborations, sharing, and iterative development is how OpenAPS, the open source artificial pancreas system movement, was created.

I tweet and talk and share frequently about how great it is having #OpenAPS in my life. Norovirus? No problem. Changes in sensitivity due to exercise? Not the biggie it used to be.

However, this technology is by no means a cure. It still requires work on the part of the person with diabetes. We still have to:

  • Change pump sites
  • Change CGM sensors
  • Calibrate regularly
  • Deal with bonked pump sites and sensors that fall out

And also, given the speed of insulin, most people are still going to engage with the system for some kind of meal bolus or announcement. This is why it’s called “hybrid” closed loop technology. (However, depending on the sophistication of the technology, you start to get to be able to choose what you want to optimize for and the behaviors you want to choose to do less of, which is great.)

In some cases, we humans know more than the technology: such as when a meal is going to happen/is coming, and when exercise is going to happen. So it’s nice to be able to interoperate your devices and be able to use your phone, watch, computer, etc. to be able to tell the system what to do differently (i.e. set higher targets in the case of activity, or lower targets to achieve “eating soon” mode , or in the case of waking up).

But in a LOT of cases, it’s tiring for the human to have to think about all the things. Such as whether a pump site is slowly dying and causing apparent insulin resistant. Or such as when you’re more sensitive 12-24 hours after exercise. Or during menstrual cycles. Or when sick. Or during a growth spurt. Or during jet lag. Or during a trip where you can’t find anything to eat. Etc. It’s a lot for us PWD’s to track, and this is where computers come in handy. Things like autosensitivity in OpenAPS to automatically detect changes in sensitivity and adjust the variables for calculations automatically; and autotune, to track the data of what’s actually happening and make recommendations for changing your underlying pump settings (ISF, carb ratio, and basal rates).

And how has this technology been developed by patients? Iteratively, as we figure out what’s possible. It’s not about boiling the ocean; it’s about approaching problems bit by bit as we have new tools to solve them, or new people with energy to think about the problem in different ways. It’s like thinking about getting a car – you wouldn’t expect the manufacturer to sell bits and pieces of the car frame, and you don’t really expect medical device manufacturers to sell bits and pieces of a pump or other device. However, patients are closest to the REAL problems in living with diabetes. Instead of a “car”, they’re looking for solutions for getting from point A to point B. And so in the car analogy, that means starting with a skateboard, scooter, or bike – and ending up with a car is great, but the car is not the point.

So no, any piece of technology isn’t going to be a cure or solve all problems or work perfectly for everyone. But that is true whether it’s DIY or a commercial tool: one size certainly does not fit all. And patients are individuals with their own lives and their own challenges with diabetes, with different motivations around what aspects of life with diabetes feel like friction and what they feel equipped to tackle and solve.

So, here’s some of what’s on my list for what I’d like CDE’s and other HCP’s to know as a result of the proliferation of technology around diabetes:

  • Yes, DIY tech is often off label. But that’s ok – it just means it’s off label; it doesn’t prevent you from listening to why patients are using it, what we think it’s doing for us, and it doesn’t prevent you from asking questions, learning more, or still advising patients.
  • Don’t make us switch providers by refusing to discuss it or listen to it, just because it’s new/different/you don’t understand it. (By the way: we don’t expect you to understand all possible technology! You can’t be experts on everything, but that doesn’t mean shunning what you don’t know.)
  • You get to take advantage of the opportunity when someone brings something new into the office – it’s probably the first of many times you’ll see it, and the first patient is often on the bleeding edge and deeply engaged and understands what they’re using, and open to sharing what they’ve learned to help you, so you can also help other patients!
  • You also get to take advantage of the open source community. It’s open, not just for patients to use, but for companies, and for CDEs and other HCPs as well. There are dozens if not hundreds of active people on Twitter, Facebook, blogs, forums, and more who are happy to answer questions and help give perspective and insight into why/how/what things are.
  • Don’t forget – many of the DIY tools provide data and insight that currently don’t exist in any traditional and/or commercially and/or FDA-approved tool. Take autotune for example – there’s nothing else out there that we know of that will tune basal rates, ISF, and carb ratio for people with pumps. And the ability of tools like Nightscout reports to show data from a patient’s disparate devices is also incredibly helpful for healthcare providers and educators to use to help patients.

And one final point specific to hybrid closed loop technology: this technology is going to solve a lot of problems and frustrations. But, it may mean that patients will shift the prioritization of other quality of life factors like ease of use over older, traditionally learned diabetes behaviors. This means things like precise carb counting may go by the wayside for general meal size estimations, because the technology yields similar outcomes. Being aware of this will be important for when CDE’s are working with patients; knowing what the patterns of behaviors are and knowing where a patient has shifted their choices will be helpful for identifying what behaviors can be adapted to yield different outcomes.

I think the increase in technology (especially various types of closed loops, DIY and commercial) will yield MORE work for CDE’s and HCP’s, rather than less. This means it’s even more important for them to get up to speed on current and evolving technology – because it’s by no means going away. And the first wave of DIY’ers have a lot we can share and teach not just other patients, but also CDE’s. So again, many thanks to AADE for the opportunity to share some of this perspective at #AADE17, and thanks to everyone for the engagement during and after the session!

Unexpected side-effect of closed looping: Body re-calibrations

It’s fascinating how bodies adapt to changing situations.

For those of us with diabetes: do you remember the first time you took insulin after diagnosis? For me, I had been fasting for ~18 hours (because I felt so bad, and hadn’t eaten anything since dinner the night before) and drinking water, and my BG was still somehow 550+ at the endo’s office.

Water did nothing for my unquenchable thirst, but that first shot of insulin first sure did.

I still remember the vivid feeling of it being an internal liquid hydration for my body, and everything feeling SO different when it started kicking in.

In case the BG of 550+, the A1c of 14+ (don’t remember exact number), and me feeling terrible for weeks wasn’t enough, that’s one of the things that really reinforced that I have diabetes and insulin is something my body desperately needs but wasn’t getting.

Over the last ~14+ years, I’ve had a handful of times that reinforced the feeling of being dependent on this life-saving drug, and the drastic difference I feel with and without it. Usually, it’s been times where a pump site ripped out, or I was sick and high and highly resistant, and then finally stopped being as resistant and my blood sugar started responding to insulin finally after hours of being really high, and I started dropping.

But I’ve had different ways to experience this feeling lately, as a result of having live with a DIY closed loop (OpenAPS) for 2+ years – and it hasn’t involved anything drastic as a HIGH BG or equipment failure. It’s a result of my body re-calibrating to the new norm of my body being able to spend more and more time close to 100% in range, in a much tighter and lower range than I ever thought possible (especially now true with some of the flexibility and freedom oref1 now offers).

I originally had a brief fleeting thought about how BGs in the low 200s used to feel like the 300s did. Then, I realized that 180 felt “high”. One day, it was 160.

Then one day, my CGM said flat in 120s and I felt “high”. (I calibrated, and turned out that it was really 140). I’ve had several other days where I’d hit 140s and feel like I used to do in the mid-200s (slightly high, and annoying, but no major high symptoms like 300-400 would cause – just enough to feel it and be annoyed).

That was odd enough as a fleeting thought, but it was really odd to wake up one morning and without even looking at my watch or CGM to see what my BGs had been all night, know that I had been running high.

I further classified “really odd” as “completely crazy” when that “running high” meant floating around the 130-140 range, instead of down in the 90-110 range, which is where I probably spend 95% of my nights nowadays.

Last night is what triggered this blog post, plus a recurring observation that because I have a DIY closed loop that does so well at handling the small, unknown variances that cause disturbances in BG levels without me having to do much work, that as result it is MUCH easier to pinpoint major influences, like my liver dumping glucose (either because of a low or because it’s ‘full up’ and needs to get rid of the excess).

In last night’s case, it was a major liver dump of glucose.

Here’s what happened:

Scott and I went on a long walk, with the plan to stop for dinner on the way home. BG started dropping as I was about half a mile out from the restaurant, but I’m stubborn 😀 and didn’t want to eat a fruit strip when I was about to sit down an eat a burger. So, my BG was dropping low when I actually ate. I expected my BG to flatten on its own, given the pause in activity, so I bolused fairly normally for my burger, and we walked the last .5 miles home.

However, I ended up not rising from the burger like I usually do, and started dropping again. It was quite a drop, and I realize my burger digestion was different because of the previous low, so I ended up eating some fruit to handle the second low. My body was unhappy at two lows, and so my liver decided to save the day by dumping a bunch of glucose to help bring my blood sugar up. Double rebound effect, then, from the liver dump and the fruit I had eaten. Oh well, that’s what a closed loop is for!

Instead of rebounding into the high 300s (which I would have expected pre-closed loop), I maxed out at 220. The closed loop did a good job of bolusing on the way up. However, because of how much glucose my liver dumped, I stayed higher longer. (Again, this probably sounds crazy to anyone not looping, as it would have sounded to me before I began looping). I sat around 180 for the first three hours of the night, and then dropped down to ~160 for most of the rest of the night, and ended up waking up around 130.

And boy, did I know I had been high all night. I felt (and still feel, hours later) like I used to years ago when I would wake up in the 300s (or higher).


recalibration_3 hourHmm, 3 hours doesn’t look so bad despite feeling it.

recalibration_6 hour6 hour view shows why I feel it.

recalibration_12 hour12 hours. Sheesh.

recalibration_24 hour24 hours shows you the full view of the double low and why my liver decided I needed some help. Thanks, liver, for still being able to help if I really needed it!

recalibrating_pebble view of renormalizing Settling back to normal below 120, hours later.

There are SO many amazing things about DIY closed looping. Better A1c, better average BG, better time in range, less effort, less work, less worrying, more sleep, more time living your life.

One of the benefits, though, is this bit of double-edged sword: your body also re-calibrates to the new “normal”, and that means the occasional extreme BG excursion (even if not that extreme!) may give you a different range of symptoms than you used to experience.

This. Matters. (Why I continue to work on #OpenAPS, for myself and for others)

If you give a mouse a cookie or give a patient their data, great things will happen.

First, it was louder CGM alarms and predictive alerts (#DIYPS).

Next, it was a basic hybrid closed loop artificial pancreas that we open sourced so other people could build one if they wanted to (#OpenAPS, with the oref0 basic algorithm).

Then, it was all kinds of nifty lessons learned about timing insulin activity optimally (do eating soon mode around an hour before a meal) and how to use things like IFTTT integration to squash even the tiniest (like from 100mg/dL to 140mg/dL) predictable rises.

It was also things like displays, button, widgets on the devices of my choice – ranging from being able to “text” my pancreas, to a swipe and button tap on my phone, to a button press on my watch – not to mention tinier sized pancreases that fit in or clip easily to a pocket.

Then it was autosensitivity that enabled the system to adjust to my changing circumstances (like getting a norovirus), plus autotune to make sure my baseline pump settings were where they needed to be.

And now, it’s oref1 features that enable me to make different choices at every meal depending on the social situation and what I feel like doing, while still getting good outcomes. Actually, not good outcomes. GREAT outcomes.

With oref0 and OpenAPS, I’d been getting good or really good outcomes for 2 years. But it wasn’t perfect – I wasn’t routinely getting 100% time in range with lower end of the range BG for a 24hour average. ~90% time in range was more common. (Note – this time in range is generally calculated with 80-160mg/dL. I could easily “get” higher time in range with an 80-180 mg/dL target, or a lot higher also with a 70-170mg/dL target, but 80-160mg/dL was what I was actually shooting for, so that’s what I calculate for me personally). I was fairly happy with my average BGs, but they could have been slightly better.

I wrote from a general perspective this week about being able to “choose one” thing to give up. And oref1 is a definite game changer for this.

  • It’s being able to put in a carb estimate and do a single, partial bolus, and see your BG go from 90 to peaking out at 130 mg/dL despite a large carb (and pure ballpark estimate) meal. And no later rise or drop, either.
  • It’s now seeing multiple days a week with 24 hour average BGs a full ~10 or so points lower than you’re used to regularly seeing – and multiple days in a week with full 100% time in range (for 80-160mg/dL), and otherwise being really darn close to 100% way more often than I’ve been before.

But I have to tell you – seeing is believing, even more than the numbers show.

I remember in the early days of #DIYPS and #OpenAPS, there were a lot of people saying “well, that’s you”. But it’s not just me. See Tim’s take on “changing the habits of a lifetime“. See Katie’s parent perspective on how much her interactions/interventions have lessened on a daily basis when testing SMB.

See this quote from Matthias, an early tester of oref1:

I was pretty happy with my 5.8% from a couple months of SMB, which has included the 2 worst months of eating habits in years.  It almost feels like a break from diabetes, even though I’m still checking hourly to make sure everything is connected and working etc and periodically glancing to see if I need to do anything.  So much of the burden of tight control has been lifted, and I can’t even do a decent job explaining the feeling to family.

And another note from Katie, who started testing SMB and oref1:

We used to battle 220s at this time of day (showing a picture flat at 109). Four basal rates in morning. Extra bolus while leaving house. Several text messages before second class of day would be over. Crazy amount of work [in the morning]. Now I just have to brush my teeth.

And this, too:

I don’t know if I’ve ever gone 24 hours without ANY mention of something that was because of diabetes to (my child).

Ya’ll. This stuff matters. Diabetes is SO much more than the math – it’s the countless seconds that add up and subtract from our focus on school/work/life. And diabetes is taking away this time not just from a person with diabetes, but from our parents/spouses/siblings/children/loved ones. It’s a burden, it’s stressful…and everything we can do matters for improving quality of life. It brings me to tears every time someone posts about these types of transformative experiences, because it’s yet another reminder that this work makes a real difference in the real lives of real people. (And, it’s helpful for Scott to hear this type of feedback, too – since he doesn’t have diabetes himself, it’s powerful for him to see the impact of how his code contributions and the features we’re designing and building are making a difference not just to BG outcomes.)

Thank you to everyone who keeps paying it forward to help others, and to all of you who share your stories and feedback to help and encourage us to keep making things better for everyone.


Choose One: What would you give up if you could? (With #OpenAPS, maybe you can – oref1 includes unannounced meals or “UAM”)

What do you have to do today (related to daily insulin dosing for diabetes) that you’d like to give up if you could? Counting carbs? Bolusing? Or what about outcomes – what if you could give up going low after a meal? Or reduce the amount that you spike?

How many of these 5 things do you think are possible to achieve together?

  • No need to bolus
  • No need to count carbs
  • Medium/high carb meals
  • 80%+ time in range
  • no hypoglycemia

How many can you manage with your current therapy and tools of choice?  How many do you think will be possible with hybrid closed loop systems?  Please think about (and maybe even write down) your answers before reading further to get our perspective.

With just pump and CGM, it’s possible to get good time in range with proper boluses, counting carbs, and eating relatively low-carb (or getting lucky/spending a lot of time learning how to time your insulin with regular meals).  Even with all that, some people still go low/have hypoglycemia.  So, let’s call that a 2 (out of 5) that can be achieved simultaneously.

With a first-generation hybrid closed loop system like the original OpenAPS oref0 algorithm, it’s possible to get good time in range overnight, but achieve that for meal times would still require bolusing properly and counting carbs.  But with the perfect night-time BGs, it’s possible to achieve no-hypoglycemia and 80% time in range with medium carb meals (and high-carb meals with Eating Soon mode etc.).  So, let’s call that a 3 (out of 5).

With some of the advanced features we added to OpenAPS with oref0 (like advanced meal assist or “AMA” as we call it), it became a lot easier to achieve a 3 with less bolusing and less need to precisely count carbs.  It also deals better with high-carb meals, and gives the user even more flexibility.  So, let’s call that a 3.5.

A few months ago, when we began discussing how to further improve daily outcomes, we also began to discuss the idea of how to better deal with unannounced meals. This means when someone eats and boluses, but doesn’t enter carbs. (Or in some cases: eats, doesn’t enter carbs, and doesn’t even bolus). How do we design to better help in that safety, all while sticking to our safety principles and dosing safely?

I came up with this idea of “floating carbs” as a way to design a solution for this behavior. Essentially, we’ve learned that if BG spikes at a certain rate, it’s often related to carbs. We observed that AMA can appropriately respond to such a rise, while not dosing extra insulin if BG is not rising.  Which prompted the question: what if we had a “floating” amount of carbs hanging out there, and it could be decayed and dosed upon with AMA if that rise in BG was detected? That led us to build in support for unannounced meals, or “UAM”. (But you’ll probably see us still talk about “floating carbs” some, too, because that was the original way we were thinking about solving the UAM problem.) This is where the suite of tools that make up oref1 came from.  In addition to UAM, we also introduced supermicroboluses, or SMB for short.  (For more background info about oref1 and SMB, read here.)

So with OpenAPS oref1 with SMB and floating carbs for UAM, we are finally at the point to achieve a solid 4 out of 5.  And not just a single set of 4, but any 4 of the 5 (except we’d prefer you don’t choose hypoglycemia, of course):

  • With a low-carb meal, no-hypoglycemia and 80+% time in range is achievable without bolusing or counting carbs (with just an Eating Soon mode that triggers SMB).
  • With a regular meal, the user can either bolus for it (triggering floating carb UAM with SMB) or enter a rough carb count / meal announcement (triggering Eating Now SMB) and achieve 80% time in range.
  • If the user chooses to eat a regular meal and not bolus or enter a carb count (just an Eating Soon mode), the BG results won’t be as good, but oref1 will still handle it gracefully and bring BG back down without causing any hypoglycemia or extended hyperglycemia.

That is huge progress, of course.  And we think that might be about as good as it’s possible to do with current-generation insulin-only pump therapy.  To do better, we’d either need an APS that can dose glucagon and be configured for tight targets, or much faster insulin.  The dual-hormone systems currently in development are targeting an average BG of 140, or an A1c of 6.5, which likely means >20% of time spent > 160mg/dL.  And to achieve that, they do require meal announcements of the small/medium/large variety, similar to what oref1 needs.  Fiasp is promising on the faster-insulin front, and might allow us to develop a future version of oref1 that could deal with completely unannounced and un-bolused meals, but it’s probably not fast enough to achieve 80% time in range on a high-carb diet without some sort of meal announcement or boluses.

But 4 out of 5 isn’t bad, especially when you get to pick which 4, and can pick differently for every meal.

Does that make OpenAPS a “real” artificial pancreas? Is it a hybrid closed loop artificial insulin delivery system? Do we care what it’s called? For Scott and me; the answer is no: instead of focusing on what it’s called, let’s focus on how different tools and techniques work, and what we can do to continue to improve them.

Being Shuttleworth Funded with a Flash Grant as an independent patient researcher

Recently, I have been working on helping OpenAPS’ers collect our data and put it to good use in research (both by traditional researchers as well as using it to enable other fellow patient researchers or “citizen scientists). As a result, I have had the opportunity to work closely with Madeleine Ball at Open Humans. (Open Humans is the platform we use for the OpenAPS Data Commons.)

It’s been awesome to collaborate with Madeleine on many fronts. She’s proven herself really willing to listen to ideas and suggestions for things to change, to make it easier for both individuals to donate their data to research and for researchers who want to use the platform. And, despite me not having the same level of technical skills, she emits a deep respect for people of all experiences and perspectives. She’s also in general a really great person.

As someone who is (perhaps uniquely) utilizing the platform as both a data donor and as a data researcher, it has been fantastic to be able to work through the process of data donation, project creation, and project utilization from both perspectives. And, it’s been great to contribute ideas and make tools (like some of my scripts to download and unpack Open Humans data) that can then be used by other researchers on Open Humans.

Madeleine was also selected this year to be a Shuttleworth Fellow, applying “open” principles to change how we share and study human health data, plus exploring new, participant-centered approaches for health data sharing, research, and citizen science. Which means that everything she’s doing is in almost perfect sync with what we are doing in the OpenAPS and #WeAreNotWaiting communities.

What I didn’t know until this past week was that it also meant (as a Shuttleworth Fellow) that she was able to make nominations of individuals for a Shuttleworth Flash Grant, which is a grant made to a collection of social change agents, no strings attached, in support of their work.

I was astonished to receive an email from the Shuttleworth Foundation saying that I had been nominated by Madeleine for a $5,000 Flash Grant, which goes to individuals they would like to support/reward/encourage in their work for social good.

Shuttleworth Funded

I am so blown away by the Flash Grant itself – and the signal that this grant provides. This is the first (of hopefully many) organizations to recognize the importance of supporting independent patient researchers who are not affiliated with an institution, but rather with an online community. It’s incredibly meaningful for this research and work, which is centered around real needs of patients in the real world, to be funded, even to a small degree.

Many non-traditional researchers like me are unaffiliated with a traditional institution or organization. This means we do the research in our own time, funded solely by our own energy (and in some case resources). Time in of itself is a valuable contribution to research (think of the opportunity costs). However, it is also costly to distribute and disseminate ideas learned from patient-driven research to more traditional researchers. Even ignoring travel costs, most scientific conferences do not have a patient research access program, which means patients in some cases are asked to pay $400 (or more) per person for a single day pass to stand beside their poster if it is accepted for presentation at a conference. In some cases, patients have personal resources and determination and are willing to pay that cost. But not every patient is able to do that. (And to do it year over year as they continue to do new ground-breaking research each year – that adds up, too, especially when you factor in travel, lodging, and the opportunity cost of being away from a day job.)

So what will I use the Flash Grant for? Here’s so far what I’ve decided to put it toward:

#1 – I plan to use it to fund my & Scott’s travel costs this year to ADA’s Scientific Sessions, where our poster on Autotune & data from the #WeAreNotWaiting community will be presented. (I’m still hoping to convince ADA to create a patient researcher program vs. treating us like an individual walking in off the street; but if they again do not choose to do so, it will take $800 for Scott and I to stand with the poster during the poster session). Being at Scientific Sessions is incredibly valuable as researchers and developers, because we can have real-time conversations with traditional researchers who have not yet been introduced to some of our tools or the data collected and donated by the community. It’s one of the most valuable places for us to be in person in terms of facilitating new research partnerships, in addition to renewing and establishing relationships with device manufacturers who could (because our stuff is all open source MIT licensed) utilize our code and tools in commercial devices to more broadly reach people with diabetes.

#2 – Hardware parts. In order to best support the OpenAPS community, Scott and I have also been supporting and contributing to the development of open source hardware like the Explorer Board. Keeping in mind that each version of the board produced needs to be tested to see if the instructions related to OpenAPS need to change, we have been buying every iteration of Explorer Board so we can ensure compatibility and ease of use, which adds up. Having some of this grant funding go toward hardware supplies to support a multitude of setup options is nice!

There are so many individuals who have contributed in various ways to OpenAPS and WeAreNotWaiting and the patient-driven research movements. I’m incredibly encouraged, with a new spurt of energy and motivation, after receiving this Flash Grant to continue to further build upon everyone’s work and to do as much as possible to support every person in our collective communities. Thank you again to Madeleine for the nomination, and to the Shuttleworth Foundation for the Flash Grant, for the financial and emotional support for our community!

Introducing oref1 and super-microboluses (SMB) (and what it means compared to oref0, the original #OpenAPS algorithm)

For a while, I’ve been mentioning “next-generation” algorithms in passing when talking about some of the work that Scott and I have been doing as it relates to OpenAPS development. After we created autotune to help people (even non-loopers) tune underlying pump basal rates, ISF, and CSF, we revisited one of our regular threads of conversations about how it might be possible to further reduce the burden of life with diabetes with algorithm improvements related to meal-time insulin dosing.

This is why we first created meal-assist and then “advanced meal-assist” (AMA), because we learned that most people have trouble with estimating carbs and figuring out optimal timing of meal-related insulin dosing. AMA, if enabled and informed about the number of carbs, is a stronger aid for OpenAPS users who want extra help during and following mealtimes.

Since creating AMA, Scott and I had another idea of a way that we could do even more for meal-time outcomes. Given the time constraints and reality of currently available mealtime insulins (that peak in 60-90 minutes; they’re not instantaneous), we started talking about how to leverage the idea of a “super bolus” for closed loopers.

A super bolus is an approach you can take to give more insulin up front at a meal, beyond what the carb count would call for, by “borrowing” from basal insulin that would be delivered over the next few hours. By adding insulin to the bolus and then low temping for a few hours after that, it essentially “front shifts” some of the insulin activity.

Like a lot of things done manually, it’s hard to do safely and achieve optimal outcomes. But, like a lot of things, we’ve learned that by letting computers do more precise math than we humans are wont to do, OpenAPS can actually do really well with this concept.

Introducing oref1

Those of you who are familiar with the original OpenAPS reference design know that ONLY setting temporary basal rates was a big safety constraint. Why? Because it’s less of an issue if a temporary basal rate is issued over and over again; and if the system stops communicating, the temp basal eventually expires and resume normal pump activity. That was a core part of oref0. So to distinguish this new set of algorithm features that depart from that aspect of the oref0 approach, we are introducing it as “oref1”. Most OpenAPS users will only use oref0, like they have been doing. oref1 should only be enabled specifically by any advanced users who want to test or use these features.

The notable difference between the oref0 and oref1 algorithms is that, when enabled, oref1 makes use of small “supermicroboluses” (SMB) of insulin at mealtimes to more quickly (but safely) administer the insulin required to respond to blood sugar rises due to carb absorption.

Introducing SuperMicroBoluses (or “SMB”)

The microboluses administered by oref1 are called “super” because they use a miniature version of the “super bolus” technique described above.  They allow oref1 to safely dose mealtime insulin more rapidly, while at the same time setting a temp basal rate of zero of sufficient duration to ensure that BG levels will return to a safe range with no further action even if carb absorption slows suddenly (for example, due to post-meal activity or GI upset) or stops completely (for example due to an interrupted meal or a carb estimate that turns out to be too high). Where oref0 AMA might decide that 1 U of extra insulin is likely to be required, and will set a 2U/hr higher-than-normal temporary basal rate to deliver that insulin over 30 minutes, oref1 with SMB might deliver that same 1U of insulin as 0.4U, 0.3U, 0.2U, and 0.1U boluses, at 5 minute intervals, along with a 60 minute zero temp (from a normal basal of 1U/hr) in case the extra insulin proves unnecessary.

As with oref0, the oref1 algorithm continuously recalculates the insulin required every 5 minutes based on CGM data and previous dosing, which means that oref1 will continually issue new SMBs every 5 minutes, increasing or reducing their size as needed as long as CGM data indicates that blood glucose levels are rising (or not falling) relative to what would be expected from insulin alone.  If BG levels start falling, there is generally already a long zero temp basal running, which means that excess IOB is quickly reduced as needed, until BG levels stabilize and more insulin is warranted.

Safety constraints and safety design for SMB and oref1

Automatically administering boluses safely is of course the key challenge with such an algorithm, as we must find another way to avoid the issues highlighted in the oref0 design constraints.  In oref1, this is accomplished by using several new safety checks (as outlined here), and verifying all output, before the system can administer a SMB.

At the core of the oref1 SMB safety checks is the concept that OpenAPS must verify, via multiple redundant methods, that it knows about all insulin that has been delivered by the pump, and that the pump is not currently in the process of delivering a bolus, before it can safely do so.  In addition, it must calculate the length of zero temp required to eventually bring BG levels back in range even with no further carb absorption, set that temporary basal rate if needed, and verify that the correct temporary basal rate is running for the proper duration before administering a SMB.

To verify that it knows about all recent insulin dosing and that no bolus is currently being administered, oref1 first checks the pump’s reservoir level, then performs a full query of the pump’s treatment history, calculates the required insulin dose (noting the reservoir level the pump should be at when the dose is administered) and then checks the pump’s bolusing status and reservoir level again immediately before dosing.  These checks guard against dosing based on a stale recommendation that might otherwise be administered more than once, or the possibility that one OpenAPS rig might administer a bolus just as another rig is about to do so.  In addition, all SMBs are limited to 1/3 of the insulin known to be required based on current information, such that even in the race condition where two rigs nearly simultaneously issue boluses, no more than 2/3 of the required insulin is delivered, and future SMBs can be adjusted to ensure that oref1 never delivers more insulin than it can safely withhold via a zero temp basal.

In some situations, a lack of BG or intermittent pump communications can prevent SMBs from being delivered promptly.  In such cases, oref1 attempts to fall back to oref0 + AMA behavior and set an appropriate high temp basal.  However, if it is unable to do so, manual boluses are sometimes required to finish dosing for the recently consumed meal and prevent BG from rising too high.  As a result, oref1’s SMB features are only enabled as long as carb impact is still present: after a few hours (after carbs all decay), all such features are disabled, and oref1-enabled OpenAPS instances return to oref0 behavior while the user is asleep or otherwise not engaging with the system.

In addition to these safety status checks, the oref1 algorithm’s design helps ensure safety.  As already noted, setting a long-duration temporary basal rate of zero while super-microbolusing provides good protection against hypoglycemia, and very strong protection against severe hypoglycemia, by ensuring that insulin delivery is zero when BG levels start to drop, even if the OpenAPS rig loses communication with the pump, and that such a suspension is long enough to eventually bring BG levels back up to the target range, even if no manual corrective action is taken (for example, during sleep).  Because of these design features, oref1 may even represent an improvement over oref0 w/ AMA in terms of avoiding post-meal hypoglycemia.

In real world testing, oref1 has thus far proven at least as safe as oref0 w/ AMA with regard to hypoglycemia, and better able to prevent post-meal hyperglycemia when SMB is ongoing.

What does SMB “look” like?

Here is what SMB activity currently looks like when displayed on Nightscout, and my Pebble watch:

First oref1 SMB OpenAPS test by @DanaMLewisFirst oref1 SMB OpenAPS test as seen on @DanaMLewis pebble watch

How do features like this get developed and tested?

SMB, like any other advanced feature, goes through extensive testing. First, we talk about it. Then, it becomes written up in plain language as an issue for us to track discussion and development. Then we begin to develop the feature, and Scott and I test it on a spare pump and rig. When it gets to the point of being ready to test it in the real world, I test it during a time period when I can focus on observing and monitoring what it is doing. Throughout all of this, we continue to make tweaks and changes to improve what we’re developing. After several days (or for something this different, weeks) of Dana-testing, we then have a few other volunteers begin to test it on spare rigs. They follow the same process of monitoring it on spare rigs and giving feedback and helping us develop it before choosing to run it on a rig and a pump connected to their body. More feedback, discussion, and observation. Eventually, it gets to a point where it is ready to go to the “dev” branch of OpenAPS code, which is where this code is now heading. Several people will review the code and approve it to be added to the “dev” branch. We will then have others test the “dev” branch with this and any other features or code changes – both by people who want to enable this code feature, as well as people who don’t want this feature (to make sure we don’t break existing setups). Eventually, after numerous thumbs up from multiple members of the community who have helped us test different use cases, that code from the “dev” branch will be “approved” and will go to the “master” branch of code where it is available to a more typical user of OpenAPS.

However, not everyone automatically gets this code or will use it. People already running on the master branch won’t get this code or be able to use it until they update their rig. Even then, unless they were to specifically enable this feature (or any other advanced feature), they would not have this particular segment of code drive any of their rig’s behavior.

Where to find out more about oref1, SMB, etc.:

  • We have updated the OpenAPS Reference Design to reflect the differences between oref0 and the oref1 features.
  • OpenAPS documentation about oref1, which as of July 13, 2017 is now part of the master branch of oref0 code.
  • Ask questions! Like all things developed in the OpenAPS community, SMB and oref1-related features will evolve over time. We encourage you to hop into Gitter and ask questions about these features & whether they’re right for you (if you’re DIY closed looping).

Special note of thanks to several people who have contributed to ongoing discussions about SMB, plus the very early testers who have been running this on spare rigs and pumps. Plus always, ongoing thanks to everyone who is contributing and has contributed to OpenAPS development!

Edison Foundation honors #OpenAPS community with 2017 Edison Innovation Award

One of my favorite things about open source projects is the amazing humans behind it. OpenAPS came into existence because of numerous open source efforts; and has continued to evolve in both software and hardware improvements because of ongoing contributions in the open source world.

Some of the contributors and their stories are fairly well known (John Costik’s work to pull data from the CGM originally, which allowed Scott/me to create #DIYPS; Ben West’s work to study pump RF communications and create tools to communicate with the pump, in addition to his work on the building blocks that make up the openaps toolkit). Others have worked on areas that have drastically changed the trajectory of our community’s tools, as well. And two of the individuals who we also owe repeated thanks for facilitating our ability to utilize pocket-sized pancreas are Oskar Pearson and Pete Schwamb.

  • Oskar wanted to find a way to replace the Carelink stick, which has dismal range. (How dismal? Back in the day, I used to have a Pi on the bedside table connected to a Carelink stick under the mattress, plus another Carelink hanging over the middle of my bed to try to keep me looping all night in case I rolled over.) He ultimately leveraged some of Ben and Pete’s other work and created “mmeowlink“, which enabled other radio sticks (think TI cc1111 stick & other radios using the cc1110 and cc1111 chipsets) to similarly communicate with our loopable pumps.  He was also (I think) one of the first users of the Intel Edison for OpenAPS. When he shared his pictures showing the potential down sized rigs, jaws dropped across the Internet. This led to a bunch of new hardware options for OpenAPS rigs; Pi/Carelink was no longer the sole go-to. You could pick Pi/TI; Edison/TI; Pi/Slice of Radio; etc. And the range on these radio sticks is such that a single rig and radio stick can read (in most cases) across room(s). It greatly improved the reliability of real-world looping, and especially was a game changer for those on the go!
  • Pete is a wizard in the world of device RF communications. He created the RileyLink (named after his daughter) to bridge communications between a phone and any Sub-1 GHz device (like say, an insulin pump…). But he’s also done some other stellar projects – like subg_rfspy (general purpose firmware for cc111x for sub-GHz RF comms, which is what is leveraged in mmeowlink); and also ccprog (which enables you to flash the cc1110 radio chip on an Explorer board (see below) without having any separate equipment). And, as someone who has been building boards and decoding RF stuff for years..he’s also incredibly generous with sharing his knowledge with other people building open source hardware boards, including with those of us who collaborated on the Explorer Board.

In addition, there have been other people outside the OpenAPS community who have been touched by our stories or by diabetes in their families and have also stepped up to contribute to open source projects. This is how the Explorer Board came into existence. Someone from Intel had stumbled across OpenAPS on Twitter and reached out to meet up with Scott and me when he was in Seattle; he invited a hardware board designer he knew (Morgan Redfield) to stop by the meetup and offered to help initiate development of a smaller board. And amazingly, that’s exactly what happened. Morgan collaborated with us & others like Pete to design, build, and iterate on a small, open source hardware board (the “900 Mhz Explorer Board“) for the Edison that had a built in cc1110 radio, further allowing us to reduce the size of our rigs.

Eventually others at Intel heard about this collaboration, and we (OpenAPS and Morgan) were nominated for the Edison Foundation’s 2017 Edison Innovation Award for “Best Use of the Intel Edison Module”.

I was blown away to find out tonight that we are honored to actually receive the award on behalf of everyone who made these projects possible. I’m incredibly proud of this community and the dozens of people who have contributed in so many ways to a) make DIY artificial pancreases a thing and b) make it more feasible for hundreds of people to be DIYing these themselves with open source software and hardware. (And, this is very much in line with Thomas Edison’s work – the Edison Foundation spoke tonight about how Edison really created the group collaboration and innovation process!)

Big thanks to Intel and the Edison Foundation for highlighting the community’s efforts…and endless hugs and ongoing appreciation for everyone who has contributed to OpenAPS and other #WeAreNotWaiting projects!

Making it possible for researchers to work with #OpenAPS or general Nightscout data – and creating a complex json to csv command line tool that works with unknown schema

This is less of an OpenAPS/DIYPS/diabetes-related post, although that is normally what I blog about. However, since we created the #OpenAPS Data Commons on Open Humans, to allow those of us who desire to donate our diabetes data to research, I have been spending a lot of time figuring out the process from uploading your data to how data is managed and shared securely with researchers. The hardest part is helping researchers figure out how to handle the data – because we PWDs produce a lot of data :) . So this post explains some of the challenges of the data management to get it to a researcher-friendly format. I have been greatly helped over the years by general purpose open-source work from other people, and one of the things that helps ME the most as a non-traditional programmer is plain language posts explaining the thought process by behind the tools and the attempted solution paths. Especially because sometimes the web pages and blog posts pop higher in search than nitty gritty tool documentation without context. (Plus, I’ve been taking my own advice about not letting myself hold me back from trying, even when I don’t know how to do things yet.) So that’s what this post is!

Background/inspiration for the project and the tools I had to build:

We’re using Nightscout, which is a remote data-viewing platform for diabetes data, made with love and open source and freely available for anyone with diabetes to use. It’s one of the best ways to display not only continuous glucose monitor (CGM) data, but also data from our DIY closed loop artificial pancreases (#OpenAPS). It can store data from a number of different kinds and brands of diabetes devices (pumps, CGMs, manual data entries, etc.), which means it’s a rich source of data. As the number of DIY OpenAPS users are growing, we estimate that our real-world use is overtaking the amount of total hours of data from clinical trials of closed loop artificial pancreas systems.  In the #WeAreNotWaiting spirit of moving quickly (rather than waiting years for research teams to collect and analyze their own data) we want to see what we can learn from OpenAPS usage, not only by donating data to help traditional researchers speed up their work, but also by co-designing research studies of the things of most value to the diabetes community.

Step 1: Data from users to Open Humans

I thought Step 1 would be the hardest. However, thanks to Madeleine Ball, John Costik, and others in the Nightscout community, a simple Nightscout Data Transfer App was created that enables people with Nightscout data to pop it into their Open Humans accounts. It’s then very easy to join different projects (like the OpenAPS Data Commons) and share your data with those projects. And as the volunteer administrator of the OpenAPS Data Commons, it’s also easy for me to provide data to researchers.

The biggest challenge at this stage was figuring out how much data to pull from the API. I have almost 3 years worth of DIY diabetes data, and I have numerous devices over time uploading all at once…which makes for large chunks of data. Not everyone has this much data (or 6-7 rigs uploading constantly ;)). Props to Madeleine for the patience in working with me to make sure the super users with large data sets will be able to use all of these tools!

Step 2: Sharing the data with researchers

This was easy. Yay for data-sharing tools like Dropbox.

Step 3: Researchers being able to use the data

Here’s where thing started to get interesting. We have large data files that come in json format from Nightscout. I know some researchers we will be working with are probably very comfortable working with tools that can take large, complex json files. However…not all will be, especially because we also want to encourage independent researchers to engage with the data for projects. So I had the belated realization that we need to do something other than hand over json files. We need to convert, at the least, to csv so it can be easily viewed in Excel.

Sounds easy, right?

According to basic searches, there’s roughly a gazillion ways to convert json to csv. There’s even websites that will do it for you, without making you run it on the command line. However, most of them require you to know the types of data and the number of types, in order to therefore construct headers in the csv file to make it readable and useful to a human.

This is where the DIY and infinite possibility nature of all the kinds of diabetes tools anyone could be using with Nightscout, plus the infinite ways they can self-describe profiles and alarms and methods of entering data, makes it tricky. Just based on an eyeball search between two individuals, I was unable to find and count the hundred+ types of data entry possibilities. This is definitely a job for the computer, but I had to figure out how to train the computer to deal with this.

Again, json to csv tools are so common I figured there HAD to be someone who had done this. Finally, after a dozen varying searches and trying a variety of command line tools, I finally found one web-based tool that would take json, create the schema without knowing the data types in advance, and convert it to csv. It was (is) super slick. I got very excited when I saw it linked to a Github repository, because that meant it was probably open source and I can use it. I didn’t see any instructions for how to use it on the command line, though, so I message the author on Twitter and found out that it didn’t yet exist and was a not-yet-done TODO for him.

Sigh. Given this whole #WeAreNotWaiting thing (and given I’ve promised to help some of the researchers in figuring this out so we can initiate some of the research projects), I needed to figure out how to convert this tool into a command line version.

So, I did.

  • I taught myself how to unzip json files (ended up picking `gzip -cd`, because it works on both Mac and Linux)
  • I planned to then convert the web tool to be able to work on the command line, and use it to translate the json files to csv.

But..remember the big file issue? It struck again. So I first had to figure out the best way to estimate the size and splice or split the json into a series of files, without splitting it in a weird place and messing up the data. That became jsonsplit.sh, a tool to split a json file based on the size you give it (and if you don’t specify, it defaults to something like 100000 records).

FWIW: 100,000 records was too much for the more complex schema of the data I was working with, so I often did it in smaller chunks, but you can set it to whatever size you prefer.

So now “all” I had to do was:

  • Unzip the json
  • Break it down if it was too large, using jsonsplit.sh
  • Convert each of these files from json to csv

Phew. Each of these looks really simple now, but took a good chunk of time to figure out. Luckily, the author of the web tool had done much of the hard json-to-csv work, and Scott helped me figure out how to take the html-based version of the conversion and make it useable in the command line using javascript. That became complex-json2csv.js.

Because I knew how hard this all was, and wanted other people to be able to easily use this tool if they had large, complex json with unknown schema to deal with, I created a package.json so I could publish it to npm so you can download and run it anywhere.

I also had to create a script that would pass it all of the Open Humans data; unzip the file; run jsonsplit.sh, run complex-json2csv.js, and organize the data in a useful way, given the existing file structure of the data. Therefore I also created an “OpenHumansDataTools” repository on Github, so that other researchers who will be using Nightscout-based Open Humans data can use this if they want to work with the data. (And, there may be something useful to others using Open Humans even if they’re not using Nightscout data as their data source – again, see “large, complex, challenging json since you don’t know the data type and count of data types” issue. So this repo can link them to complex-json2csv.js and jsonsplit.sh for discovery purposes, as they’re general purpose tools.) That script is here.

My next TODO will be to write a script to take only slices of data based on information shared as part of the surveys that go with the Nightscout data; i.e. if you started your DIY closed loop on X data, take data from 2 weeks prior and 6 weeks after, etc.

I also created a pull request (PR) back to the original tool that inspired my work, in case he wants to add it to his repository for others who also want to run his great stuff from the command line. I know my stuff isn’t perfect, but it works :) and I’m proud of being able to contribute to general-purpose open source in addition to diabetes-specific open source work. (Big thanks as always to everyone who devotes their work to open source for others to use!)

So now, I can pass researchers json or csv files for use in their research. We have a number of studies who are planning to request access to the OpenAPS Data Commons, and I’m excited about how work like this to make diabetes data more broadly available for research will help improve our lives in the short and long term!

The only thing to fear is fear itself

(Things I didn’t realize were involved in open-sourcing a DIY artificial pancreas: writing “yes you can” style self-help blog posts to encourage people to take the first step to TRY and use the open source code and instructions that are freely available….for those who are willing to try.)

You are the only thing holding yourself back from trying. Maybe it’s trying to DIY closed loop at all. Maybe it’s trying to make a change to your existing rig that was set up a long time ago.  Maybe it’s doing something your spouse/partner/parent has previously done for you. Maybe it’s trying to think about changing the way you deal with diabetes at all.

Trying is hard. Learning is hard. But even harder (I think) is listening to the negative self-talk that says “I can’t do this” and perhaps going without something that could make a big difference in your daily life.

99% of the time, you CAN do the thing. But it primarily starts with being willing to try, and being ok with not being perfect right out of the gate.

I blogged last year (wow, almost two years ago actually) about making and doing and how I’ve learned to do so many new things as part of my OpenAPS journey that I never thought possible. I am not a traditional programmer, developer, engineer, or anything like that. Yes, I can code (some)…because I taught myself as I went and continue to teach myself as I go. It’s because I keep trying, and failing, then trying, and succeeding, and trying some more and asking lots of questions along the way.

Here’s what I’ve learned in 3+ years of doing DIY, technical diabetes things that I never thought I’d be able to accomplish:

  1. You don’t need to know everything.
  2. You really don’t particularly need to have any technical “ability” or experience.
  3. You DO need to know that you don’t know it all, even if you already know a thing or two about computers.
  4. (People who come into this process thinking they know everything tend to struggle even more than people who come in humble and ready to learn.)
  5. You only need to be willing to TRY, try, and try again.
  6. It might not always work on the first try of a particular thing…
  7. …but there’s help from the community to help you learn what you need to know.
  8. The learning is a big piece of this, because we’re completely changing the way we treat our diabetes when we go from manual interventions to a hybrid closed loop (and we learned some things to help do it safely).
  9. You can do this – as long as you think you can.
  10. If you think you can’t, you’re right – but it’s not that you can’t, it’s that you’re not willing to even try.

This list of things gets proved out to me on a weekly basis.

I see many people look at the #OpenAPS docs and think “I can’t do that” (and tell me this) and not even attempt to try.

What’s been interesting, though, is how many non-technical people jumped in and gave autotune a try. Even with the same level of no technical ability, several people jumped in, followed the instructions, asked questions, and were able to spin up a Linux virtual machine and run beta-level (brand new, not by any means perfect) code and get output and results. It was amazing, and really proved all those points above. People were deeply interested in getting the computer to help them, and it did. It sometimes took some work, but they were able to accomplish it.

OpenAPS, or anything else involving computers, is the same way. (And OpenAPS is even easier than most anything else that requires coding, in my opinion.) Someone recently estimated that setting up OpenAPS takes only 20 mouse clicks; 29 copy and paste lines of code; 10 entries of passwords or logins; and probably about 15-20 random small entries at prompts (like your NS site address or your email address or wifi addresses). There’s a reference guide, documentation that walks you through exactly what to do, and a supportive community.

You can do it. You can do this. You just have to be willing to try.