data visualization

What you should know about closed looping (DIY like #OpenAPS or otherwise)

by Dana Lewis

I’ve been wearing a DIY closed loop for something like 979 days..which means something like ~20,000 hours with this technology. Additionally, I’m not the only one. At the time of writing this post (see the latest count here), there are (n=1)*369+ (and that’s an undercount just based on who’s told us they’re looping) other DIYers out there, so the community has an estimated 1,800,000+ hours of cumulative experience, too.

Suffice to say, we’ve all learned a lot about this technology and how hybrid closed loop makes a difference in life with diabetes.

I previously gave a talk almost two years ago to the Sports & Diabetes Group Northwest here in Seattle, talking about #DIYPS, how we closed the loop, and #OpenAPS. (And you can see a recent TEDX talk I gave on OpenAPS here.) That was a springboard for meeting some awesome individuals who became very early DIY loopers in the Seattle area. And one of them (who also wore a pancreas at HIS wedding :)) had suggested we do another talk for SDGNW to update on some of what we have learned since then. But unfortunately, he got called out of town for work and couldn’t join me for presenting, so I went solo (ish, because Scott also came and contributed). I used a new analogy, because I think there’s a lot to think about before choosing and using closed loop technology, whether it’s DIY or commercial, and wanted to write it up for sharing here.

what_to_know_about_looping_danamlewis

First, some reminders for those familiar and some context for those who are not close to this technology. We’re talking about a hybrid closed loop, which is what I’m referring to when I say “artificial pancreas” or “AP” here. This type of technology makes small adjustments every few minutes to provide more or less insulin with the goal of keeping blood glucose (BG) levels in range. It’s complicated by the fact that insulin often peaks at 60-90 minutes…but food hits in ~15 minutes. So there’s often “catch up” being done with insulin to deal with food eaten previously, and also with hormones and other things that impact BGs that aren’t measurable. (This is also why it’s called hybrid, because for best outcomes people will still be doing some kind of meal announcement/bolus to deal with insulin timing.) As a result, even with pumps and CGMs, diabetes is still hard. A closed loop can do the needed math every five minutes, doesn’t go to sleep, and is very precise. It can respond more quickly (because it’s paying attention) than a human will in most situations, because we’re out living our lives/working/sleeping and not paying attention ONLY to diabetes. It’s not a cure, but it helps make living with diabetes better than it used to be.

However, I equate it to being a pilot who has seen technology on planes evolve to include “autopilot”. Even with hybrid closed loop technology, we’re still flying the “plane”.

looping_is_like_flying_plane_danamlewis

Here’s what I mean. There are stages for picking out and deciding to use the technology; preparing to use it/getting in the mode where you CAN use it; using it successfully; getting ready for the times when you can’t use it; and smoothing the way for the next time you use it.

It’s not perfect 24/7, you see, because we’re still using pump sites and continuous glucose monitor (CGM) sensors. The CGM sensor may last for 7 days, but then you have to change it out (or cough restart it cough), and you have a gap in data, which means you can’t loop. So you have this type of cycle regularly, and here’s what you need to know about each of these stages, regardless of whether we’re talking about DIY (like OpenAPS) or a commercial closed loop solution.

Preparing for takeoff

prepare_for_looping_danamlewisWhen you’re getting into the plane, you have a flight plan. You know when you will and won’t use the technology on board. Same for diabetes & closed looping. Make sure to think about the following for your tech of choice:

When will your loop work? When does it not? What happens if it breaks? What are your back up tools? How do you operate it: what happens if your sensor loses data, or you don’t calibrate? How does the algorithm work? What will it target your BG to be? What behaviors will you have to do (meal bolus or announcement, etc.) and how can you alter those to optimize performance? Also, what are the warning signs of failure to let you know when you need to take additional action with corrective insulin or eating carbs?

Taking off and the new technology learning curve

taking_off_learning_curve_danamlewisJust like switching from MDI pump (or even iPhone to Android and vice versa), you have a learning curve. When you go into looping or automated insulin delivery mode, you have to figure things out. You need to be able to figure out what’s happening and why it’s doing what it’s doing, so if you’re not happy with what’s happening, you can make a change. Why are you running high? Why are you running low? Knowing why it’s doing what it’s doing is critical for adjusting – either tweaking the closed loop settings, if you can, or adjusting your own behavior. Especially in the first few cycles of new tech, you’ll have a lot of learning around “I used to do things like X, but now I need to do them like Y.”

Why you might not be taking off and able to loop

blocking_takeoff_danamlewisYou also need to know why you can’t loop. There are three major categories of things that will prevent you from looping:

  1. No sensor, no looping.
  2. In some systems, wonky or missing data, no looping
  3. Communication errors between pieces of a system.

Some of these are obvious fixes (put in a new sensor if one fell out, or decide to put in a new sensor if the old one is bad), but depending on the system may involve some troubleshooting to get things going again.

Also, some of the commercial systems will kick you out of looping for various reasons (including lack of calibration), in addition to preventing you from looping in the first place without them, so knowing what these basic things are required for looping is useful to make sure you CAN automate.

Flying high: maintenance when you’re actually looping

maintenance_when_looping_danamlewisThere are some critical behaviors required for looping. (After all, when flying, there’s always a pilot present in the cockpit..right?!)

Some of these are basic behaviors you’ll be used to if you’ve been wearing a pump and CGM previously: keeping pump sites changed so the insulin works, and changing and calibrating CGM sensors.

HOWEVER – many people who “stretch” their CGM sensors find that they don’t want to stretch their sensors as far, as the data degrades over time. You do you, but keep in mind this might change when you’re looping vs. not, because you’re relying on good data to operate the system.

That being said, in addition to good sensor life, calibration hygiene is critical. You don’t want to loop off of wonky data, but also some commercial systems will kick you out if your calibration is way off and/or if you miss a calibration. (Personal opinion on this is a big ugh, which is why no DIY system that I know of does this.)

But if you keep your sites and sensors in good condition, this is where life is good. You’re looping! It’s microadjusting and helping keep things in range. Yay! This means better sleep, more time in range, and feeling better all around.

However, you still have diabetes, you’re still in the plane, so you still need to keep an eye on things. Monitoring the system is important (to make sure you’re still in autopilot and don’t need to actually fly the plane manually), so make sure you know how you (and your loved ones) can monitor the system’s operation, and know what your backup alarms are in case of system failures.

Note: there are approximately eleventy bajillion ways to remote monitor in DIY systems, but even if you have a commercial system that comes pre-baked without remote monitoring… you can add a DIY solution for that. So don’t feel like if you have a commercial AP that you can never use anything DIY – you can totally mix and match!

Dealing with turbulence

turbulence_danamlewisWhat kind of airplane/flight analogy would this be without including turbulence? :)

Like the things that can prevent looping in the first place, there are things that can throw off your looping. I already mentioned wonky sensor data that may mean either a blip in your looping time, or may kick you off looping. Again, your sensor life and your calibration practices will likely change.

But the other big disturbance, so to speak, is around body sensitivity changes. You know all the ways it can happen: you’re getting sick, recovering from getting sick, getting ready for/or are on/or are right after your period, or have an adrenaline spike, or have hormones surging, or have a growth spurt, or just exercised, etc.

This is what makes diabetes oh so hard so often. But this is where different closed loop systems can help, so this is one area you should ask about when picking a system: how does it adjust and adapt to sensitivity changes, and on what time frame? (In the DIY world, we use a number of techniques with this, ranging from autosensitivity to adapt on a 24 hour rolling scale of sensitivity changes, as well as using autotune to track bigger picture trends and changes needed to underlying settings. Reminder – anyone can use autotune if they’re willing to log bolus & carb data in Nightscout, not just closed loopers, so check that out if you’re interested! All DIY closed loop systems also use dynamic carbohydrate absorption in their respective algorithms, so that if you have slowed digestion for ANY reason, ranging from gastroparesis to getting glutened if you have celiac to merely walking after a meal, the system takes that into account and adjusts accordingly.)

The other things that can help you tough out some turbulence? Setting different modes, like an activity mode for exercise. The two things to know about exercise are:

  1. You don’t want to go into exercise with a bucket of IOB, so set activity mode WELL BEFORE you go out for activity. Depending on how much netIOB you have, that time may vary, but planning ahead with an activity mode makes a big difference for not going low during activity – even with a closed loop.
  2. Your sensitivity may be impacted for hours afterward, into the next day. See above about having a system that can respond to sensitivity changes like that, but also think about having multiple targets you can use temporarily (if your system allows it) so you can give the system a bigger buffer while it sorts out your body’s sensitivity changes.

Preparing for landing and making time between loops more smooth

prepare_for_landing_danamlewisJust like you’ll want to plan to go on the closed loop, you’ll want to plan for how to cycle off and then back on again. Depending on your system, there may be things you can do to smooth things out. One of the things I do is pre-soak a CGM sensor to skip the first day jumpy numbers. That makes a big difference for the first hours back on a “new” looping session. The other thing I do is stagger receiver start times (where I have two receivers that I stop/start at different times, so I’m not stuck for two hours without BG data to loop on).

But even if you can’t do that, you can do some other general planning ahead – like making sure your looping session doesn’t end in the middle of a big meal that’s being digested, or overnight. Those are the times when you’ll want to be looping the most.

Landing and preparing for the next looping session

Landing_danamlewisJust like learning to fly where you take a lot of training flights and review how the flight went, you’ll want to think about how things went and what you might change behavior-wise for your next looping session. Some of the things that may change over time as you learn more about your tech of choice:

  • Timing of meal announcement or boluses
  • Precision (if needed, or otherwise lack thereof) around carb counting
  • Using things like “eating soon” mode to optimize meal-time insulin effectiveness and reduce post-meal spikes
  • Using different activity patterns and targets to get ideal outcomes around exercise
  • Tweaking underlying settings (if you can)

General thoughts on looping

general_looping_reminders_danamlewisSome last thoughts about closed looping in general, regardless of the tech you might choose now or in the future:

  1. Picking one kind of technology does NOT lock you into it forever. If you’re DIYing now, you can choose commercial later. If you start on a commercial system, you can still try a DIY system.
  2. Don’t compare the original iPhone with an iPhone 6. Let’s be blunt: the Dexcom 7plus is a different beast than the Dexcom G4/G5. Similarly, Medtronic’s original “harpoon” sensor is different than their newest sensor tech. The Abbott Navigator is different than their Libre. Don’t be held up by perceptions of the old tech – make sure to check out the new stuff with a somewhat open mind.
  3. (Similarly, hopefully, in the future we’ll get to say the same about first-generation devices and algorithms. These things in commercial systems should change over time in terms of algorithm capabilities, targets, features, and usability. They certainly have in DIY – we’ve gotten smaller pancreases, algorithm improvements, all kinds of interoperability integration, etc.)
  4. All systems (both DIY and commercial) have pros and cons. They also each will have their own learning curves. Some of that learning is generalized, and will translate between systems. But again, iPhone to Android or vice versa – your cheese gets moved and there will be learning to do if you switch systems.
  5. Remember, everyone learns differently – and everyone’s diabetes is different. Figure out what works well for you, and rock it!

 

Unexpected side-effect of closed looping: Body re-calibrations

by Dana Lewis

It’s fascinating how bodies adapt to changing situations.

For those of us with diabetes: do you remember the first time you took insulin after diagnosis? For me, I had been fasting for ~18 hours (because I felt so bad, and hadn’t eaten anything since dinner the night before) and drinking water, and my BG was still somehow 550+ at the endo’s office.

Water did nothing for my unquenchable thirst, but that first shot of insulin first sure did.

I still remember the vivid feeling of it being an internal liquid hydration for my body, and everything feeling SO different when it started kicking in.

In case the BG of 550+, the A1c of 14+ (don’t remember exact number), and me feeling terrible for weeks wasn’t enough, that’s one of the things that really reinforced that I have diabetes and insulin is something my body desperately needs but wasn’t getting.

Over the last ~14+ years, I’ve had a handful of times that reinforced the feeling of being dependent on this life-saving drug, and the drastic difference I feel with and without it. Usually, it’s been times where a pump site ripped out, or I was sick and high and highly resistant, and then finally stopped being as resistant and my blood sugar started responding to insulin finally after hours of being really high, and I started dropping.

But I’ve had different ways to experience this feeling lately, as a result of having live with a DIY closed loop (OpenAPS) for 2+ years – and it hasn’t involved anything drastic as a HIGH BG or equipment failure. It’s a result of my body re-calibrating to the new norm of my body being able to spend more and more time close to 100% in range, in a much tighter and lower range than I ever thought possible (especially now true with some of the flexibility and freedom oref1 now offers).

—

I originally had a brief fleeting thought about how BGs in the low 200s used to feel like the 300s did. Then, I realized that 180 felt “high”. One day, it was 160.

Then one day, my CGM said flat in 120s and I felt “high”. (I calibrated, and turned out that it was really 140). I’ve had several other days where I’d hit 140s and feel like I used to do in the mid-200s (slightly high, and annoying, but no major high symptoms like 300-400 would cause – just enough to feel it and be annoyed).

That was odd enough as a fleeting thought, but it was really odd to wake up one morning and without even looking at my watch or CGM to see what my BGs had been all night, know that I had been running high.

I further classified “really odd” as “completely crazy” when that “running high” meant floating around the 130-140 range, instead of down in the 90-110 range, which is where I probably spend 95% of my nights nowadays.

—

Last night is what triggered this blog post, plus a recurring observation that because I have a DIY closed loop that does so well at handling the small, unknown variances that cause disturbances in BG levels without me having to do much work, that as result it is MUCH easier to pinpoint major influences, like my liver dumping glucose (either because of a low or because it’s ‘full up’ and needs to get rid of the excess).

In last night’s case, it was a major liver dump of glucose.

Here’s what happened:

Scott and I went on a long walk, with the plan to stop for dinner on the way home. BG started dropping as I was about half a mile out from the restaurant, but I’m stubborn 😀 and didn’t want to eat a fruit strip when I was about to sit down an eat a burger. So, my BG was dropping low when I actually ate. I expected my BG to flatten on its own, given the pause in activity, so I bolused fairly normally for my burger, and we walked the last .5 miles home.

However, I ended up not rising from the burger like I usually do, and started dropping again. It was quite a drop, and I realize my burger digestion was different because of the previous low, so I ended up eating some fruit to handle the second low. My body was unhappy at two lows, and so my liver decided to save the day by dumping a bunch of glucose to help bring my blood sugar up. Double rebound effect, then, from the liver dump and the fruit I had eaten. Oh well, that’s what a closed loop is for!

Instead of rebounding into the high 300s (which I would have expected pre-closed loop), I maxed out at 220. The closed loop did a good job of bolusing on the way up. However, because of how much glucose my liver dumped, I stayed higher longer. (Again, this probably sounds crazy to anyone not looping, as it would have sounded to me before I began looping). I sat around 180 for the first three hours of the night, and then dropped down to ~160 for most of the rest of the night, and ended up waking up around 130.

And boy, did I know I had been high all night. I felt (and still feel, hours later) like I used to years ago when I would wake up in the 300s (or higher).

—

Visuals

recalibration_3 hourHmm, 3 hours doesn’t look so bad despite feeling it.

recalibration_6 hour6 hour view shows why I feel it.

recalibration_12 hour12 hours. Sheesh.

recalibration_24 hour24 hours shows you the full view of the double low and why my liver decided I needed some help. Thanks, liver, for still being able to help if I really needed it!

recalibrating_pebble view of renormalizing Settling back to normal below 120, hours later.

—

There are SO many amazing things about DIY closed looping. Better A1c, better average BG, better time in range, less effort, less work, less worrying, more sleep, more time living your life.

One of the benefits, though, is this bit of double-edged sword: your body also re-calibrates to the new “normal”, and that means the occasional extreme BG excursion (even if not that extreme!) may give you a different range of symptoms than you used to experience.

This. Matters. (Why I continue to work on #OpenAPS, for myself and for others)

by Dana Lewis

If you give a mouse a cookie or give a patient their data, great things will happen.

First, it was louder CGM alarms and predictive alerts (#DIYPS).

Next, it was a basic hybrid closed loop artificial pancreas that we open sourced so other people could build one if they wanted to (#OpenAPS, with the oref0 basic algorithm).

Then, it was all kinds of nifty lessons learned about timing insulin activity optimally (do eating soon mode around an hour before a meal) and how to use things like IFTTT integration to squash even the tiniest (like from 100mg/dL to 140mg/dL) predictable rises.

It was also things like displays, button, widgets on the devices of my choice – ranging from being able to “text” my pancreas, to a swipe and button tap on my phone, to a button press on my watch – not to mention tinier sized pancreases that fit in or clip easily to a pocket.

Then it was autosensitivity that enabled the system to adjust to my changing circumstances (like getting a norovirus), plus autotune to make sure my baseline pump settings were where they needed to be.

And now, it’s oref1 features that enable me to make different choices at every meal depending on the social situation and what I feel like doing, while still getting good outcomes. Actually, not good outcomes. GREAT outcomes.

—

With oref0 and OpenAPS, I’d been getting good or really good outcomes for 2 years. But it wasn’t perfect – I wasn’t routinely getting 100% time in range with lower end of the range BG for a 24hour average. ~90% time in range was more common. (Note – this time in range is generally calculated with 80-160mg/dL. I could easily “get” higher time in range with an 80-180 mg/dL target, or a lot higher also with a 70-170mg/dL target, but 80-160mg/dL was what I was actually shooting for, so that’s what I calculate for me personally). I was fairly happy with my average BGs, but they could have been slightly better.

I wrote from a general perspective this week about being able to “choose one” thing to give up. And oref1 is a definite game changer for this.

  • It’s being able to put in a carb estimate and do a single, partial bolus, and see your BG go from 90 to peaking out at 130 mg/dL despite a large carb (and pure ballpark estimate) meal. And no later rise or drop, either.
  • It’s now seeing multiple days a week with 24 hour average BGs a full ~10 or so points lower than you’re used to regularly seeing – and multiple days in a week with full 100% time in range (for 80-160mg/dL), and otherwise being really darn close to 100% way more often than I’ve been before.

But I have to tell you – seeing is believing, even more than the numbers show.

I remember in the early days of #DIYPS and #OpenAPS, there were a lot of people saying “well, that’s you”. But it’s not just me. See Tim’s take on “changing the habits of a lifetime“. See Katie’s parent perspective on how much her interactions/interventions have lessened on a daily basis when testing SMB.

See this quote from Matthias, an early tester of oref1:

I was pretty happy with my 5.8% from a couple months of SMB, which has included the 2 worst months of eating habits in years.  It almost feels like a break from diabetes, even though I’m still checking hourly to make sure everything is connected and working etc and periodically glancing to see if I need to do anything.  So much of the burden of tight control has been lifted, and I can’t even do a decent job explaining the feeling to family.

And another note from Katie, who started testing SMB and oref1:

We used to battle 220s at this time of day (showing a picture flat at 109). Four basal rates in morning. Extra bolus while leaving house. Several text messages before second class of day would be over. Crazy amount of work [in the morning]. Now I just have to brush my teeth.

And this, too:

I don’t know if I’ve ever gone 24 hours without ANY mention of something that was because of diabetes to (my child).

—

Ya’ll. This stuff matters. Diabetes is SO much more than the math – it’s the countless seconds that add up and subtract from our focus on school/work/life. And diabetes is taking away this time not just from a person with diabetes, but from our parents/spouses/siblings/children/loved ones. It’s a burden, it’s stressful…and everything we can do matters for improving quality of life. It brings me to tears every time someone posts about these types of transformative experiences, because it’s yet another reminder that this work makes a real difference in the real lives of real people. (And, it’s helpful for Scott to hear this type of feedback, too – since he doesn’t have diabetes himself, it’s powerful for him to see the impact of how his code contributions and the features we’re designing and building are making a difference not just to BG outcomes.)

—

Thank you to everyone who keeps paying it forward to help others, and to all of you who share your stories and feedback to help and encourage us to keep making things better for everyone.

 

Why guess when you don’t have to? (#OpenAPS logs & why they’re handy)

by Dana Lewis

One of the biggest benefits (in my very biased opinion) of a DIY closed loop is this: it’s designed to be understandable to the person using it.

You don’t have to guess “what did it do at 2am?” or “why did it do a temp basal and not an SMB?”

Well, you COULD guess – but you don’t have to. Guessing is a choice ;).

Because we’ve been designing a system that a person has to decide to trust, it provides information about everything it’s doing and the information it has. That’s what “the logs” are for, and you can get information from “the logs” from a couple of places:

  • The OpenAPS “pill” in Nightscout
  • Secondary logging sources like Papertrail
  • Information that shows up on your Pebble watch
  • The full logs from SSH’ing into a rig (usually what we mean when we ask, “what do your logs say?”)

Here’s an example of the information the OpenAPS pill provides me in Nightscout:

Example OpenAPS pill info in Nightscout

This tells me that at 11:03 am, my BG was 121; I had no carbs on board; was dropping a tiny bit as expected and was likely going to end up slightly below my target; and the current temporary basal rate running was about equivalent to what OpenAPS thought I needed at the time. I had 0.47 netIOB, all from basal adjustments. It also specifies some of the eventual numbers that are what trigger the “purple line predictions” displayed in Nightscout, so if you can’t tell where the line is (90 or 100?), you can use the pill information to determine that more easily.

(Here’s the instructions for setting up Nightscout for OpenAPS)

Here’s an example of a log from Papertrail and what it tells us:

Example papertrail usage for OpenAPS

This example is from Katie, who described her daughter’s patterns in the morning: when Anna leaves her rig in the bedroom and went to take a shower, you can see the tune change at around 6:55, meaning she’s out of range of the rig. After the shower, getting dressed, and getting back to the rig around 7:25, it goes back to “normal” tuning (which means reading and writing to the pump as usual).

Papertrail is handy for figuring out if a rig is working or not remotely and high level why it might not be, especially if it’s a communication or power problem. But I generally find it to be most helpful when you know what kind of problem it is, and use it to drill down on a particular thing. However, it’s not going to give you absolutely all the details needed for every problem – so make sure to read about how to access the traditional logs, too, and be able to do that on the go.

(Here’s the instructions for getting Papertrail going for OpenAPS)

Here’s what the logs ported to my Pebble tell me:

OpenAPS logs on Pebble watch @DanaMLewis example

There’s several helpful things that display on my watch (using the excellent “Urchin” watchface designed by Mark Wilson, which you can customize to suit your personal preference): BGs, basal activity, and then some detailed text, similar to what’s in the OpenAPS pill (current BG, the change in BG, timestamp of BG, my netIOB, my eventual BGs, and any temp basal activity). In this case, it’s easy for me to glance and see that I was a bit low for a while; am now flat but have negative net IOB so it’s been high temping a bit to level out the netIOB.

(I’ve always preferred a data-rich watchface – even back in the days of “open looping” with #DIYPS:)

https://twitter.com/danamlewis/status/652566409537433600/photo/1

(Here’s more about the Urchin watchface)

Here’s what the full logs from the rig tell me:

Example OpenAPS logs from the rig

This has a LOT of information in it (which is why it’s so awesome). There are messages being shared by each step of the loop – when it’s listening for “silence” to make sure it can talk successfully to the pump; refreshing pump history; checking the clocks on devices and for fresh BGs; and then processing through the math on what the BG is, where it’s headed, and what needs to happen. You can also see from this example where autosensitivity is kicking in, adjust basals slightly up, target down, and sensitivity down, etc. (And for those who aren’t testing oref1 features like SMB and UAM yet, you’ll get a glimpse of how there’s now additional information in the logs about if those features are currently enabled.)

(Here are some other logs you can watch, and how to run them)

Pro tip for OpenAPS users: if you’re logged into your rig, you just have to type l (the letter “L” but lower case) for it to bring up your logs!

So if you find yourself wondering: what did OpenAPS do/why did it do <thing>? Instead of wondering, start by looking at the logs.

And remember, if you don’t know what the problem is – the full logs are the best source of information for spotting what the main problem is. You can then use the information from the logs to ask about how to resolve a particular problem (Gitter is great for this!)– but part of troubleshooting well/finding out more is taking the first step to pull up your logs, because anyone who is going to help you troubleshoot will need that information to figure out a solution.

—

And if you ever see someone say “RTFL”, instead of “read the manual” or “read the docs”, it means “read the logs”. 😉 :)

Making it possible for researchers to work with #OpenAPS or general Nightscout data – and creating a complex json to csv command line tool that works with unknown schema

by Dana Lewis

This is less of an OpenAPS/DIYPS/diabetes-related post, although that is normally what I blog about. However, since we created the #OpenAPS Data Commons on Open Humans, to allow those of us who desire to donate our diabetes data to research, I have been spending a lot of time figuring out the process from uploading your data to how data is managed and shared securely with researchers. The hardest part is helping researchers figure out how to handle the data – because we PWDs produce a lot of data :) . So this post explains some of the challenges of the data management to get it to a researcher-friendly format. I have been greatly helped over the years by general purpose open-source work from other people, and one of the things that helps ME the most as a non-traditional programmer is plain language posts explaining the thought process by behind the tools and the attempted solution paths. Especially because sometimes the web pages and blog posts pop higher in search than nitty gritty tool documentation without context. (Plus, I’ve been taking my own advice about not letting myself hold me back from trying, even when I don’t know how to do things yet.) So that’s what this post is!

OH that I "certainly stress tested" a tool with lots of data

Background/inspiration for the project and the tools I had to build:

We’re using Nightscout, which is a remote data-viewing platform for diabetes data, made with love and open source and freely available for anyone with diabetes to use. It’s one of the best ways to display not only continuous glucose monitor (CGM) data, but also data from our DIY closed loop artificial pancreases (#OpenAPS). It can store data from a number of different kinds and brands of diabetes devices (pumps, CGMs, manual data entries, etc.), which means it’s a rich source of data. As the number of DIY OpenAPS users are growing, we estimate that our real-world use is overtaking the amount of total hours of data from clinical trials of closed loop artificial pancreas systems.  In the #WeAreNotWaiting spirit of moving quickly (rather than waiting years for research teams to collect and analyze their own data) we want to see what we can learn from OpenAPS usage, not only by donating data to help traditional researchers speed up their work, but also by co-designing research studies of the things of most value to the diabetes community.

Step 1: Data from users to Open Humans

I thought Step 1 would be the hardest. However, thanks to Madeleine Ball, John Costik, and others in the Nightscout community, a simple Nightscout Data Transfer App was created that enables people with Nightscout data to pop it into their Open Humans accounts. It’s then very easy to join different projects (like the OpenAPS Data Commons) and share your data with those projects. And as the volunteer administrator of the OpenAPS Data Commons, it’s also easy for me to provide data to researchers.

The biggest challenge at this stage was figuring out how much data to pull from the API. I have almost 3 years worth of DIY diabetes data, and I have numerous devices over time uploading all at once…which makes for large chunks of data. Not everyone has this much data (or 6-7 rigs uploading constantly ;)). Props to Madeleine for the patience in working with me to make sure the super users with large data sets will be able to use all of these tools!

Step 2: Sharing the data with researchers

This was easy. Yay for data-sharing tools like Dropbox.

Step 3: Researchers being able to use the data

Here’s where thing started to get interesting. We have large data files that come in json format from Nightscout. I know some researchers we will be working with are probably very comfortable working with tools that can take large, complex json files. However…not all will be, especially because we also want to encourage independent researchers to engage with the data for projects. So I had the belated realization that we need to do something other than hand over json files. We need to convert, at the least, to csv so it can be easily viewed in Excel.

Sounds easy, right?

According to basic searches, there’s roughly a gazillion ways to convert json to csv. There’s even websites that will do it for you, without making you run it on the command line. However, most of them require you to know the types of data and the number of types, in order to therefore construct headers in the csv file to make it readable and useful to a human.

This is where the DIY and infinite possibility nature of all the kinds of diabetes tools anyone could be using with Nightscout, plus the infinite ways they can self-describe profiles and alarms and methods of entering data, makes it tricky. Just based on an eyeball search between two individuals, I was unable to find and count the hundred+ types of data entry possibilities. This is definitely a job for the computer, but I had to figure out how to train the computer to deal with this.

Again, json to csv tools are so common I figured there HAD to be someone who had done this. Finally, after a dozen varying searches and trying a variety of command line tools, I finally found one web-based tool that would take json, create the schema without knowing the data types in advance, and convert it to csv. It was (is) super slick. I got very excited when I saw it linked to a Github repository, because that meant it was probably open source and I can use it. I didn’t see any instructions for how to use it on the command line, though, so I message the author on Twitter and found out that it didn’t yet exist and was a not-yet-done TODO for him.

Sigh. Given this whole #WeAreNotWaiting thing (and given I’ve promised to help some of the researchers in figuring this out so we can initiate some of the research projects), I needed to figure out how to convert this tool into a command line version.

So, I did.

  • I taught myself how to unzip json files (ended up picking `gzip -cd`, because it works on both Mac and Linux)
  • I planned to then convert the web tool to be able to work on the command line, and use it to translate the json files to csv.

But..remember the big file issue? It struck again. So I first had to figure out the best way to estimate the size and splice or split the json into a series of files, without splitting it in a weird place and messing up the data. That became jsonsplit.sh, a tool to split a json file based on the size you give it (and if you don’t specify, it defaults to something like 100000 records).

FWIW: 100,000 records was too much for the more complex schema of the data I was working with, so I often did it in smaller chunks, but you can set it to whatever size you prefer.

So now “all” I had to do was:

  • Unzip the json
  • Break it down if it was too large, using jsonsplit.sh
  • Convert each of these files from json to csv

Phew. Each of these looks really simple now, but took a good chunk of time to figure out. Luckily, the author of the web tool had done much of the hard json-to-csv work, and Scott helped me figure out how to take the html-based version of the conversion and make it useable in the command line using javascript. That became complex-json2csv.js.

Because I knew how hard this all was, and wanted other people to be able to easily use this tool if they had large, complex json with unknown schema to deal with, I created a package.json so I could publish it to npm so you can download and run it anywhere.

I also had to create a script that would pass it all of the Open Humans data; unzip the file; run jsonsplit.sh, run complex-json2csv.js, and organize the data in a useful way, given the existing file structure of the data. Therefore I also created an “OpenHumansDataTools” repository on Github, so that other researchers who will be using Nightscout-based Open Humans data can use this if they want to work with the data. (And, there may be something useful to others using Open Humans even if they’re not using Nightscout data as their data source – again, see “large, complex, challenging json since you don’t know the data type and count of data types” issue. So this repo can link them to complex-json2csv.js and jsonsplit.sh for discovery purposes, as they’re general purpose tools.) That script is here.

My next TODO will be to write a script to take only slices of data based on information shared as part of the surveys that go with the Nightscout data; i.e. if you started your DIY closed loop on X data, take data from 2 weeks prior and 6 weeks after, etc.

I also created a pull request (PR) back to the original tool that inspired my work, in case he wants to add it to his repository for others who also want to run his great stuff from the command line. I know my stuff isn’t perfect, but it works :) and I’m proud of being able to contribute to general-purpose open source in addition to diabetes-specific open source work. (Big thanks as always to everyone who devotes their work to open source for others to use!)

—

So now, I can pass researchers json or csv files for use in their research. We have a number of studies who are planning to request access to the OpenAPS Data Commons, and I’m excited about how work like this to make diabetes data more broadly available for research will help improve our lives in the short and long term!

The only thing to fear is fear itself

by Dana Lewis

(Things I didn’t realize were involved in open-sourcing a DIY artificial pancreas: writing “yes you can” style self-help blog posts to encourage people to take the first step to TRY and use the open source code and instructions that are freely available….for those who are willing to try.)

—

You are the only thing holding yourself back from trying. Maybe it’s trying to DIY closed loop at all. Maybe it’s trying to make a change to your existing rig that was set up a long time ago.  Maybe it’s doing something your spouse/partner/parent has previously done for you. Maybe it’s trying to think about changing the way you deal with diabetes at all.

Trying is hard. Learning is hard. But even harder (I think) is listening to the negative self-talk that says “I can’t do this” and perhaps going without something that could make a big difference in your daily life.

99% of the time, you CAN do the thing. But it primarily starts with being willing to try, and being ok with not being perfect right out of the gate.

—

I blogged last year (wow, almost two years ago actually) about making and doing and how I’ve learned to do so many new things as part of my OpenAPS journey that I never thought possible. I am not a traditional programmer, developer, engineer, or anything like that. Yes, I can code (some)…because I taught myself as I went and continue to teach myself as I go. It’s because I keep trying, and failing, then trying, and succeeding, and trying some more and asking lots of questions along the way.

Here’s what I’ve learned in 3+ years of doing DIY, technical diabetes things that I never thought I’d be able to accomplish:

  1. You don’t need to know everything.
  2. You really don’t particularly need to have any technical “ability” or experience.
  3. You DO need to know that you don’t know it all, even if you already know a thing or two about computers.
  4. (People who come into this process thinking they know everything tend to struggle even more than people who come in humble and ready to learn.)
  5. You only need to be willing to TRY, try, and try again.
  6. It might not always work on the first try of a particular thing…
  7. …but there’s help from the community to help you learn what you need to know.
  8. The learning is a big piece of this, because we’re completely changing the way we treat our diabetes when we go from manual interventions to a hybrid closed loop (and we learned some things to help do it safely).
  9. You can do this – as long as you think you can.
  10. If you think you can’t, you’re right – but it’s not that you can’t, it’s that you’re not willing to even try.

This list of things gets proved out to me on a weekly basis.

I see many people look at the #OpenAPS docs and think “I can’t do that” (and tell me this) and not even attempt to try.

What’s been interesting, though, is how many non-technical people jumped in and gave autotune a try. Even with the same level of no technical ability, several people jumped in, followed the instructions, asked questions, and were able to spin up a Linux virtual machine and run beta-level (brand new, not by any means perfect) code and get output and results. It was amazing, and really proved all those points above. People were deeply interested in getting the computer to help them, and it did. It sometimes took some work, but they were able to accomplish it.

OpenAPS, or anything else involving computers, is the same way. (And OpenAPS is even easier than most anything else that requires coding, in my opinion.) Someone recently estimated that setting up OpenAPS takes only 20 mouse clicks; 29 copy and paste lines of code; 10 entries of passwords or logins; and probably about 15-20 random small entries at prompts (like your NS site address or your email address or wifi addresses). There’s a reference guide, documentation that walks you through exactly what to do, and a supportive community.

You can do it. You can do this. You just have to be willing to try.

Automating “wake up” mode with IFTTT and #OpenAPS to blunt morning hormonal rises

by Dana Lewis

tl;dr – automate a trigger to your #OpenAPS rig to start “wake up” mode (or “eating soon”, assuming you eat breakfast) without you having to remember to do it.

—

Yesterday morning, I woke up and headed to my desk to start working. Because I’m getting some amazing flat line overnights now, thanks to my DIY closed loop (#OpenAPS), I’m more attuned to the fact that after I wake up and start moving around, my hormones kick in to help wake me up (I guess), and I have a small BG rise that’s not otherwise explained by anything else. (It’s not a baseline basal problem, because it happens after I wake up regardless of it being 6am or 8am or even 10:30am if I sleep in on a weekend. It’s also more pronounced when I feel sleep deprived, like my body is working even harder to wake me up.)

Later in the morning, I took a break to jot down my thoughts in response to a question about normal meal rises on #OpenAPS and strategies to optimize mealtimes. It occurred to me later, after being hyper attuned to my lunch results, that my morning wake-up rise up from 1oo perfectly flat to ~140 was higher than the 131 peak I hit after my lunchtime bowl of potato soup.

Hmm, I thought. I wish there was something I could do to help with those morning rises. I often do a temporary target down to 80 mg/dL (a la “eating soon” mode) once I spot the rise, but that’s after it’s already started and very dependent on me paying attention/noticing the rise.

I also have a widely varied schedule (and travel a lot), so I don’t like the idea of scheduling the temp target, or having recurring calendar events that is yet another thing to babysit and change constantly.

What I want is something that is automatically triggered when I wake up, so whether I pop out of the bed or read for 15 minutes first, it kicks in automatically and I (the non-morning person) don’t have to remember to do one more thing. And the best trigger that I could think of is when I end Sleep Cycle, the sleep tracking app I use.

I started looking online to see if there was an easy IFTTT integration with Sleep Cycle. (There’s not. Boo.) So I started looking to see if I could stick my Sleep Cycle data elsewhere that could be used with IFTTT. I stumbled across this blog post describing Sleep Cycle -> iOS Apple HealthKit -> UP -> Google Spreadsheet -> Zapier -> Add to Google Calendar. And then I thought I would add another IFTTT trigger for when the calendar entry was added, to then send “waking up” mode to #OpenAPS. But I don’t need all of the calendar steps. The ideal recipe for me then might be Sleep Cycle ->  iOS Health Kit -> UP -> IFTTT sends “waking up mode” -> Nightscout -> my rig. However, I then learned that UP doesn’t necessarily automatically sync the data from HealthKit, unless the app is open. Hmm. More rabbit holing. Thanks to the tweet-a-friend option, I talked to Ernesto Ramirez (long time QS guru and now at Fitabase), who found the same blog post I did (above) and when I described the constraints, then pointed me to Hipbone to grab Healthkit sleep data and stuff it into Dropbox.

(Why Sleep Cycle? It is my main sleep tracker, but there’s IFTTT integration with Fitbit, Jawbone Up, and a bunch of other stuff, so if you’re interested in this, figure out how to plug your data into IFTTT, otherwise follow the OpenAPS docs for using IFTTT to get data into Nightscout for OpenAPS, and you’ll be all set. I’m trying to avoid having to go back to my Fitbit as the sleep tracker, since I’m wearing my Pebble and I was tired of wearing 2 things. And for some reason my Pebble is inconsistent and slow about showing the sleep data in the morning, so that’s not reliable for this purpose. )

Here’s how I have enabled this “wake up” mode trigger for now:

  1. If you’re using Sleep Cycle, enable it to write sleep analysis data to Apple HealthKit.
  2. Download the Hipbone app for iPhone, connect it with your Dropbox, and allow Hipbone to read sleep data from HealthKit.
  3. Log in or create an account in IFTTT.com and create a recipe using Dropbox as the trigger, and Maker as the action to send a web request to Nightscout. (Again, see the OpenAPS docs for using IFTTT triggers to post to Nightscout, there’s all kinds of great things you can do with your Pebble, Alexa, etc. thanks to IFTTT.) To start, I made “waking up” soon a temporary target to 80 for 30 minutes.

Guess what? This morning, I woke up, ended sleep cycle, and ~10-11 minutes later got notifications that I had new data in Dropbox and checked and found “waking up” mode showing in Nightscout! Woohoo. And it worked well for not having a hormone-driven BG rise after I started moving around.

First "waking up" mode in #OpenAPS automation success

Ideally, this would run immediately, and not take 10-11 minutes, but it went automatically without me having to open Hipbone (or any other app), so this is a great interim solution for me until we find an app that will run more quickly to get the sleep data from HealthKit.

We keep finding great ways to use IFTTT triggers, so if you have any other cool ones you’ve added to your DIY closed loop ecosystem, please let me know!

Autosensitivity (automatically adjusting insulin sensitivity factor for insulin dosing with #OpenAPS)

by Dana Lewis

There’s a secret behind why #OpenAPS was able to deal so well with my BGs during norovirus. Namely, “autosensitivity”.

Autosensitivity (or “autosens”, for short hand) is an advanced feature that can optionally be enabled in OpenAPS.

We know how hard it is for a PWD (person with diabetes) to pay attention to all the numbers and all the things and realize when something is “off”. This could be a bad pump site, a pump site going bad, hormones from growth, hormones from menstrual cycles, sensitivity from exercise the day before, etc. So at the beginning of the year, Scott and I started brainstorming with the community about automatically detecting when the PWD is more or less sensitive to insulin than normal, and adjusting accordingly. Building on the success we’d had in DIYPS with fixed “sensitivity” and “resistance” modes, we built the feature to assess how sensitive or resistant the body is (compared to normal), rather than just a binary mode that sets a predefined response.

How OpenAPS calculates autosensitivity/how it works

It looks at each BG data point for the last 24 hours and calculates the delta (actual observed change) over the last 5 minutes. It then compares it to “BGI” (blood glucose impact, which is how much BG *should* be dropping from insulin alone), and assesses the “deviations” (differences between the delta and BGI).

When sensitivity is normal and basals are well tuned, we expect somewhere between 45-50% of non-meal deviations to be negative, and the remaining 50-55% of deviations should be positive. (To exclude meal-related deviations, we exclude overly large deviations from the sample.) So if you’re outside of that range, you are probably running sensitive or resistant, and we want to adjust accordingly. The output of the detect-sensitivity code is a single ratio number, which is then used to adjust both the baseline basal rate as well as the insulin sensitivity factor (and, optionally, BG targets).

Autosens is designed to detect to food-free downward drift, due to basal rates being too high for the current state of the body, and will adjust basals downward to compensate. The other meal-assist related portion of the algorithms do a pretty good job of dealing with larger than expected post-meal spikes due to resistance: auto-sensitivity mostly comes into play for resistance when you’re sick or otherwise riding high even without food.

Does this calculate basals?

No. Similar to everything else in OpenAPS, this works from your established basals – meaning the baseline basal rates in your pump are what the sensitivity calculations are adjusting from. If you run a marathon and your sensitivity is normally 40, it might adjust your sensitivity to 60 (meaning 1u of insulin would drop your BG an expected 60mg/dl instead of 40 mg/dl) and temporarily adjust your baseline basal rate of 1u to .6u/hour, for example.

This algorithm is simply saying “there’s something going on, let’s adjust proportionately to deal with the lower-than-usual or higher-than-usual sensitivity, regardless of cause”. It easily detects “your basals are too high and/or your ISF is too low” or “your basals are too low and/or your ISF is too high”, but actually differentiating between the effect of basal and ISF is a bit more difficult to do with a simple algorithm like this, so we’re working on a number of new algorithms and tools (see “oref0 issue 99” for our brainstorming on basal tuning and the subsequent issues linked from there) to tackle this in the future.

#OpenAPS’s autosensitivity adjustments during norovirus

After I got over the worst of the norovirus, I started looking at what OpenAPS was calculating for my sensitivity during this time. I was especially curious what would happen during the 2-3 days when I was eating very little.

My normal ISF is 40, but OpenAPS gradually calculated the shift in my sensitivity all the way to 50. That’s really sensitive, and in fact I don’t remember ever seeing a sensitivity adjustment that dramatic – but makes sense given that I usually don’t go so long without eating. (Usually when I notice I’m a little sensitive, I’ll check and see that autosens has been adjusting based on an estimated 43 or so sensitivity.)

And in later days, as expected when sick, I shifted to being more resistant. So autosens continued to assess the data and began adjusting to an estimated sensitivity of 38 as my body continued fighting the virus.

—

It is so nice to have the tools to automatically make these assessments and adjustments, rather than having to manually deal with them on top of being sick!

 

Half life

by Dana Lewis

I have now lived with diabetes for more than half of my life.

That also means I have now lived less than half of my life without diabetes.

This somehow makes the passing of another year living with diabetes seem much more impactful to me. Maybe not to you, or to someone else with a different experience of living with diabetes and a different timeline of life before and after diagnosis…but to me this is a big one.

I’m happy to have context, though, to help me keep things in perspective. For example, I’ve now lived with a closed loop artificial pancreas (or automated insulin delivery) system for almost two full years.

(That’s almost as significant a marker of a “with” vs. “without” comparison as living “with” vs. “without” diabetes.)

And because I ended up with type 1 diabetes, I found out that doing things for other people and the communities you’re a part of is a powerful way to help yourself, both in the short term and the long term. That’s what drove me to figure out a way to take #DIYPS closed loop and make it something open source. And by doing that, I learned so much more about open source, and have been able to partner with incredible people innovating in hardware and software. These collaborations have resulted in an incredibly rich community of passionate people I like to call #OpenAPS-ers.

While #OpenAPS is by no means a cure, and no artificial pancreas will be a cure, they provide an immeasurably improved quality of life that a lot of us didn’t realize was possible with diabetes. Someone told me he can get the same results for his child living with diabetes, but with #OpenAPS it requires about 85% less work. And given the enormous time and cognitive burden of diabetes, this is a HUGE reduction.

And now doors are opening for us collectively to make even more of a significant impact on the diabetes community, and our fellow patient communities. Yesterday, while at the White House Frontiers conference, NIH Director Dr. Francis Collins was in the audience during my panel. At the end of the day, he stopped me to ask questions about my experiences and perspective on the FDA and what we need from the government. I was able to talk with him about the need for FDA & other parts of the government to help foster and support open source innovation. We talked about the importance of data access for patients, and the need for data visibility on commercially approved medical devices.

Showing former NIH Director Francis Collins my OpenAPS rig and talking about data interoperability.

This is not just a need of people with diabetes (although it’s certainly very applicable for all of the manufacturers with pipelines full of artificial pancreas products): these are universal needs of people dealing with serious health conditions.

Given what I heard yesterday, it’s working. The #WeAreNotWaiting spirit is infusing our partners in these other areas. We are planting seeds, building relationships, and working in collaboration with those at the FDA, NIH, HHS in addition to those in industry and academia. I know they were working toward these same goals before, but social media has helped raise up our collective voices about the burning need to make things better, sooner, for more people.

So if I have to live the rest of my life at a ratio where more than half of it has been spent living with diabetes, I look forward to continuing to work to get to an 85% reduction in the burden of daily life with diabetes for everyone.

 

What a FDA approved commercial hybrid closed loop artificial pancreas system (670G) means for #OpenAPS

by Dana Lewis

You probably heard that a commercial hybrid closed loop (the 670G) has been approved by the U.S. FDA and, like everyone else, are wondering what that means for #OpenAPS.

First commercial AID finally became available in 2016

First, here’s our initial reaction:

Thoughts-on-commercial-AID-DanaMLewis

And here are some longer form thoughts:

  • Yes, this is exciting. FDA moved months more quickly then expected (hmm, we are sensing a theme when the #WeAreNotWaiting community is involved ;)) to get this tech approved. And as we’ve experienced (check out this self-reported outcomes study with better outcomes than the pivotal trial for this new device), the results of using a hybrid closed loop are outstanding. It’s disappointing that they won’t be ready to ship until Spring 2017, but…
  • …This means the company has time to work on user guides and usability. As we’ve told every device company we’ve encountered, we (the #OpenAPS community) are happy to share everything we’ve learned. And we have learned a lot, including what it takes to trust a system, how much info is needed to help determine if additional human action is needed, what to do in all kinds of real-world situations, and more. We hope the companies continue to work with people with diabetes who have experience with this technology from both clinical trials and the DIY world, where we’ve racked up 350,000+ hours with this type of technology. Because setting expectations with users for this technology will be key for successful and sustained adoption.

This doesn’t really mean anything for #OpenAPS, though. The first generation of AP technology is similar to #OpenAPS in that it’s a hybrid closed loop that still requires the human to input carbs into the system, but it unfortunately has a set point that can not be adjusted below 120mg/dl.  For many people, this is not a big deal. But for others, this will be a deal breaker. For DIYers, that lack of customization will likely be frustrating. And for many families, the lack of remote data visualization may be another deal breaker. And, like with all new technology and devices, getting this stuff covered by insurance may be an uphill battle. So while optimistically this enables many people in the U.S. to finally access this technology (yay) without having to DIY, it won’t necessarily be truly available to everyone from a cost or access perspective for many years to come. So #OpenAPS and other DIY technology may still be needed from a cost/access perspective to continue to help fill gaps compared to current status quo with basic, non-connected diabetes devices (i.e. standalone pump and CGMs).

I also know that many of the parents of kids with T1D are disappointed, because the initial approval is for kids 14+, and it even notes that the system is not recommended for kids <7 or those taking less than 8u of insulin every day (usually young kids). I asked, suspecting it was related to occlusion, but it sounds more like they just don’t have enough data to say for sure that the system is safe with that small amount of insulin, and they’re working on additional studies to get data in that area.

Ditto, too, for more studies allowing different set points. They stuck with a 120mg/dl set point in order to speed to approval, but fingers crossed they get other studies done and new approvals from FDA before this device ships in the spring – that would be awesome. And I was glad to hear that they do have an “exercise” target of 150. That’s a bit of good…but I’m still hesitant that it is enough. From my personal experience knowing net IOB (here’s why net IOB matters) an hour before and when starting exercise is required information to help me decided whether or not I will need carbs to prevent lows during exercise. I don’t think this device will report on net IOB, but I admittedly haven’t seen the device and hopefully I’ll be proved wrong and the data available will be good enough for this purpose!

So in summary: this is good news. But we still need more FDA approved commercial options, and even with a single “commercial approved option”, it’s still ~6+ months away from reaching the hands of people with diabetes…so we as a #WeAreNotWaiting movement continue to have work to do to help speed up the processes for getting enhanced diabetes technology approved and available on the market, with access to view data the ways we need it.

—

*(Yes, in the title of the post I called it a commercial hybrid closed loop artificial pancreas system. It’s a hybrid closed loop, as is #OpenAPS, but it’s also on the road/part of the suite of more complex artificial pancreas technology. I realize to many PWDs “artificial pancreas” means a lot of different things. Quite certainly, regardless of definition, an artificial pancreas or hybrid closed loop still requires a lot of work. It’s not a cure by any stretch of the imagination. But it’s easy for the media to describe it as an AP, and I also find it a lot easier to describe the small device accompanying my pump when strangers ask as an “artificial pancreas” followed by an explanation rather than saying “hybrid closed loop”.

If anything, I think having the media broadly categorize it as an AP will encourage the diabetes community to ask more questions about what exactly this tech does, leading to greater understanding and better expectations about what the device will/won’t be able to do. So this may result in a good thing.)