What we learned from “Convening The Center”

Like our previous RWJF-funded project, “Opening Pathways”, where we took time to reflect at the end of the project and also openly shared our grant-end narrative report that we submit to RWJF, John Harlow and I wanted to also share the grant-end narrative report that we created for RWJF for the Convening The Center project. The questions are based on their template. If you have any other questions about the Convening The Center project (background about it here), please do ask!

  1. What was the goal of your project? Do you have measures of your performance?

Our original goal was to convene THE center of healthcare, which means patients and caregivers and those working to effect change in the healthcare system from the ‘outside’. We originally planned for an all-expenses paid in-person physical meeting, gathering people from within the U.S. at a central location that would be relatively easy (within 2-3 hours of flying) to travel to for most individual participants. We aimed to gather 25 participants.

However, we were awarded our grant in December 2019 and saw the impact of COVID-19 early on in our communities (especially PI Lewis’s community of Seattle, where COVID-19 was first detected in the US in late January/early February 2020), and knew we would need to postpone the physical meeting from 2020 to at least spring 2021 at the earliest. As months passed, we realized the pandemic would not in fact be ‘over’, and debated between cancelling the grant or converting to a digital experience. We did not want to lose the opportunity to gather this type of community, and chose to switch to a digital meeting.

We spent significant amounts of time considering how to achieve the goals of our meeting (bringing together 25 people who didn’t necessarily know each other or have shared goals, beyond a broad overarching goal of improving healthcare, and giving them space to connect without forcing an agenda upon them). We ultimately decided to make our digital meeting a three-phase “experience:”

  • The first phase would involve one-to-one conversations that would allow us to deeply listen and understand the perspectives of each participant. We would use a visual notetaker to illustrate their story and work as a way to reflect back what we heard, as well as offer the artwork as a gift to participants as a thank you for sharing their experiences with us. These conversations would then shape the following phases.
  • The second phase was small-group conversations of up to 8 people maximum, which we chose based on a combination of availability and ensuring a mixed group of participants where there wasn’t necessarily one person or personality that would dominate a group conversation. There was no agenda, but we used Google Slides with some introductory activities to help people introduce themselves or their work in a non-threatening way, and facilitated topics of conversation for the group to dive into. We had four total groups in phase 2. We again had visual notetaking to represent each group’s conversation.
  • The third phase was a single meeting with all 25 participants present. We chose a mix of small group breakouts, based on thematic topics that were discussed in phase 2 and voted upon by participants; as well as providing a small group mix based on people they had not yet met in previous groupings; and also small groups based on affinity groups that the PI/Co-PI selected based on what we learned of people’s work in phase 1-2. After the rounds of breakouts, the group returned together for a discussion with all 25 participants based on whatever topic they wished to discuss.

And, at the end of the project we had $9,000 USD remaining as a result of the pivot to a digital meeting. We decided to select nine individuals (through additional public recruitment) for “internet scholarships”, to continue to address the needs of this community. We successfully selected 9 recipients who each accepted the scholarship.

The project overall was a success.

  • We were able to convene 25 participants from around the world and allow them to discuss whatever topics were most important to them.
  • Because we went digital/virtual, we were able to facilitate participation from non-US based participants which greatly enriched the discussions.
  • Participants consistently communicated surprise and delight after each phase of the project regarding how well they felt listened to, respected, and treated during the experience.
  • We used a visual notetaker as a subcontractor, and her work was a critical factor of our success. Rebeka Ryvola is an experienced conference visual notetaker and artist, and although she had not previously worked in healthcare, her ability to listen to a deeply technical healthcare conversation and reflect high level themes from individual stories as well as across a diverse group of participants is unparalleled. Her art as an artifact of each discussion was critical for allowing participants to feel heard and respected, as well as providing a way to introduce themselves to each other within the cohort.
  • Rebeka’s art for the individuals in phase 1 as well as the Phase 2 and Phase 3 pieces of art is already being widely shared and touted.
  • All 9 selected recipients of the internet scholarships accepted them.
  1. Do you have any stories that capture the impact of this project?

    One of our goals was to pay people for their time. Patients and caregivers are seldom paid for their time and expertise, although they offer invaluable expertise and solutions for improving healthcare.One individual, a parent with their own health situation as well as a parent of children with their own health situations, had served on a hospital advisory committee and numerous projects. However, until Convening The Center, this individual had never been paid for their time or work. They mentioned this numerous times throughout the project, almost in disbelief, that they were being paid for the first time for this type of time commitment. It was almost embarrassing to us for being the first people to pay them for their time on a project, although we are grateful for the resources of this grant that enabled us to pay each participant for their time.

    In general, while we as PI/Co-PI know the power of bringing people together through social media and virtually, and we expected many of the participants (by virtue of finding this project) were already connected to numerous patient communities or organizations, we were surprised by the number of comments that participants made about the power of this convening. Two quotes stuck out to us, from an experienced patient advocate and from a newer patient advocate:

  • “Getting to meet you all, truly made me feel not alone in what often feels overwhelming and hard.”
  • [It was powerful] “bringing so many people from all corners together. I’m still building my confidence as a speaker and these opportunities to connect in a Round Robin sort of way was impactful, for someone who is still finding their voice as an advocate”(See Appendix at end for additional stories!)
  1. Did RWJF assist or hinder your project in any way?

RWJF assisted immensely by allowing us to submit a re-budget request and allowing us to shift to a virtual meeting while maintaining the existing level of budget. This was significant, because had we originally submitted a proposal for a virtual meeting, I think the grant would have been rejected/not awarded. Or, it may have been awarded with a significantly lower budget amount.

However, keeping the level of budget allowed us to spend significant amounts of time designing an inclusive, immersive digital experience that allowed us to bridge the participants’ physical worlds with our virtual meeting. We were able to do this by designing a “kit” to send to each participant, including international participants, with thoughtfully designed and curated items to aid them in their participation in this project. A typical virtual meeting would not have supported the budget for this type of ‘kit’ nor the PI/Co-PI’s increased time to design a thoughtful, effective, inclusive virtual meeting.

It also allowed us to facilitate the inclusion of participants from around the world. With a physical meeting, we were limited in budget to only US participants because of the travel cost variance with international travel. We were able to include participants from Costa Rica, Sweden, India, Pakistan, as well as across the US. We were also able to reach more diverse, under-resourced (including under-funded and under-included), and often minority perspective community members who maybe would not have been able to participate without it being virtual – even with us paying for their time and travel, because of their other family or community commitments.

Finally, because we went to a virtual experience and chose to do three ‘phases’ to build up to the final meeting, rather than a one-day in person meeting, we were able to get to know each participant and build trust over the phases that would not have happened by dropping 25 people into a room together for a physical meeting.

As a result, the permission and support with the same budget to shift to a virtual meeting greatly enriched the experience beyond what we would have originally predicted, and we hope RWJF considers this moving forward when thinking about facilitating similar gatherings of communities or projects.

Specifically within RWJF, our project manager Paul Tarini was helpful as always throughout the project. When we chose to pivot to a virtual meeting, we asked him for his perspective on thoughtful digital gatherings, and he shared not only his own experiences but also introduced us to a number of other RWJF grantees or collaborators to help us research best practices for online engagement for conferences and meetings. Many of the ideas we learned from collaborators such as New Public ended up shaping the phases of our work.

  1. If the project encountered internal or external challenges, how did they affect the project and how did you address them? Was there something RWJF could have done to assist you?

Our main challenges were the COVID-19 pandemic and the timing overall for our grant, because the primary goal was hosting a physical in-person meeting, Per the above section, RWJF assisted us by allowing us to re-budget from a physical in-person meeting to a digital gathering, while permitting the same overall level of budget. This was crucial for our success, because simply hosting a one-time 25 person meeting would not have achieved the goal without the additional design work that was done.

  1. Has your organization received funding from other foundations, corporations or government bodies for the project RWJF has been supporting?

No.

  1. When considering the design and implementation of this project, what lessons did you learn that might help other grantees implement similar work in this field?

We learned quite a lot regarding designing digital experiences that we hope other grantees will be able to leverage, and we hope RWJF will take this feedback into consideration and support other future projects that host virtual convenings.

For example, we learned that it takes more time to design impactful virtual gatherings that are not ‘just another zoom’. It takes design of the meeting itself with a clear ‘run of show’ or agenda, as well as clear pre-communication to participants about what to expect and how the meeting or gathering will go. In some cases, such as for our project, we also found it necessary to break the gathering up into multiple stages, to allow us to get to know participants and build trust to have the open, thoughtful discussions that emerged in phase 2 and 3. Had we simply plopped 25 people together in a virtual meeting as a one-off, it would not have been successful. We also were cognizant of the demand on participants in terms of overall time commitment – people don’t have the stamina for more than 2 hours on a video call – and the demands on internet bandwidth and personal energy for requesting a video call for that time period. We chose max 2 hours for each phase, and encouraged people to choose for themselves whether they had video on or off. We also designed activities to facilitate trust and comfort in the digital environment.

As a result, we learned that going from individual conversations to small group to larger conversations worked well for establishing safe spaces for open conversations. This also enabled relationships to begin growing throughout the project and not only after the ‘main event’ of phase 3. This facilitated the network within the cohort that began to grow as a community. You can’t force a community by dumping people in a place, but you can create a space and facilitate interactions that lead to relationship and network growth, and ultimately a community did evolve.

We also ended up developing a physical ‘kit’ to send to participants. It included a variety of useful items (such as a device cleaning cloth and a device stand, since many people are spending increased amounts of time on devices during the pandemic as well as we were asking them to spend more time on devices for this project). We also custom designed a few special items to honor people’s participation in the project. One of these items was a pack of playing cards that they could give to family or the people supporting them to help them be able to make the space for participating in the project. We also created a custom CTC keychain and provided several additional keychains that each participant could gift to others to honor other advocates, clinicians, and “doers” in the healthcare space who have helped them in their journey or that they want to honor their work. We hoped these keychains would also serve as a memento of their time in the project and be something they could physically hold in the future to give them strength, if they need it. This kit also included a whiteboard and markers, which we used in a variety of ways throughout the project including holding up to show something on screen, which we aimed to tie the offline/online experiences together. We didn’t want to send people “junk” “swag” that would end up in a landfill, and so we included things that we thought would be used by everyone in the cohort and had meaningful ties to the project.

Overall, one of our key design principles was to consistently signal that our gathering was and would be different from random meetings and conferences where people show up, say the same thing, and leave unchanged. We aimed to achieve this by doing everything different, from paying participants, to surprising people with their CTC ‘kit’, and to providing the visual note art as a gift after phase 1 in addition to doing visual notetaking from phase 2 and phase 3 as well. We consistently heard surprise and delight from participants beyond what you would typically receive from participants at a meeting or a conference, so we believe all of these elements of doing a gathering differently were successful, and that “surprise and delight” is an effective design principle for building relationships, creating spaces, and encouraging participation.

  1. What impact do you think the project has had to date?

The project was impactful in and of itself by successfully gathering 25 diverse individuals who have not previously had the opportunity to gather without an agenda forced upon them. Additionally, we were successful by paying each and every participant for their time. (Several individuals had never been paid before for their contributions to meetings, conferences, advisory committees, and/or research projects.)

In addition, it was successful for creating connections to enable network development and growth of relationships for people who don’t have traditional ‘professional development resources’ but benefit greatly from seeing other people ‘like them’ who are working to improve healthcare from the outside.

Through this project, people were able to surface similar challenges and experiences among individuals who felt isolated and ‘lonely’ in their work. They also were able to recognize shared challenges and solutions across disease areas, when they previously were not aware of resources. One example is a participant who shared research advocate training program materials from a specific cancer community, with other participants planning to leverage or mirror those resources in other disease spaces.

Additionally, participants began recognizing similarities across disease communities, with consistent gaps around areas such as transitioning out of pediatric to (young) adult care; lack of inclusivity with established advocacy organizations and online communities; and challenges with interacting with healthcare providers.

As PI/Co-PI we have also developed a novel framework for mapping the efforts of individuals by convening the center of health and healthcare. This is an innovative framework that assesses a spectrum of patient experiences based on what patients do when they go beyond navigating their personal or individual level of lived healthcare experiences and transition toward community or systemic level involvement. We have written up this framework and the results of thematic discussions from CTC in a research article, which we plan to submit to a peer-reviewed publication (and then share here soon!). We hope to inspire further work with this proposed model for facilitating improved matching between individuals and their current or future levels of interest and involvement with researchers, advocacy organizations, and other opportunities.

  1. What are post-grant plans for the project, if it does not conclude with the grant?

No specific plans, as the project technically concludes with the end of the grant.

However, many individuals who participated in this project are planning to work together in the future. For example, several post-meeting meetings have already happened among small groups within the cohort. One such meeting involved a discussion around patient-led research publications and strategy for utilizing blog posts and mainstream media compared to academic journals and traditional research conferences as methods of dissemination of patient community knowledge.

  1. With a perspective on the entire project, what were its most effective communications and advocacy approaches, its key publications, and its national/regional communications activities?

Our most effective communication was through social media. We publicized the project via a blog post shared across Facebook, Twitter, and LinkedIn. It was successful as measured by receiving applications from >60% of individuals that the PI did not recognize the name. From within the selected cohort, there were ~75% unknown participants to the PI/Co-PI, which indicated success in outreach to new networks and communities.

We believe the most effective advocacy approach was empowering individual participants. This project was not about name recognition of the project itself, but we believe by surprising and delighting participants and giving them a positive experience in the project, the ripple effects of this project and RWJF’s funding will continue to be felt for years to come.

APPENDIX:

We shared a draft version of this report with participants of the Convening The Center Cohort, asking for any additional feedback and stories we should include. The following stories and comments were shared as a result:

  1. “I appreciate how you have effectively captured the essence of our Convening The Center experiences. Surprise and delight are positive reinforcements and pragmatic concepts that can catalyze trust leading to trustworthiness; and overall the gatherings were very beneficial in developing a community of individuals who had similar interests with advancing patient and caregiver engagement. I do believe we were each pleased with the concrete extras including; fair compensation, bio-‘art’ifacts, CTC shareable reminders and reflected stories that we each will carry forward. The three tiered model was an effective method allowing for Conveners to listen and learn from each other.”
  2. “I think you captured the essence of my personal experience (I can’t/won’t/shouldn’t speak to the experiences of the others). The only thing I have to say is a hearty thank you to RWJF for allowing this project to proceed virtually. It was an enriching experience, filled with far more diversity (as you pointed out).”
  3. “Many thanks to CTC & grantor RWJF for allowing the power of the pivot to take place!

    Do you have any stories that capture the impact of this project?

    I would add that this experience presented an opportunity to amplify voices of women of color. This group was accepting of my perspective and participating in it further ignited my passion to embrace my efforts as an advocate for an underserved patient population.

    What impact do you think the project has had to date?

    The project inspired me to challenge old ideas of self-doubt and redefine what advocacy means to me. Since the convening, I’ve been empowered to participate in projects that bring forth HPV, Cervical, & Gynecologic Cancer awareness for the Black-Hispanic population. This was a huge moment of growth and development for someone who struggles with social anxiety.

    When considering the design and implementation of this project, what lessons did you learn that might help other grantees implement similar work in this field

    I just wished to add, if others would like to duplicate a similar idea in a virtual capacity, it is important to have systems in place that allow for free flowing communication. This was my first time using Slack and it performed well in my opinion. The platform could be accessed on both pc or mobile device.

    I always felt connected & well-informed. The [Slack] app made it easy to contribute to the discussion with the group throughout the entire project. It also presented the opportunity for members to learn more about one another through introductions and our artwork; all prior to meeting each other virtually.”


A huge thank you to each participant who was a part of Convening The Center!

 

New Convening The Center Update – Help Us Find People Who Could Use Internet Scholarships to Do Good In Healthcare?

You may have previously read a blog post about Convening The Center, a RWJF-grant-funded project with the aim of bringing together 25 diverse individuals who are working to change healthcare in nontraditional ways. The main part of the CTC project has finished (more about that soon!), but we also realized that we had a little bit of budget left over from the project, and pitched to RWJF a new plan to use the remaining funds.

We want to give individuals working to make a difference in health and healthcare – and the health of their (online, geographic, or disease) communities – by providing 9 internet scholarships of $1,000 USD each. This is estimated to cover about a year’s worth of internet access for each individual. Individuals who are applying should be able to articulate their past, current, or future efforts as it relates to making a difference in health/care.

There are no strings attached to this ‘internet scholarship.’ You don’t have to do anything particular, or commit to any projects if you’re selected, other than write us a few (say, 250 or so) words within the next year to let us know what it meant to you to have your internet paid for. That’s it. This feedback (which can be given privately to us, or posted publicly – your call) is the only requirement for receiving these funds.

Can you help us find people who could use Internet scholarships to do good in healthcare?

Why are we doing this?

We learned (and re-learned) from working with the cohort from the original CTC project that internet access is something many of us take for granted, and that we shouldn’t. Many of us may assume, from a privileged position, that access to high speed internet is table stakes and that everyone has it, so when invited to take a seat at the table, anyone invited could get there. But that’s not the case.

This is relevant to the space we are working in with CTC, where we are seeking to support patients (people living with diseases) or carers who are working to improve healthcare and their communities, often from non-resourced settings. The ability to afford high-speed internet access therefore might be a barrier for enabling patients/carers to take a seat at the table, when invited – or from building their own table.

We realize that $9,000 won’t solve all the problems of equitable access and facilitate online participation of everyone who needs it. But it’s a start, and could be the thing that makes a difference for 9 individuals, and it’s the best use we can envision for this remaining budget.

So our ask, if you’re reading this:

  • Please consider nominating someone or applying (self-nominating) for the Convening The Center Internet Scholarship, by filling out this Google form by November 14.
  • Please share this blog post (https://bit.ly/CTC-Internet-Scholarships) with your online and offline networks, including with those you know in rural settings where internet cost may be a bigger barrier.

John and I are excited to facilitate this last use of our CTC project budget. We will close the nomination Google form on November 14; select recipients by the end of November; and aim to provide payments of the CTC Internet Scholarships (administered by Trailhead Institute, our fiscal sponsor) in early December (all 2021). Within the next year after we receive feedback from all participants, we will also (anonymously, at an aggregate level) share the feedback and what we learned from using the remaining budget funds for this purpose with the broader community, to help inform others who are looking to create similar initiatives in the future.

In summary:

  • Who: People who are looking to make a difference in health/care who might benefit from having a year’s worth of internet costs covered
  • What: Up to 9 individuals will receive $1,000 USD, estimated to cover a year’s worth of typical high speed internet plans.
  • How: fill out this Google form and nominate yourself or someone else. Multiple nominations are welcome, there is no limit.
  • When: Please apply by November 14, and recipients will be selected in November 2021.

Designing digital interactive activities that aren’t traditional icebreakers

A participant from Convening The Center recently emailed and asked what technology we had used for some of our interactive components within the phase 2 and 3 gatherings for the project. The short answer was “Google Slides” but there was a lot more that went into the choice of tech and the design of activities, so I ended up writing this blog post in case it was helpful to anyone else looking for ideas for interactive activities, new icebreakers for the digital era, etc.

Design context:

We held four small (8 people max) gatherings during “Phase 2” of CTC and one large (25 participants) gathering for “Phase 3”, and used Zoom as our videoconference platform of choice. But throughout the project, we knew we were bringing together random strangers to a meeting with no agenda (more about the project here, for background), and wanted to have ways to help people introduce themselves without relying on rote introductions that often fall back to name, title/organization (which often did not exist in this context!), or similar credentials.

We also had a few activities during the meeting where we wanted people to interact, and so the “icebreakers” (so to speak) were a low-stress way to introduce people to the types of activities we’d repeat later in the meeting.

Technology choice:

I’ve seen people use Jamboard (made by Google) for this purpose (icebreakers or introductory activities), and it was one that came to mind. However, I’ve been a participant on a Jamboard for a different type of meeting, and there are a few problems with it. There’s a limit to the number of participants; it requires participants to create the item they want to put on the board (e.g. figure out how to add a sticky note), and the examples I’ve seen content-wise ended up using it in a very binary way. That in some cases was due to the people designing the activity (more on content design, below), but given that we wanted to also use Google Slides to display information to participants and also enable notetaking in the same location, it also became easy to replicate the basic functionality in Google Slides instead. (PS – this article was helpful for comparing pros/cons of Jamboard and Google Slides.)

Content choices:

The “icebreakers” we chose served a few purposes. One, as mentioned above, was familiarizing people with the platform so we could use it for meeting-related activities. The other was the point of traditional icebreakers, which is to help everyone feel comfortable and also enable people to introduce themselves. That being said, most of the time introductions rely on credentials, and this was specifically a credential-less or non-credential-focused gathering, so we brainstormed quite a bit to think of what type of activities would allow people to get comfortable interacting with Google Slides and also introduce themselves in non-stressful ways.

The first activity we did for the small groups was a world map image and asked people to drag and drop their image to “if you could be anywhere in the world right now, where would you be?”. (I had asked all participants to send some kind of image in advance, and if they didn’t, supplied an image and told them what it was during the meeting.) I had the images lined up to the side of the map, and in this screenshot you can see the before and after from one of the groups where they dragged and dropped their images.

Visual of a world map with images representing individuals and different places they want to be in the world

The second activity was a slide where we asked everyone to type “one boring or uninteresting fact about themselves”. Again, this was a push back against traditional activities of “introduce yourself by credentials/past work” that feels performative and competitive. I had everyone’s names listed on the slide, so each could type in their fact. It ended up being a really fun discussion and we got to see people’s personalities early on! In some cases, we had people drop in images (see screenshot of example) when there was cross-cultural confusion about the name of something, such as the name of a vegetable that varies worldwide! (In this case, it was okra!)

List of people's names and a boring fact about themselves

We also did the same type of “type in” activity for “Ask me about my expertise in..” and asked people to share an expertise they have personally, or professionally. This is the closest we got to ‘traditional’ introductions but instead of being about titles and organizations it was about expertise in activities.

Finally, we did the activity most related to our meeting that I had wanted people to be comfortable with dragging and dropping their image for. We had a slide, again with everyone’s image present, and a variety of types of activities listed. We queried participants about “where do you spend most of your time now?”. Participants dragged and dropped their images accordingly. In some cases, they duplicated their image (right click, duplicate in Google Slides) to put themselves in multiple categories. We also had an “other” category listed where people could add additional core activities.

Example of slide activity where people drag their image to portray activities they're doing now and want to do in the future

Then, we had another slide asking where do they want to spend most of their time in the future? The point of this was to be able to switch back and forth between each slide and visualize the changes for group members – and also so they could see what types of activities their fellow participants might have experience in.

Some of these activities are similar to what you might do in person at meetings by “dot voting” on topics. This type of slide is a way to achieve the same type of interactivity digitally.

Facilitating or moderating these types of interactive activities

In addition to choosing and designing these activities, I also feel that moderating or facilitating these activities played a big role in the success of them for this project.

As I had mentioned in the technology choice section,  I’ve previously been a participant in other meeting-driven activities (using Jamboard or other tech) where the questions/activities were binary and unrelated to the meeting. Questions such as “are you a dog or cat person? Pick one.” or “Is a hot dog a sandwich?” are binary, and in some cases a meeting facilitator may fall into the trap of then ascribing characteristics to participants based on their response. In a meeting where you’re trying to use these activities to create a comfortable environment for participation amongst virtual strangers…that can backfire and actually cause people to shut down and limit participation in the meeting following those introductory activities.

As a result of having been on the receiving end of that experience, I really wanted to design activities with relevance to our meeting (both in terms of technology used and the content) as well as enough flexibility to support whatever level of involvement people wanted to do. That included being prepared to move people’s images or type in for them, especially if they were on the road and not able to sit stationary and use google slides. (We had recommended people be stationary for this meeting, but knew it wasn’t always possible, and were prepared to still help them verbally direct us to move their image, type in their fact, etc. This also can be very important for people with vision impairment as well, so be prepared to assist people in completing the activities for whatever reason, and also to verbally describe what is going on the slides/boards as people move things or type in their facts. This can aid those with vision impairment and also those who are on the go and can’t look at a screen during the meeting for whatever reason.)

One other reason we used Google Slides is so we’d end up with a slide for each breakout group to be able to take notes, and a “parking lot” slide at the end of the deck for people to add questions or comments they wanted to bring back up in the main group or moving forward in future discussions. Because people already had the Google Slide deck open for the activity, it was easy for them to scroll down and be in the notetaking slide for their breakout group (we colored the background of the slides, and told people they were in the purple, blue, green, etc. slides to make it easier to jump into the right slide).

One other note regarding facilitation with Zoom + Google Slides is that the chat feature in Zoom doesn’t show previous chat to people who join the Zoom meeting after that message is sent. So if you want to use Zoom chat to share the Google Slides link, have your link saved elsewhere and assign someone to copy and paste that message into the chat frequently, so all participants have access and can open the URL as they join the meeting. (This also includes if someone leaves and re-enters the meeting: you may need to re-post the link yet again into chat.)

TLDR, we used Google Slides to facilitate meeting note taking, digital “dot voting” and other interactive icebreaker activities alongside Zoom.

Update – 2021 Convening The Center!

2020 did not go exactly as planned, and that includes Convening the Center (see original announcement/plan here), which we had intended to be an awesome, in-person gathering of individuals who are new or have previous experience working to improve healthcare through advocacy, innovation, design, research, entrepreneurship, or some other category of “doing” and “fixing” problems they see for themselves and their community. But, as an early “I see COVID-19 is going to be a problem” person (see this post Scott and I posted March 7 begging people to stay home), by early February I was warning my co-PI and RWJF contacts that we would likely be postponing Convening the Center, and by May that was pretty clear. So we decided to request (and received) an extension on our grant from RWJF to enable us to push the grant into 2021…and ultimately, ::waves hand at everything still going on:: decided to shift to an all-virtual experience.

I’ll be honest – I was a little disappointed! But now, after several more months of work with John (Harlow, my Co-PI), I’m now very excited about the opportunities an all-virtual experience for Convening the Center will bring. First and foremost, although we planned to pay participants for ALL travel costs, hotel, food, AND for their time, I knew there would likely be people who would still not be able to travel to participate. I am hoping with a virtual experience (where we still pay people for their time!), the reduced time commitment to participate will enable those people to potentially participate.

Secondly, we’ve been thinking quite a bit about the design of virtual meetings and gatherings and have some ideas up our sleeve (which we’ll share as we finish developing them!) about how to achieve the goals of our gathering, online, without triggering video conference fatigue. If you’ve had any fantastic virtual experiences in 2020 (or ever), please let us know what they were, and what you loved (or what to avoid!), so that we can draw on as many inputs as possible to design this virtual experience.

Here’s what Convening the Center will now look like:

  • Starting now: recruitment. We are looking to solicit interest from individuals who are new or have some experience working to change or improve health, healthcare, communities, etc. If that’s you, please self-nominate yourself here, and/or please also consider sharing this with your communities or a friend from another community!
  • January: we will reach out to nominees with another short form to gather a bit more information to help us create the cohort.
  • Early February: we will notify selected participants.
  • February: Phase 1 (2 hours scheduled time commitment from participants, plus some asynchronous opportunities)
  • April: Phase 2 (2-4 hour schedule time commitment from participants, plus some asynchronous opportunities)
  • June: Phase 3 (2-4 hour scheduled time commitment from participants, plus some asynchronous opportunities)

We’ll be sharing more in the future about what the “phases” look like, and this virtual format will allow us to also invite participation from a broader group beyond the original cohort of participants. Stay tuned!

Again, here is the nomination link you can self-nominate or nominate others at. Thanks!

Nominate someone you know for Convening The Center!

How to deal with wildfire smoke and air quality issues during COVID-19

2020. What a year. We’ve been social distancing since late February and being very careful in terms of minimizing interactions even with family, for months. We haven’t traveled, we haven’t gone out to eat, and we basically only go out to get exercise (with a mask when it’s on hiking trails/around anyone) or Scott goes to the grocery store (n95 masked). We’ve been working on CoEpi (see CoEpi.org – an open source exposure notification app based on symptom reports) and staying on top of the scientific literature around COVID-19, regarding NPIs like distancing and masking; at-home diagnostics like temperature and pulse oximetry monitoring, prophylactics and treatments like zinc, quercetine, and even MMR vaccines; and the impact of ventilation and air quality on COVID-19 transmission and susceptibility.

And we live in Washington, so the focus on air quality got very real very quickly during this year’s wildfire season, where we had wildfires across the state of Washington, then got pummeled for over a week with hazardous levels of wildfire smoke coming up from Oregon and California to cover our existing smoke layer. But, one of our DIY air quality hacks for COVID-19 gave us a head start on air quality improvements for smoke-laden air, which I’ll describe below.

Here are various things we’ve gotten and have been using in our personal attempts to thwart COVID-19:

  • Finger pulse oximeter.
    • Just about any cheap pulse oximeter you can find is fine. The goal is to get an idea of your normal baseline oxygen rates. If you dip low, that might be a reason to go to urgent care or the ER or at least talk to your doctor about it. For me, I am typically 98-99% (mine doesn’t read higher than 99%), and my personal plan would be to talk to a healthcare provider if I was sick and started dropping below 94%.
  • Thermometer
    • Use any thermometer that you’ll actually use. I have previously used a no-touch thermometer that could read foreheads but found it varied widely and inconsistently, so I went back to an under the tongue thermometer and took my temperature for several months at different times to figure out my baselines. If sick or you have a suspected exposure, it’s good to be checking at different times of the day (people often have lower temps in the morning than in the evening, so knowing your daily differences may help you evaluate if you’re elevated for you or not).
    • Note: women with menstrual cycles may have changes related to this; such as lower baseline temps at the start of the cycle and having a temperature upswing around or after the mid-point in their cycle. But not all do. Also, certain medications or birth controls can impact basal temperatures, so be aware of that.
  • Originally, n95 masks with outlet valves.
    • Note: n95 masks with valves cannot be used by medical professionals, because the valves make them less effective for protecting others. (So don’t freak out at people who had a box of valved n95 masks from previous wildfire smoke seasons, as we did. Ahem.) 
    • We had a box we bought after previous years’ wildfire smoke, and they work well for us (in low-risk non-medical settings) for repeated use. They’re Scott’s go-to choice. If you’re in a setting where the outlet valve matters (indoors in a doctor’s/medical setting, or on a plane), you can easily pop a surgical/procedure mask over the valve to block the valve to protect others from your exhaust, while still getting good n95-level protection for yourself.
    • They were out of stock since February, but given the focus on n95 without valves for medical PPE, there have been a few boxes of n95 masks with outlet valves showing up online at silly prices ($7 per mask or so). But, kn95’s are a cheaper per mask option that are generally more available – see below.
    • (June 2021 note – they are back to reasonable prices, in the $1-2 range per mask on Amazon, and available again.)
  • kn95 masks.
    • kn95 masks are a different standard than US-rated n95; but they both block 95% of tiny (0.3 micron) particles. For non-medical usage, we consider them equivalent. But like n95, the fit is key.
    • We originally bought these kn95s, but the ear loops were quite big on me. (See below for options if this is the case on any you get.) They aren’t as hardy as the n95s with valves (above); the straps have broken off, tearing the mask, after about 4-5 long wears. That’s still worth it for them being $2-3 each (depending on how many you buy at a time) for me, but I’d always pack a spare mask (of any kind) just in case.
      • Option one to adjust ear loops: I loop them over my ponytail, making them head loops. This has been my favorite kn95 option because I get a great fit and a tight seal with this method.
      • Option two to adjust ear loops: tie knots in the ear loops
      • Option three to adjust ear loops: use things like this to tighten the ear loops
    • We also got a set of these kn95s. They don’t fit quite as well in terms of a tight face fit, but these actually work as ear loops (as designed), and I was able to wear this inside the house on the worst day of air quality.
  • Box fan with a filter to reduce COVID-19 particles in the air:
    • We read this story about using an existing AC air furnace filter on a box fan to help reduce the number of COVID-19 particles in the air. We already had a box fan, so we took one of our spare 20×20 filters and popped it on. I’m allergic to dust, cats (which we just got), trees, grass, etc, so I knew it would also help with regular allergens. There are different levels of filter – all the way up to HEPA filters – but we had MERV 12 so that’s what we used.
  • Phone/object UV sanitizer
    • We got a PhoneSoap Pro (in lavender, but there are other colors). Phones are germy, and being able to pop the phone in (plus keys or any other objects like credit cards or insurance cards that might have been handled by another human) to disinfect has been nice to have.
    • The Pro is done sanitizing in 5 minutes, vs the regular one takes 10 minutes. It’s not quite 2x the price as the non-pro, but I’ve found it to be worthwhile because otherwise, I would be impatient to get my phone back out. I usually pop my phone in it when I get home from my walk, and by the time I’m done washing my hands and all the steps of getting home, the phone is about or already done being sanitized.
  • Bonus (but not as useful to everyone as the above, and pricey): Oura ring
    • Scott and I also both got Oura rings. They are pricey, but every morning when we wake up we can see our lowest resting heart rate (RHR), heart rate variability (HRV), temperature deviations, and respiratory rate (RR). There have been studies showing that HRV, RHR, overnight temperature, and RR changes happen early in COVID-19 and other infections, which can give an early warning sign that you might be getting sick with something. That can be a good early warning sign (before you get to the point of being symptomatic and highly infectious) that you need to mask up and work from home/social distance/not interact with other people if you can help it. I find the data soothing, as I am used to using a lot of diabetes data on a daily and real-time basis (see also: invented an open source artificial pancreas). Due to price and level of interest in self-tracking data, this may not be a great tool for everyone.
    • Note this doesn’t tell you your temperature in real time, or present absolute values, but it’s helpful to see, and get warnings about, any concerning trends in your body temperature data. I’ve seen several anecdotal reports of this being used for early detection of COVID-19 infection and various types of relapses experienced by long-haulers.

And here are some things we’ve added to battle air quality during wildfire smoke season:

  • We were already running a box fan with a filter (see above for more details) for COVID-19 and allergen reduction; so we kept running it on high speed for smoke reduction.
    • Basic steps: get box fan, get a filter, and duct tape or strap it on. Doesn’t have to be cute, but it will help.
    • I run this on high speed during the day in my bedroom, and then on low speed overnight or sleep with earplugs in.
  • We already had a small air purifier for allergens, which we also kept running on high. This one hangs out in our guest bedroom/my office.
  • We caved and got a new, bigger air purifier, since we expect future years to be equally and unfortunately as smoky. This is the new air purifier we got. (Scott chose the 280i version that claims to cover 279 sq. ft.). It’s expensive, but given how miserable I was even inside the house with decent air quality thanks to my box fan and filter, little purifier, and our A/C filtered air… I consider it to be worth the investment.
    • We plugged it in and validated that with our A/C-filtered air combined with my little air purifier and the box fan with filter running on high, we already had ‘good’ air quality (but not excellent). We also stuck it out in the hallway to see what the hallway air quality was running – around 125 ug/m^3 – yikes. Turns out that was almost as high as the outside air, which is I’ve had to wear a kn95 mask even to walk hallway laps, and why my eyes are irritated. example air quality difference between hallway and our kitchen. hallway is much higher.
  • Check your other filters while you’re on air quality monitoring alert. We found our A/C intake duct vent had not had the air filter changed since we moved in over a year ago… and turns out it’s a non-standard size and had a hand-cut stuffed in there, so we ordered a correctly sized one for the vent, and taped a different one over the outside in the interim.
  • The other thing to fight the smoke is having n95 with valves or kn95 masks to wear when we have to go outside, or if it gets particularly bad inside. Our previous strategy was to have several on hand for wildfire season, and we’ll continue to do this. (See above in the COVID-19 section for descriptions in more detail about different kinds of masks we’ve tried.)

Wildfires, their smoke, and COVID-19 combined is a bit of a mess for our health. Stay inside when you can, wear masks when you’re around other people outside your household that you have to share air with, wash your hands, and good luck.

Poster and presentation content from @DanaMLewis at #ADA2020 and #DData20

In previous years (see 2019 and 2018), I mentioned sharing content from ADA Scientific Sessions (this year it’s #ADA2020) with those not physically present at the conference. This year, NO ONE is present at the event, and we’re all virtual! Even more reason to share content from the conference. :)

I contributed to and co-authored two different posters at Scientific Sessions this year:

  • “Multi-Timescale Interactions of Glucose and Insulin in Type 1 Diabetes Reveal Benefits of Hybrid Closed Loop Systems“ (poster 99-LB) along with Azure Grant and Lance Kriegsfeld, PhD.
  • “Do-It-Yourself Artificial Pancreas Systems for Type 1 Diabetes Reduce Hyperglycemia Without Increasing Hypoglycemia” (poster 988-P in category 12-D Clinical Therapeutics/New Technology—Insulin Delivery Systems), alongside Jennifer Zabinsky, MD MEng, Haley Howell, MSHI, Alireza Ghezavati, MD, Andrew Nguyen, PhD, and Jenise Wong, MD PhD.

And, while not a poster at ADA, I also presented the “AID-IRL” study funded by DiabetesMine at #DData20, held in conjunction with Scientific Sessions. A summary of the study is also included in this post.

First up, the biological rhythms poster, “Multi-Timescale Interactions of Glucose and Insulin in Type 1 Diabetes Reveal Benefits of Hybrid Closed Loop Systems” (poster 99-LB). (Twitter thread summary of this poster here.)

Building off our work as detailed last year, Azure, Lance, and I have been exploring the biological rhythms in individuals living with type 1 diabetes. Why? It’s not been done before, and we now have the capabilities thanks to technology (pumps, CGM, and closed loops) to better understand how glucose and insulin dynamics may be similar or different than those without diabetes.

Background:

Mejean et al., 1988Blood glucose and insulin exhibit coupled biological rhythms at multiple timescales, including hours (ultradian, UR) and the day (circadian, CR) in individuals without diabetes. The presence and stability of these rhythms are associated with healthy glucose control in individuals without diabetes. (See right, adapted from Mejean et al., 1988).

However, biological rhythms in longitudinal (e.g., months to years) data sets of glucose and insulin outputs have not been mapped in a wide population of people with Type 1 Diabetes (PWT1D). It is not known how glucose and insulin rhythms compare between T1D and non-T1D individuals. It is also unknown if rhythms in T1D are affected by type of therapy, such as Sensor Augmented Pump (SAP) vs. Hybrid Closed Loop (HCL). As HCL systems permit feedback from a CGM to automatically adjust insulin delivery, we hypothesized that rhythmicity and glycemia would exhibit improvements in HCL users compared to SAP users. We describe longitudinal temporal structure in glucose and insulin delivery rate of individuals with T1D using SAP or HCL systems in comparison to glucose levels from a subset of individuals without diabetes.

Data collection and analysis:

We assessed stability and amplitude of normalized continuous glucose and insulin rate oscillations using the continuous wavelet transformation and wavelet coherence. Data came from 16 non-T1D individuals (CGM only, >2 weeks per individual) from the Quantified Self CGM dataset and 200 (n = 100 HCL, n = 100 SAP; >3 months per individual) individuals from the Tidepool Big Data Donation Project. Morlet wavelets were used for all analyses. Data were analyzed and plotted using Matlab 2020a and Python 3 in conjunction with in-house code for wavelet decomposition modified from the “Jlab” toolbox, from code developed by Dr. Tanya Leise (Leise 2013), and from the Wavelet Coherence toolkit by Dr. Xu Cui. Linear regression was used to generate correlations, and paired t-tests were used to compare AUC for wavelet and wavelet coherences by group (df=100). Stats used 1 point per individual per day.

Wavelets Assess Glucose and Insulin Rhythms and Interactions

Wavelet Coherence flow for glucose and insulin

Morlet wavelets (A) estimate rhythmic strength in glucose or insulin data at each minute in time (a combination of signal amplitude and oscillation stability) by assessing the fit of a wavelet stretched in window and in the x and y dimensions to a signal (B). The output (C) is a matrix of wavelet power, periodicity, and time (days). Transform of example HCL data illustrate the presence of predominantly circadian power in glucose, and predominantly 1-6 h ultradian power in insulin. Color map indicates wavelet power (synonymous with Y axis height). Wavelet coherence (D) enables assessment of rhythmic interactions between glucose and insulin; here, glucose and insulin rhythms are highly correlated at the 3-6 (ultradian) and 24 (circadian) hour timescales.

Results:

Hybrid Closed Loop Systems Reduce Hyperglycemia

Glucose distribution of SAP, HCL, and nonT1D
  • A) Proportional counts* of glucose distributions of all individuals with T1D using SAP (n=100) and HCL (n=100) systems. SAP system users exhibit a broader, right shifted distribution in comparison to individuals using HCL systems, indicating greater hyperglycemia (>7.8 mmol/L). Hypoglycemic events (<4mmol/L) comprised <5% of all data points for either T1D dataset.
  • B) Proportional counts* of non-T1D glucose distributions. Although limited in number, our dataset from people without diabetes exhibits a tighter blood glucose distribution, with the vast majority of values falling in euglycemic range (n=16 non-T1D individuals).
  • C) Median distributions for each dataset.
  • *Counts are scaled such that each individual contributes the same proportion of total data per bin.

HCL Improves Correlation of Glucose-Insulin Level & Rhythm

Glucose and Insulin rhythms in SAP and HCL

SAP users exhibit uncorrelated glucose and insulin levels (A) (r2 =3.3*10-5; p=0.341) and uncorrelated URs of glucose and insulin (B) (r2 =1.17*10-3; p=0.165). Glucose and its rhythms take a wide spectrum of values for each of the standard doses of insulin rates provided by the pump, leading to the striped appearance (B). By contrast, Hybrid Closed Loop users exhibit correlated glucose and insulin levels (C) (r2 =0.02; p=7.63*10-16), and correlated ultradian rhythms of glucose and insulin (D) (r2 =-0.13; p=5.22*10-38). Overlays (E,F).

HCL Results in Greater Coherence than SAP

Non-T1D individuals have highly coherent glucose and insulin at the circadian and ultradian timescales (see Mejean et al., 1988, Kern et al., 1996, Simon and Brandenberger 2002, Brandenberger et al., 1987), but these relationships had not previously been assessed long-term in T1D.

coherence between glucose and insulin in HCL and SAP, and glucose swings between SAP, HCL, and non-T1DA) Circadian (blue) and 3-6 hour ultradian (maroon) coherence of glucose and insulin in HCL (solid) and SAP (dotted) users. Transparent shading indicates standard deviation. Although both HCL and SAP individuals have lower coherence than would be expected in a non-T1D individual,  HCL CR and UR coherence are significantly greater than SAP CR and UR coherence (paired t-test p= 1.51*10-7 t=-5.77 and p= 5.01*10-14 t=-9.19, respectively). This brings HCL users’ glucose and insulin closer to the canonical non-T1D phenotype than SAP users’.

B) Additionally, the amplitude of HCL users’ glucose CRs and URs (solid) is closer (smaller) to that of non-T1D (dashed) individuals than are SAP glucose rhythms (dotted). SAP CR and UR amplitude is significantly higher than that of HCL or non-T1D (T-test,1,98, p= 47*10-17 and p= 5.95*10-20, respectively), but HCL CR amplitude is not significantly different from non-T1D CR amplitude (p=0.61).

Together, HCL users are more similar than SAP users to the canonical Non-T1D phenotype in A) rhythmic interaction between glucose and insulin and B) glucose rhythmic amplitude.

Conclusions and Future Directions

T1D and non-T1D individuals exhibit different relative stabilities of within-a-day rhythms and daily rhythms in blood glucose, and T1D glucose and insulin delivery rhythmic patterns differ by insulin delivery system.

Hybrid Closed Looping is Associated With:

  • Lower incidence of hyperglycemia
  • Greater correlation between glucose level and insulin delivery rate
  • Greater correlation between ultradian glucose and ultradian insulin delivery rhythms
  • Greater degree of circadian and ultradian coherence between glucose and insulin delivery rate than in SAP system use
  • Lower amplitude swings at the circadian and ultradian timescale

These preliminary results suggest that HCL recapitulates non-diabetes glucose-insulin dynamics to a greater degree than SAP. However, pump model, bolusing data, looping algorithms and insulin type likely all affect rhythmic structure and will need to be further differentiated. Future work will determine if stability of rhythmic structure is associated with greater time in range, which will help determine if bolstering of within-a-day and daily rhythmic structure is truly beneficial to PWT1D.
Acknowledgements:

Thanks to all of the individuals who donated their data as part of the Tidepool Big Data Donation Project, as well as the OpenAPS Data Commons, from which data is also being used in other areas of this study. This study is supported by JDRF (1-SRA-2019-821-S-B).

(You can download a full PDF copy of the poster here.)

Next is “Do-It-Yourself Artificial Pancreas Systems for Type 1 Diabetes Reduce Hyperglycemia Without Increasing Hypoglycemia” (poster 988-P in category 12-D Clinical Therapeutics/New Technology—Insulin Delivery Systems), which I co-authored alongside Jennifer Zabinsky, MD MEng, Haley Howell, MSHI, Alireza Ghezavati, MD, Andrew Nguyen, PhD, and Jenise Wong, MD PhD. There is a Twitter thread summarizing this poster here.

This was a retrospective double cohort study that evaluated data from the OpenAPS Data Commons (data ranged from 2017-2019) and compared it to conventional sensor-augmented pump (SAP) therapy from the Tidepool Big Data Donation Project.

Methods:

  • From the OpenAPS Data Commons, one month of CGM data (with more than 70% of the month spent using CGM), as long as they were >1 year of living with T1D, was used. People could be using any type of DIYAPS (OpenAPS, Loop, or AndroidAPS) and there were no age restrictions.
  • A random age-matched sample from the Tidepool Big Data Donation Project of people with type 1 diabetes with SAP was selected.
  • The primary outcome assessed was percent of CGM data <70 mg/dL.
  • The secondary outcomes assessed were # of hypoglycemic events per month (15 minutes or more <70 mg/dL); percent of time in range (70-180mg/dL); percent of time above range (>180mg/dL), mean CGM values, and coefficient of variation.
Methods_DIYAPSvsSAP_ADA2020_DanaMLewis

Demographics:

  • From Table 1, this shows the age of participants was not statistically different between the DIYAPS and SAP cohorts. Similarly, the age at T1D diagnosis or time since T1D diagnosis did not differ.
  • Table 2 shows the additional characteristics of the DIYAPS cohort, which included data shared by a parent/caregiver for their child with T1D. DIYAPS use was an average of 7 months, at the time of the month of CGM used for the study. The self-reported HbA1c in DIYAPS was 6.4%.
Demographics_DIYAPSvsSAP_ADA2020_DanaMLewis DIYAPS_Characteristics_DIYAPSvsSAP_ADA2020_DanaMLewis

Results:

  • Figure 1 shows the comparison in outcomes based on CGM data between the two groups. Asterisks (*) indicate statistical significance.
  • There was no statistically significant difference in % of CGM values below 70mg/dL between the groups in this data set sampled.
  • DIYAPS users had higher percent in target range and lower percent in hyperglycemic range, compared to the SAP users.
  • Table 3 shows the secondary outcomes.
  • There was no statistically significant difference in the average number of hypoglycemic events per month between the 2 groups.
  • The mean CGM glucose value was lower for the DIYAPS group, but the coefficient of variation did not differ between groups.
CGM_Comparison_DIYAPSvsSAP_ADA2020_DanaMLewis SecondaryOutcomes_DIYAPSvsSAP_ADA2020_DanaMLewis

Conclusions:

    • Users of DIYAPS (from this month of sampled data) had a comparable amount of hypoglycemia to those using SAP.
    • Mean CGM glucose and frequency of hyperglycemia were lower in the DIYAPS group.
    • Percent of CGM values in target range (70-180mg/dL) was significantly greater for DIYAPS users.
    • This shows a benefit in DIYAPS in reducing hyperglycemia without compromising a low occurrence of hypoglycemia. 
Conclusions_DIYAPSvsSAP_ADA2020_DanaMLewis

(You can download a PDF of the e-poster here.)

Finally, my presentation at this year’s D-Data conference (#DData20). The study I presented, called AID-IRL, was funded by Diabetes Mine. You can see a Twitter thread summarizing my AID-IRL presentation here.

AID-IRL-Aim-Methods_DanaMLewis

I did semi-structured phone interviews with 7 users of commercial AID systems in the last few months. The study was funded by DiabetesMine – both for my time in conducting the study, as well as funding for study participants. Study participants received $50 for their participation. I sought a mix of longer-time and newer AID users, using a mix of systems. Control-IQ (4) and 670G (2) users were interviewed; as well as (1) a CamAPS FX user since it was approved in the UK during the time of the study.

Based on the interviews, I coded their feedback for each of the different themes of the study depending on whether they saw improvements (or did not have issues); had no changes but were satisfied, or neutral experiences; or saw negative impact/experience. For each participant, I reviewed their experience and what they were happy with or frustrated by.

Here are some of the details for each participant.

AID-IRL-Participant1-DanaMLewisAID-IRL-Participant1-cont_DanaMLewis1 – A parent of a child using Control-IQ (off-label), with 30% increase in TIR with no increased hypoglycemia. They spend less time correcting than before; less time thinking about diabetes; and “get solid uninterrupted sleep for the first time since diagnosis”. They wish they had remote bolusing, more system information available in remote monitoring on phones. They miss using the system during the 2 hour CGM warmup, and found the system dealt well with growth spurt hormones but not as well with underestimated meals.

AID-IRL-Participant2-DanaMLewis AID-IRL-Participant2-cont-DanaMLewis2 – An adult male with T1D who previously used DIYAPS saw 5-10% decrease in TIR (but it’s on par with other participants’ TIR) with Control-IQ, and is very pleased by the all-in-one convenience of his commercial system.He misses autosensitivity (a short-term learning feature of how insulin needs may very from base settings) from DIYAPS and has stopped eating breakfast, since he found it couldn’t manage that well. He is doing more manual corrections than he was before.

AID-IRL-Participant5-DanaMLewis AID-IRL-Participant5-cont_DanaMLewis5 – An adult female with LADA started, stopped, and started using Control-IQ, getting the same TIR that she had before on Basal-IQ. It took artificially inflating settings to achieve these similar results. She likes peace of mind to sleep while the system prevents hypoglycemia. She is frustrated by ‘too high’ target; not having low prevention if she disables Control-IQ; and how much she had to inflate settings to achieve her outcomes. It’s hard to know how much insulin the system gives each hour (she still produces some of own insulin).

AID-IRL-Participant7-DanaMLewis AID-IRL-Participant7-cont-DanaMLewis7 – An adult female with T1D who frequently has to take steroids for other reasons, causing increased BGs. With Control-IQ, she sees 70% increase in TIR overall and increased TIR overnight, and found it does a ‘decent job keeping up’ with steroid-induced highs. She also wants to run ‘tighter’ and have an adjustable target, and does not ever run in sleep mode so that she can always get the bolus corrections that are more likely to bring her closer to target.

AID-IRL-Participant3-DanaMLewis AID-IRL-Participant3-cont-DanaMLewis3 – An adult male with T1D using 670G for 3 years didn’t observe any changes to A1c or TIR, but is pleased with his outcomes, especially with the ability to handle his activity levels by using the higher activity target.  He is frustrated by the CGM and is woken up 1-2x a week to calibrate overnight. He wishes he could still have low glucose suspend even if he’s kicked out of automode due to calibration issues. He also commented on post-meal highs and more manual interventions.

AID-IRL-Participant6-DanaMLewis AID-IRL-Participant6-contDanaMLewis6 – Another adult male user with 670G was originally diagnosed with T2 (now considered T1) with a very high total daily insulin use that was able to decrease significantly when switching to AID. He’s happy with increased TIR and less hypo, plus decreased TDD. Due to #COVID19, he did virtually training but would have preferred in-person. He has 4-5 alerts/day and is woken up every other night due to BG alarms or calibration. He does not like the time it takes to charge CGM transmitter, in addition to sensor warmup.

AID-IRL-Participant4-DanaMLewis AID-IRL-Participant4-contDanaMLewis4 – The last participant is an adult male with T1 who previously used DIYAPS but was able to test-drive the CamAPS FX. He saw no TIR change to DIYAPS (which pleased him) and thought the learning curve was easy – but he had to learn the system and let it learn him. He experienced ‘too much’ hypoglycemia (~7% <70mg/dL, 2x his previous), and found it challenging to not have visibility of IOB. He also found the in-app CGM alarms annoying. He noted the system may work better for people with regular routines.

You can see a summary of the participants’ experiences via this chart. Overall, most cited increased or same TIR. Some individuals saw reduced hypos, but a few saw increases. Post-meal highs were commonly mentioned.

AID-IRL-UniversalThemes2-DanaMLewis AID-IRL-UniversalThemes-DanaMLewis

Those newer to CGM have a noticeable learning curve and were more likely to comment on number of alarms and system alerts they saw. The 670G users were more likely to describe connection/troubleshooting issues and CGM calibration issues, both of which impacted sleep.

This view highlights those who more recently adopted AID systems. One noted their learning experience was ‘eased’ by “lurking” in the DIY community, and previously participating in an AID study. One felt the learning curve was high. Another struggled with CGM.

AID-IRL-NewAIDUsers-DanaMLewis

Both previous DIYAPS users who were using commercial AID systems referenced the convenience factor of commercial systems. One DIYAPS saw decreased TIR, and has also altered his behaviors accordingly, while the other saw no change to TIR but had increased hypo’s.

AID-IRL-PreviousDIYUsers-DanaMLewis

Companies building AID systems for PWDs should consider that the onboarding and learning curve may vary for individuals, especially those newer to CGM. Many want better displays of IOB and the ability to adjust targets. Remote bolusing and remote monitoring is highly desired by all, regardless of age. Post-prandial was frequently mentioned as the weak point in glycemic control of commercial AID systems. Even with ‘ideal’ TIR, many commercial users still are doing frequent manual corrections outside of mealtimes. This is an area of improvement for commercial AID to further reduce the burden of managing diabetes.

AID-IRL-FeedbackForCompanies-DanaMLewis

Note – all studies have their limitations. This was a small deep-dive study that is not necessarily representative, due to the design and small sample size. Timing of system availability influenced the ability to have new/longer time users.

AID-IRL-Limitations-DanaMLewis

Thank you to all of the participants of the study for sharing their feedback about their experiences with AID-IRL!

(You can download a PDF of my slides from the AID-IRL study here.)

Have questions about any of my posters or presentations? You can always reach me via email at Dana@OpenAPS.org.

Convening The Center

(Update: see the latest about Convening the Center in 2021 here)

Patients and care partners who want to make a difference in health care are advised to give up our day jobs, create non-profits, or change previously identified career paths to “go work for a healthcare organization.” These formal constructs are not the only ways to achieve change or make a difference.

Those who choose to work outside of traditional pathways often end up with fewer resources and fewer opportunities (not just financial, but also the opportunity of collaborations and connections).

Thinking about these gaps in resources and opportunities has been swimming around my head since the Convening we hosted as part of the Opening Pathways project (more about it here). As a project, we learned so much from the conversations we had when we were able to just bring people together.

The feedback we received from non-traditional healthcare stakeholders was one of the most surprising results of the Convening. These are people who are not necessarily working professionally in healthcare, but doing a lot of work in the nontraditional spaces. In the year since the Convening we’ve repeatedly heard how valuable it was for this group to come together, in person, to connect with others with a similar drive and passion.

Fast forward to early last year. My friend Liz Salmi (of #BTSM) reached out Alicia Staley (of #BCSM) and me to share about an exciting, random conversation and brainstorm she had with Steve Downs from Robert Wood Johnson Foundation (RWJF).  The idea: What if there was an ‘unconference’ to bring together more of these individuals–those working outside of traditional pathways–to learn and collaborate, without the agenda driven by an existing organization, association, established conference, or company?

This concept sounded great to me! It feels like a next logical step to take with Opening Pathways especially if we pair it with a few structured activities similar to what we did at the Convening to create more equitable participation opportunities for patients and care partners to help people feel comfortable engaging together in person.

When Liz said she didn’t have time to lead this project I volunteered to take it on. Liz and Alicia agreed and expressed their full support.

I put together a proposal in partnership with John Harlow who also worked on Opening Pathways, and was instrumental in designing the original Convening. We submitted a proposal to RWJF, did a few rounds of feedback and discussion about the proposal, waited a bit, and found out right around the new year that the proposal was accepted and had been awarded funding! Yay!

We’re calling this project “Convening The Center.” This both picks up on the name of the previous Convening, and emphasizes the people/patients as the center on which all of health and healthcare should be focused.

Convening The Center: What if there was a gathering for individuals working outside of traditional healthcare pathways?

What this means:

  • We have funding to put together a ~2 day meeting for ~25 individuals who are doing both the possible and the impossible to change and improve healthcare.
  • The funding includes travel (ground transportation, flights), lodging (hotel), food during the event, and an honorarium for the participants’ time.
  • The meeting was originally scheduled to be sometime in 2020 (August or September was goal; COVID-19 disrupted this planning, TBD for new dates but looking at 2021 instead).

Who will be involved:

Convening The Center project team:

  • Dana Lewis (me), Principal Investigator (PI)
  • John Harlow, Co-Principal Investigator (PI)
  • Convening Advisors: Liz Salmi, Alicia Staley, Nick Dawson

Who can participate?:

  • TBD! Here’s why and how:

Why must we convene the Center?

If you’re reading this, you likely have your own story of doing the “impossible” — you’ve faced barriers and obstacles, but have found a way to innovate, overcome, or steer around. There are a LOT of people doing this “work,” whether it’s their professional work, their personal passion, or a necessity driving them to improve things for themselves or a loved one, building and supporting their communities as unfunded labors of love. But we also know that geography, socioeconomic background, and financial resources, among other reasons, commonly leave some of these individuals siloed, or prevent them and their work from reaching its full potential.

We know there is a lack of connectedness among individual innovators, researchers, and advocates who are not employed in the traditional healthcare system. While there have been a handful of attempts to convene patient advocates to share ideas and connect with opportunities and resources, none have been devoted solely to this type of community. Existing attempts have included ad-hoc social media groups and inclusion at existing conferences and meetings. Both face serious limitations.

Social media is limited by one’s ability to stumble across a network, while conferences or meetings—which are traditionally held by legacy institutions—usually include people who are already “in” a network that invites them to such physical events, and are thus already “doing” the work, but these do not do enough to encourage new participants. Additionally, conferences and meetings prioritize the hosting organization’s agenda rather than facilitating the development of non-traditional innovators. Given the limitations of social media and existing conferences, the status quo leads new “doers” to (unknowingly and repeatedly) duplicate the work of others and fail to effectively share knowledge and scale tools that could help others. Overall, there are not a lot of resources for people who do this outside of a professional job.

Therefore, we aim to do something different to identify participants for this meeting.

Rather than just invite the same individuals who have the resources to participate, or have already succeeded somewhat, even in the face of all the existing barriers, we plan to solicit attendees from a mix of health communities, from a range of experiences, with diverse demographics, including those who are newly working in this space, as well as experienced individuals with established credibility.

How will we reach all of these different communities and individuals? This is where we need your help!

We have a two-phase recruitment process to identify potential attendees.

Phase 1 (right now)

  • Fill out this form! 
    • We’d love for you to nominate yourself, if you’re potentially interested in participating.
    • But a crucial part of this is to ALSO nominate someone else – a friend or someone you know who may not otherwise hear about this opportunity.
  • We’d also love for you to help share this form widely and help us reach people in different networks. If you TikTok, post it on TikTok. If you’re on LinkedIn, share it on your LinkedIn or a group. If you’re part of an offline support group, talk about it there. Or reach out and share the link with your advocacy organization and encourage them to nominate other advocates and ‘doers’ that they know.

Nominate someone you know for Convening The Center!
Phase 2 (in a few weeks):

  • Based on the first wave of nominated folks, we’ll work to make sure we’re striking the balance between people who are longer-timers in this space and people who are newly emerging in this type of work.
  • We’ll reach out to a selection of folks identified in phase 1 and ask for a little bit more information to help determine the final cohort of participants for the in-person meeting. (Goal: ~25 participants).

We’ve learned through Opening Pathways and other work in this space that more — and perhaps different — resources are needed for “doers” in healthcare who are not traditionally employed in this space.

We don’t expect the outcome of this project to solve all problems or identify a one-size-fits-all resource. However, we do hope to help manifest a new, more inclusive, and more effective vision for changing the future of healthcare.

The future we seek augments the existing health efforts of legacy institutions by coordinating the work of individual innovators, researchers, and advocates in a more inclusive community of practice. We do not think this will solve all problems around under-representation and the static network of those already “in” and doing this work, but it’s an important step and one we’re happy to be able to take.

FREQUENTLY ASKED QUESTIONS

  • Who is funding this project? How is it being funded? What organization are you partnering with?Robert Wood Johnson Foundation (RWJF) is a great partner, and I’m proud that they’re willing to fund this meeting. Paul Tarini is our project officer at RWJF. While my co-PI is based at an academic institution, we decided to experiment with using a fiscal sponsorship organization to manage the grant. We identified and selected Trailhead Institute, a 501(c)(3) organization that works with a variety of projects and organizations in the public health space. I’ll write more about this in the future, but so far they have been GREAT administrative partners and have been seamless to work with during the application and kickoff of the grant process. Also, we learned from the past Convening that it would be beneficial to directly fund a meeting planner to do logistics work (rather than me), so we included in our budget a meeting planner that is coming from Trailhead to help with administrative and logistics planning for the meeting. Yay!
  • How will you select participants?Our goal is to gain a diverse slate of people, including diversity in socioeconomic background, ethnicity, gender, education, area of healthcare, type of work, how long they have been doing the work, etc. Before finalizing the list of participants we will collect information from potential participants and make sure they’d be interested and available to participate once the date is selected.
  • What are the outputs?We anticipate one primary output from this meeting to be relationships among attendees. After observing the strength and resilience generated for individuals by participating in our Opening Pathways convening, we see relationships as a powerful support for the efforts of healthcare “doers”. By relationships, we do not mean a community of 25. Community building is long-term labor-intensive work. Rather, we hope that some attendees will find common ground and collaborate in various ways after Convening the Center.We do not expect to produce a particular report or website from this work. However, we do expect to write blog posts about our process of developing the meeting, the experience of facilitating the meeting, and the insights derived from conversations at the meeting. We anticipate those insights to be about the wants and needs of healthcare doers, what they wish they had when they started out, what they’d tell their younger selves, and how to refine and scale various healthcare improvement efforts.
  • What about COVID-19?While we have been planning this meeting for August or September 2020, we are aware that currently (in March 2020) there is a lot of uncertainty about how COVID-19 may impact meetings after the next few months. While we are beginning virtual recruitment of participants, we will work with public health officials to get guidance on whether August/September still makes sense, and if not, work with both participants and public health to determine a suitable alternative timeline for holding the meeting. If that’s not feasible, we may find ways to meet this goal virtually.Update: Obviously, it does not make sense to convene the center physically for an in-person meeting in 2020. We are aiming for a gathering – in-person if safe and appropriate, otherwise adapting to virtual – in 2021. We’ll keep everyone posted!

(Update: see the latest about Convening the Center in 2021 here)

How the sausage gets made – guest editing and peer reviewing for scientific journals (and advice for future publications)

I’m not an academic, but I have spent a lot of time (especially lately) writing, editing, submitting, and reviewing for “peer-reviewed” scientific publications. As a result, I wanted to share some of my experiences and insights gained that may help others who are planning to write, submit, or review similar peer-reviewed process pieces!

My background in publishing in peer-review journals

In 2016, I presented my first poster at a scientific meeting. This was a big deal, because I’m not an academic, I don’t have an academic degree, and I didn’t “work” my day job in the space I was presenting in. After the conference, I was given an invitation to write an article with the results of the study I had presented the poster on. I was nervous, but accepted, and did it. It turns out, it wasn’t that hard. (Granted, it was a Letter to the Editor, rather than a longer format ‘original research article’, but it still wasn’t as hard as I had perceived it to be). My article was successfully published in a scientific journal.

In the years since, I have subsequently decided to write up more of my research and results of work happening in the open source, do-it-yourself diabetes community. Why? As I wrote in this post, I realize that not all HCPs are willing or able to stay up to date with the bleeding edge of what’s being created and innovated on in the diabetes community. If we want HCPs to get up to speed more quickly, we need to play a role in taking the information to them. Thus, I work to publish in journals (since they’re more likely to read or stumble across those than blog posts). (If you’re interested, most of my publications are listed in Google Scholar if you want to see the types of things I’ve been writing and contributing to.)

My new hat: guest editing for a journal

This year, though, I started having a whole set of new experiences with regards to the process of journal publications. I was asked to serve as Guest Editor for the forthcoming special “DIY” issue in the Journal of Diabetes Science and Technology.

Whoa. Hello, imposter syndrome! Who was I, a non-academic, non-MD, non-PhD, non-all-the-things, to play a role in what goes in the literature?! But I said yes anyway, because I figured it would be a good learning process for my own future efforts to publish. And it has been! (Although it is, like writing your own articles and peer-reviewing other people’s articles, unpaid work.)

Here’s what I do as guest editor:

  • First, I dreamed up a list of people who should write for the special issue and likely had new insights not already in the literature, or had new research that would be a good fit for the issue. I sent the list to the production editor, who sent out official invitations to submit, and got people to commit to writing for the special issue.
  • As manuscripts come in, it’s my job to review the submissions and recommend reviewers (usually 2-3) for each manuscript. Thankfully, I think every peer reviewer I have nominated has been willing to review the manuscripts we’ve sent to them – if you’re one of those folks, a big thank you!  
  • As editor, I then review the reviewer comments and make sure they’re appropriate to send back to the author. They have all been, so far. (This has been a super educational process in and of its own, more on that below.)
  • The authors then revise their article, write a response to the reviewer comments, and send it back. It’s my job to review the revisions and response. I can either, based on reviewer feedback: reject it, accept it as revised, have the reviewers re-review it, or in a few cases, I’ve made a few edits myself (when inaccuracies were introduced in the revision, particularly a new added section) and asked the authors to approve or further revise those edits before I accept it for the journal.

Here’s some of what I’ve learned as a result:

I’ve learned a lot from getting to read the reviewer comments on other manuscripts. It’s been really helpful, because I have my own opinions when reading the manuscript in the first pass for picking reviewers, and then I can compare my own perspective on how it might be improved with what the other reviewers have flagged as needing adjustment before publication.

Also, this is especially helpful because I somehow have started getting a lot of reviewer requests myself (separate from my guest editing role) from both diabetes and non-diabetes publications, and this helps with my deer-in-the-headlights feeling of not knowing how to write reviews, other than the reviews I’ve read on my own previous work. What I’ve learned by observing a lot of these other reviews now is that on the one hand, as an author, it can feel nice to get a short, sweet, and positive review. However, as an author who wants the strongest manuscript out in the world, a longer, detailed review with both thematic comments and specific recommendations for improvements both helps the publication in the short term, and helps me write better future publications as well.

Similarly, seeing the variety of author responses to reviewer commentary have been educational. The best responses both respond in a separate document and describe what adjustments or changes should be made in the manuscript, but also highlight (either using different colored font or tracked changes) in the manuscript what those changes are. It’s a lot harder to review the revisions when the edits are all accepted/not colored to be easily spotted.

To be fair, it’s not always easy as the author(s) to make the changes in track changes like this. I just participated in a revision of a publication where I’m a co-author: this was a 19 page manuscript with over a dozen co-authors and likely hundreds, if not thousands, of changes. That revision was a LOT of work. But when there are obvious and few changes, and you’re an author, if you don’t already, consider using tracked changes or coloring the edits/additions. It makes it easier for the (guest) editor(s) to review and accept your revision!

How this has influenced my own reviews and future articles:

I also have a better idea of how to do reviews in the future, too. I know now that if there are many flaws that would prevent the publication from getting accepted with only minor edits, I try to stay high level (thanks to Aaron Neinstein for this feedback!) and note the major revision areas, instead of getting stuck in the weeds, because major revisions mean a lot of details will change underneath. I also try to specify where my recommendations go – i.e. make them in order as I read the manuscript, note major section headings or line numbers (although page/line numbers can be hard depending on whether someone is looking at a PDF with the cover page and abstract page and then the article, or just the original article).

Also, I now have a much better sense of the time it takes to do a review. I always try to do a quick skim of the article first. If I only mentally make small, minor or pedantic comments/suggestions, the review itself should only take 15-30 minutes to write and upload/submit the review. However, a manuscript with major flaws and major revision needed should have at least an hour scheduled. I learned this the hard way: a manuscript I procrastinated reviewing because it needed a lot of work took about 45 minutes to provide detailed (but needed) feedback. My review ended up running more than 1,000 words! This has happened several times now, but at least I know to budget an hour for those reviews.

And as a result, the major things I learned from reviewing that will help me with my own articles that I write in the future will be to check for gaps in logic where I assume common understanding that may not exist, and to make sure not to mix commentary in the middle of an article when I’m presenting background or factual information. These are common issues I regularly provide feedback on when reviewing other articles, and so I plan to check my own writing for logical flow and to make sure that discussion points are gathered correctly in the discussion and conclusion sections instead of sprinkled throughout.

—-

I’m not done learning: I imagine I’ll continue having new insights as to the most effective way to write, provide reviews, and make edits to my own work in the future. But when I mentioned that I didn’t feel equipped to peer review at first, my brother (a professor with a PhD in math) wisely pointed out that academics don’t really get training in peer reviewing, or editing, either – so we’re all in the same boat of learning as we go along!

If you’ve ever guest edited or edited a journal, or served as a peer reviewer, what have you learned in the process that has been helpful for writing and submitting your own articles? What advice would you share? Please do share with us here!

Presentations and poster content from @DanaMLewis at #ADA2019

Like I did last year, I want to share the work being presented at #ADA2019 with those who are not physically there! (And if you’re presenting at #ADA2019 or another conference and would like suggestions on how to share your content in addition to your poster or presentation, check out these tips.) This year, I’m co-author on three posters and an oral presentation.

  • 1056-P in category 12-D Clinical Therapeutics/New Technology–Insulin Delivery Systems, Preliminary Characterization of Rhythmic Glucose Variability In Individuals With Type 1 Diabetes, co-authored by Dana Lewis and Azure Grant.
    • Come see us at the poster session, 12-1pm on Sunday! Dana & Azure will be presenting this poster.
  • 76-OR, In-Depth Review of Glycemic Control and Glycemic Variability in People with Type 1 Diabetes Using Open Source Artificial Pancreas Systems, co-authored by Andreas Melmer, Thomas Züger, Dana Lewis, Scott Leibrand, Christoph Stettler, and Markus Laimer.
    • Come hear our presentation in room S-157 (South, Upper Mezzanine Level), 2:15-2:30 pm on Saturday!
  • 117-LB, DIWHY: Factors Influencing Motivation, Barriers and Duration of DIY Artificial Pancreas System Use Among Real-World Users, co-authored by Katarina Braune, Shane O’Donnell, Bryan Cleal, Ingrid Willaing, Adrian Tappe, Dana Lewis, Bastian Hauck, Renza Scibilia, Elizabeth Rowley, Winne Ko, Geraldine Doyle, Tahar Kechadi, Timothy C. Skinner, Klemens Raille, and the OPEN consortium.
    • Come see us at the poster session, 12-1pm on Sunday! Scott will be presenting this poster.
  • 78-LB, Detailing the Lived Experiences of People with Diabetes Using Do-it-Yourself Artificial Pancreas Systems – Qualitative Analysis of Responses to Open-Ended Items in an International Survey, co-authored by Bryan Cleal, Shane O’Donnell, Katarina Braune, Dana Lewis, Timothy C. Skinner, Bastian Hauck, Klemens Raille, and the OPEN consortium.
    • Come see us at the poster session, 12-1pm on Sunday! Bryan Cleal will be presenting this poster.

See below for full written summaries and pictures from each poster and the oral presentation.

First up: the biological rhythms poster, formally known as 1056-P in category 12-D Clinical Therapeutics/New Technology–Insulin Delivery Systems, Preliminary Characterization of Rhythmic Glucose Variability In Individuals With Type 1 Diabetes!

Lewis_Grant_BiologicalRhythmsT1D_ADA2019

As mentioned in this DiabetesMine interview, Azure Grant & I were thrilled to find out that we have been awarded a JDRF grant to further this research and undertake the first longitudinal study to characterize biological rhythms in T1D, which could also be used to inform improvements and personalize closed loop systems. This poster is part of the preliminary research we did in order to submit for this grant.

There is also a Twitter thread for this poster:

Background:

  • Human physiology, including blood glucose, exhibits rhythms at multiple timescales, including hours (ultradian, UR), the day (circadian, CR), and the ~28-day female ovulatory cycle (OR).
  • Individuals with T1D may suffer rhythmic disruption due not only to the loss of insulin, but to injection of insulin that does not mimic natural insulin rhythms, the presence of endocrine-timing disruptive medications, and sleep disruption.
  • However, rhythms at multiple timescales in glucose have not been mapped in a large population of T1D, and the extent to which glucose rhythms differ in temporal structure between T1D and non-T1D individuals is not known.

Data & Methods:

  • The initial data set used for this work leverages the OpenAPS Data Commons. (This data set is available for all researchers  – see www.OpenAPS.org/data-commons)
  • All data was processed in Matlab 2018b with code written by Azure Grant. Frequency decompositions using the continuous morlet wavelet transformation were created to assess change in rhythmic composition of normalized blood glucose data from 5 non-T1D individuals and anonymized, retrospective CGM data from 19 T1D individuals using a DIY closed loop APS. Wavelet algorithms were modified from code made available by Dr. Tanya Leise at Amherst College (see http://bit.ly/LeiseWaveletAnalysis)

Results:

  • Inter and Intra-Individual Variability of Glucose Ultradian and Circadian Rhythms is Greater in T1D
Figure_BiologicalRhythms_Lewis_Grant_ADA2019

Figure 1. Single individual blood glucose over ~ 1 year with A.) High daily rhythm stability and B.) Low daily rhythm stability. Low glucose is shown in blue, high glucose in orange.

Figure 2. T1D individuals (N=19) showed a wide range of rhythmic power at the circadian and long-period ultradian timescales compared to individuals without T1D (N=5).

A). Individuals’ CR and UR power, reflecting amplitude and stability of CRs, varies widely in T1D individuals compared to those without T1D. UR power was of longer periodicity (>= 6 h) in T1D, likely due to DIA effects, whereas UR power was most commonly in the 1-3 hour range in non-T1D individuals (*not shown).  B.) On average, both CR and UR power were significantly higher in T1D (p<.05, Kruskal Wallis). This is most likely due to the higher amplitude of glucose oscillation, shown in two individuals in C.

Conclusions:

  • This is the first longitudinal analysis of the structure and variability of multi-timescale biological rhythms in T1D, compared to non-T1D individuals.
  • Individuals with T1D show a wide range of circadian and ultradian rhythmic amplitudes and stabilities, resulting in higher average and more variable wavelet power than in a smaller sample of non-T1D individuals.
  • Ultradian rhythms of people with T1D are of longer periodicity than individuals without T1D. These analyses constitute the first pass of a subset of these data sets, and will be continued over the next year.

Future work:

  • JDRF has recently funded our exploration of the Tidepool Big Data Donation Project, the OpenAPS Data Commons, and a set of non-T1D control data in order to map biological rhythms of glucose/insulin.
  • We will use signal processing techniques to thoroughly characterize URs, CRs, and ORs in the glucose/insulin for T1D; evaluate if stably rhythmic timing of glucose is associated with improved outcomes (lower HBA1C); and ultimately evaluate if modulation of insulin delivery based on time of day or time of ovulatory cycle could lead to improved outcomes.
  • Mapping population heterogeneity of these rhythms in people with and without T1D will improve understanding of real-world rhythmicity, and may lead to non-linear algorithms for optimizing glucose in T1D.

Acknowledgements:

We thank the OpenAPS community for their generous donation of data, and JDRF for the grant award to further this work, beginning in July 2019.

Contact:

Feel free to contact us at Dana@OpenAPS.org or azuredominique@berkeley.edu.

Next up, 78-LB, Detailing the Lived Experiences of People with Diabetes Using Do-it-Yourself Artificial Pancreas Systems – Qualitative Analysis of Responses to Open-Ended Items in an International Survey, co-authored by Bryan Cleal, Shane O’Donnell, Katarina Braune, Dana Lewis, Timothy C. Skinner, Bastian Hauck, Klemens Raille, and the OPEN consortium.

78-LB_LivedExperiencesDIYAPS_OPEN_ADA2019

There is also a Twitter thread for this poster:

Introduction

There is currently a wave of interest in Do-it-Yourself Artificial Pancreas Systems (DIYAPS), but knowledge about how the use of these systems impacts on the lives of those that build and use them remains limited. Until now, only a select few have been able to give voice to their experiences in a research context. In this study we present data that addresses this shortcoming, detailing the lived experiences of people using DIYAPS in an extensive and diverse way.

Methods

An online survey with 34 items was distributed to DIYAPS users recruited through the Facebook groups “Looped” (and regional sub-groups) and Twitter pages of the Diabetes Online Community (DOC). Participants were posed two open-ended questions in the survey, where personal DIYAPS stories were garnered; including knowledge acquisition, decision-making, support and emotional aspects in the initiation of DIYAPS, perceived changes in clinical and quality of life (QoL) outcomes after initiation and difficulties encountered in the process. All answers were analyzed using thematic content analysis.

Results

In total, 886 adults responded to the survey and there were a combined 656 responses to the two open-ended items. Knowledge of DIYAPS was primarily obtained via exposure to the communication fora that constitute the DOC. The DOC was also a primary source of practical and emotional support (QUOTES A). Dramatic improvements in clinical and QoL outcomes were consistently reported (QUOTES B). The emotional impact was overwhelmingly positive, with participants emphasizing that the persistent presence of diabetes in everyday life was markedly reduced (QUOTES C). Acquisition of the requisite devices to initiate DIYAPS was sometimes problematic and some people did find building the systems to be technically challenging (QUOTE D). Overcoming these challenges did, however, leave people with a sense of accomplishment and, in some cases, improved levels of understanding and engagement with diabetes management (QUOTE E).

QuotesA_OPEN_ADA2019 QuotesB_OPEN_ADA2019 QuotesC_OPEN_ADA2019 QuotesD_OPEN_ADA2019 QuotesE_OPEN_ADA2019

Conclusion

The extensive testimony from users of DIYAPS acquired in this study provides new insights regarding the contours of this evolving phenomenon, highlighting factors inspiring people to adopt such solutions and underlining the transformative impact effective closed-loop systems bring to bear on the everyday lives of people with diabetes. Although DIYAPS is not a viable solution for everyone with type 1 diabetes, there is much to learn from those who have taken this route, and the life-changing results they have achieved should inspire all with an interest in artificial pancreas technology to pursue and dream of a future where all people with type 1 diabetes can reap the benefits that it potentially provides.

Also, see this word cloud generated from 665 responses in the two open-ended questions in the survey:

Wordle_OPEN_ADA2019

Next up is 117-LB, DIWHY: Factors Influencing Motivation, Barriers and Duration of DIY Artificial Pancreas System Use Among Real-World Users, co-authored by Katarina Braune, Shane O’Donnell, Bryan Cleal, Ingrid Willaing, Adrian Tappe, Dana Lewis, Bastian Hauck, Renza Scibilia, Elizabeth Rowley, Winne Ko, Geraldine Doyle, Tahar Kechadi, Timothy C. Skinner, Klemens Raille, and the OPEN consortium.

DIWHY_117-LB_OPEN_ADA2019

There is also a Twitter thread for this poster:

Background

Until recently, digital innovations in healthcare have typically followed a ‘top-down’ pathway, with manufacturers leading the design and production of technology-enabled solutions and patients involved only as users of the end-product. However, this is now being disrupted by the increasing influence and popularity of more ‘bottom-up’ and patient-led open source initiatives. A primary example is the growing movement of people with diabetes (PwD) who create their own “Do-it-Yourself” Artificial Pancreas Systems (DIY APS) through remote-control of medical devices employing an open source algorithm.

Objective

Little is known about why PwD leave traditional care pathways and turn to DIY technology. This study aims to examine the motivations of current DIYAPS users and their caregivers.

Research Design and Methods

An online survey with 34 items was distributed to DIYAPS users recruited through the Facebook groups “Looped” (and regional sub-groups) and Twitter pages of the “DOC” (Diabetes Online Community). Self-reported data was collected, managed and analyzed using the secure REDCap electronic data capture tools hosted at Charité – Universitaetsmedizin Berlin.

Results

1058 participants from 34 countries (81.3 % Europe, 14.7 % North America, 6.0 % Australia/WP, 3.1 % Asia, 0.1 % Africa), responded to the survey, of which the majority were adults (80.2 %) with type 1 diabetes (98.9 %) using a DIY APS themselves (43.0 % female, 56.8 % male, 0.3 % other) with a median age of 41 y and an average diabetes duration of 25.2y ±13.3. 19.8 % of the participants were parents and/or caregivers of children with type 1 diabetes (99.4 %) using a DIY APS (47.4 % female, 52.6 % male) with a median age of 10 y and an average diabetes duration of 5.1y ± 3.8. People used various DIYAPS (58.2 % AndroidAPS, 28.5 % Loop, 18.8 % OpenAPS, 5.7 % other) on average for a duration of 10.1 months ±17.6 and reported an overall HbA1c-improvement of -0.83 % (from 7.07 % ±1.07 to 6.24 % ±0.68 %) and an overall Time in Range improvement of +19.86 % (from 63.21 % ±16.27 to 83.07 % ±10.11). Participants indicated that DIY APS use required them to pay out-of-pocket costs in addition to their standard healthcare expenses with an average amount of 712 USD spent per year.

Primary motivations for building a DIYAPS were to improve the overall glycaemic control, reduce acute and long-term complication risk, increase life expectancy and to put diabetes on ‘auto-pilot’ and interact less frequently with the system. Lack of commercially available closed loop systems and improvement of sleep quality was a motivation for some. For caregivers, improvement of their own sleep quality was the leading motivation. For adults, curiosity (medical or technical interest) had a higher impact on their motivation compared to caregivers. Some people feel that commercial systems do not suit their individual needs and prefer to use a customizable system, which is only available to them as a DIY solution. Other reasons, like costs of commercially available systems and unachieved therapy goals played a subordinate role. Lack of medical or psychosocial support was less likely to be motivating factors for both groups.

Figure_OPEN_DIWHY_ADA2019

Conclusions

Our findings suggest that people using Do-it-Yourself Artificial Pancreas systems and their caregivers are highly motivated to improve their/their children’s diabetes management through the use of this novel technology. They are also able to access and afford the tools needed to use these systems. Currently approved and available commercial therapy options may not be sufficiently flexible or customizable enough to fulfill their individual needs. As part of the project “OPEN”, the results of the DIWHY survey may contribute to a better understanding of the unmet needs of PwD and current challenges to uptake, which will, in turn, facilitate dialogue and collaboration to strengthen the involvement of open source approaches in healthcare.

This is a written version of the oral presentation, In-Depth Review of Glycemic Control and Glycemic Variability in People with Type 1 Diabetes Using Open Source Artificial Pancreas Systems, co-authored by Andreas Melmer, Thomas Züger, Dana Lewis, Scott Leibrand, Christoph Stettler, and Markus Laimer.

APSComponents_Melmer_ADA2019

Artificial Pancreas Systems (APS) now exist, leveraging a CGM sensor, pump, and control algorithm. Faster insulin can play a role, too.  Traditionally, APS is developed by commercial industry, tested by clinicians, regulated, and then patients can access it. However, DIYAPS is designed by patients for individual use.

There are now multiple different kinds of DIYAPS systems in use: #OpenAPS, Loop, and AndroidAPS. There are differences in hardware, pump, and software configurations. The main algorithm for OpenAPS is also used in AndroidAPS.  DIYAPS can work offline; and also leverage the cloud for accessing or displaying data, including for remote monitoring.OnlineOffline_Melmer_ADA2019

This study analyzed data from the OpenAPS Data Commons (see more here). At the time this data set was used, there were n=80 anonymized data donors from the #OpenAPS community, with a combined 53+ years worth of CGM data.

TIR_PostLooping_Melmer_ADA2019Looking at results for #OpenAPS data donors post-looping initiation, CV was 35.5±5.9, while eA1c was 6.4±0.7. TIR (3.9-10mmol/L) was 77.5%. Time spent >10 was 18.2%; time <3.9 was 4.3%.

SubcohortData_Melmer_ADA2019We selected a subcohort of n=34 who had data available from before DIY closed looping initiation (6.5 years combined of CGM records), as well as data from after (12.5 years of CGM records).

For these next set of graphs, blue is BEFORE initiation (when just on a traditional pump); red is AFTER, when they were using DIYAPS.

TIR_PrePost_Melmer_ADA2019Time in a range significantly increased for both wider (3.9-10 mmol/L) and tighter (3.9-7.8 mmol/L) ranges.

TOR_PrePost_Melmer_ADA2019Time spent out of range decreased. % time spent >10 mmol/L decreased -8.3±8.6 (p<0.001); >13 mmol/L decreased -3.3±5.0 (p<0.001). Change in % time spent <3.9 mmol/L (-1.1±3.8 (p=0.153)), and <3.0 mmol/L (-0.7±2.2 (p=0.017)) was not significant.

We also analyzed daytime and nightime (the above was reflecting all 24hr combined; these graphs shows the increase in TIR and decrease in time out of range for both day and night).

TIR_TOR_DayAndNight_Melmer_ADA2019

Hypoglemic_event_reduction_Melmer_ADA2019There were less CGM records in the hypoglycemic range after initiating DIYAPS.

Conclusion: this was a descriptive study analyzing available CGM data from  #OpenAPS Data Commons. This study shows OpenAPS has potential to support glycemic control. However, DIYAPS are currently not regulated/approved technology. Further research is recommended.

Conclusion_Melmer_ADA2019

(Note: a version of this study has been submitted and accepted for publication in the Journal of Diabetes. Obesity, and Metabolism.)

Presentations and poster content from @DanaMLewis at #2018ADA

DanaMLewis_ADA2018As I mentioned, I am honored to have two presentations and a co-authored poster being presented at #2018ADA. As per my usual, I plan to post all content and make it fully available online as the embargo lifts. There will be three sets of content:

  • Poster 79-LB in Category 12-A Detecting Insulin Sensitivity Changes for Individuals with Type 1 Diabetes using “Autosensitivity” from OpenAPS’ poster, co-authored by Dana Lewis, Tim Street, Scott Leibrand, and Sayali Phatak.
  • Content from my presentation Saturday, The Data behind DIY Diabetes—Opportunities for Collaboration and Ongoing Research’, which is part of the “The Diabetes Do-It-Yourself (DIY) Revolution” Symposium!
  • Content from my presentation Monday, Improvements in A1c and Time-in-Range in DIY Closed-Loop (OpenAPS) Users’, co-authored by Dana Lewis, Scott Swain, and Tom Donner.

First up: the autosensitivity poster!

Dana_Scott_ADA2018_autosens_posterYou can find the full write up and content of the autosensitivity poster in a post over on OpenAPS.org. There’s also a twitter thread if you’d like to share this poster with others on Twitter or elsewhere.

Summary: we ran autosensitivity retrospectively on the command line to assess patterns of sensitivity changes for 16 individuals who had donated data in the OpenAPS Data Commons. Many had normal distributions of sensitivity, but we found a few people who trended sensitive or resistant, indicating underlying pump settings could likely benefit from a change.
2018 ADA poster on Autosensitivity from OpenAPS by DanaMLewis

 

Presentation:
The Data behind DIY Diabetes—Opportunities for Collaboration and Ongoing Research’

This presentation was a big deal to me, as it was flanked by 3 other excellent presentations on the topic of DIY and diabetes. Jason Wittmer gave a great overview and context setting of DIY diabetes, ranging from DIY remote monitoring and CGM tools all the way to DIY closed loops like OpenAPS. Jason is a dad who created OpenAPS rigs for his son with T1D. Lorenzo Sandini spoke about the clinician’s perspective for when patients come into the office with DIY tools. He knows it from both sides – he’s using OpenAPS rigs, and also has patients who use OpenAPS. And after my presentation, Joyce Lee also spoke about the overarching landscape of diabetes and the role DIY plays in this emerging technology space.

Why did I present as part of this group today? One of the roles I’ve taken on in the last few years in the OpenAPS community (among others) is a collaborator and facilitator of research with and about the community. I put together the first outcomes study (see here in JDST or here in a blog post form on OpenAPS.org) in 2016. We presented a poster on Autotune last year at ADA (see here in a blog post form on OpenAPS.org). I’ve also worked to create and manage the OpenAPS Data Commons, as well as build tools for researchers to use this data, so individuals can easily and anonymously donate their DIY closed loop data for other research projects, lowering the friction and barriers for both patients and researchers. And, I’ve co-led or led several research projects with the community’s data as a result.

My presentation was therefore about setting the stage with background on OpenAPS & how we ended up creating the OpenAPS Data Commons; presenting a selection of research projects that have utilized data from the community; highlighting other research projects working with the OpenAPS community; announcing a new international collaboration (OPEN – more coming on that in the future!) for research with the DIY community; and hopefully encouraging other diabetes researchers to think about sharing their work, data, methods, tools, and insights as openly possible to help us all move forward with improving the lives of people with diabetes.

That is, of course, quite an abbreviated summary! I’ve shared a thread on Twitter that goes into detail on each of the key points as part of the presentation, or there’s a version of this Twitter/presentation content also written below.

If you’re someone who wants to do research with retrospective data from the OpenAPS Data Commons, you can find out more about it here (including instructions on how to request data). And if you’re interested in prospective research, please do reach out as well!

Full content for those who don’t want to read Twitter:

Patients are often seen as passive recipients of care, but many of us PWDs have discovered that problems are opportunities to change things. My journey to DIY began after I was frustrated by my inability to hear CGM alarms at night. 4 years ago, there was no way for me to access my own device data in real time OR retrospectively. Thanks to John Costik for sharing his code, I was able to get my CGM data & send it to the cloud and down to my phone, creating a louder alarm. Scott and I created an algorithm to push notifications to me to take action. This was an ‘open loop’ system we called #DIYPS. With Ben West’s help, we realized could combine our algorithm with small, off-the-shelf hardware & a radio stick to automate insulin delivery. #OpenAPS was thus created, open sourcing all components of DIY closed loop system so others could close the loop, too. An #OpenAPS rig consists of a small computer, radio chip, & battery. The hardware is constantly evolving. Many of us also use Nightscout to visualize our closed loop data, and share with loved ones.

2018ADA_slide12018ADA_slide 42018ADA_slide 32018ADA_Slide 2

 

 

 

 

 

 

I closed the loop in December of 2015. As people learned about it, I got pushback: “It works for you, but how do you know it’s going to work for others?” I didn’t, and I said so. But that didn’t mean I shouldn’t share what was working for me.

Once we had dozens of users of #OpenAPS, we presented a research study at #2016ADA, with 18 individuals sharing outcomes data on A1c, TIR, and QOL improvements. (See that publication here: https://twitter.com/danamlewis/status/763782789070192640 ). I was often asked to share my data for people to analyze, but I’m not representative of entire #OpenAPS community. Plus, the community has kept growing: we estimate there are more than (n=1)*710+ (as of June 2018) people worldwide using different kinds of DIY APs. (Note: if you’d like to keep track of the growing #OpenAPS community, the count of loopers worldwide is updated periodically at  https://openaps.org/outcomes ).  I began to work with Open Humans to build the #OpenAPS Data Commons, enabling individuals to anonymously upload their data and consent to share it with the Data Commons.

2018ADA_Slide 52018ADA_Slide 62018ADA_Slide 72018ADA_Slide 8

 

 

 

 

 

Criteria for using the #OpenAPS Data Commons:

  • 1) share insights back with the community, especially if you find something about an individual’s data set where we should notify them
  • 2) publish in an accessible (and preferably open) manner

I’ve learned that not many are prepared to take advantage of the rich (and complex) data available from #OpenAPS users; and many researchers have varying background and skillsets.  To aid researchers, I created a series of open source tools (described here: http://bit.ly/2l5ypxq, and tools available at https://github.com/danamlewis/OpenHumansDataTools ) to help researchers & patients working with data.

2018ADA_Slide 10 2018ADA_Slide 9

 

 

 

We have a variety of research projects that have leveraged the anonymously donated, DIY closed loop data from the #OpenAPS Data Commons.

  • 2018ADA_Slide 112018ADA_Slide 12One research project, in collaboration with a Stanford team, evaluated published machine learning model predictions & #OpenAPS predictions. Some models (particularly linear regression) = accurate predictions in short term, but less so longer term when insulin peaks. This study is pending publication, but I’d like to note the challenge of more traditional research keeping pace with DIY innovation: the code (and data) studied was from January 2017. #OpenAPS prediction code has been updated 2x since then.
  • In response to the feedback from the #2016ADA #OpenAPS Outcomes study we presented, a follow up study on #OpenAPS outcomes was created in partnership with a team at Johns Hopkins. That study will be presented on Monday, 6-6:15pm (352-OR).
  • 2018ADA_Slide 13Many people share publicly online their outcomes with DIY closed loops. Sulka Haro has shared his script to evaluate the reduction in daily manual diabetes interventions after they began using #OpenAPS. Before: 4.5/day manual corrections; now they treat <1/day.
  • #OpenAPS features such as autosensitivity automatically detect sensitivity changes and insulin needs, improving outcomes. (See above at the top of this post for the full poster content).
  • If you missed it at #2017ADA (see here: http://bit.ly/2rMBFmn) , Autotune is a tool for assessing changes to basal rates, ISF, and carb ratio. Developed for #OpenAPS users but can also be used by traditional pumpers (and some MDI users also utilize it).

I’m also thrilled to share a new tool we’ve created: an #OpenAPS simulator to allow us to more easily back-test and compare settings changes & feature changes in #OpenAPS code.
2018ADA_Slide 14

  • Screen Shot 2018-06-22 at 4.48.06 PM2018ADA_Slide 16  We pulled a recent week of data for n=1 adult PWD who does no-bolus, rough carb entry meal announcements, and ran the simulator to predict what the outcomes would be for no-bolus and no meal-announcement.

 

  • 2018ADA_Slide 172018ADA_Slide 18 We also ran the simulator on n=1 teen PWD who does no-bolus and no-meal-announcement in real life. The simulator tracked closely to his actual outcomes (validated this week with a lab-A1c of 6.1)

 

 

 

The new #OpenAPS simulator will allow us to better test future algorithm changes and features across a diverse data set donated by DIY closed loop users.

There are many other studies & collaborations ongoing with the DIY community.

  • Michelle Litchman, Perry Gee, Lesly Kelly, and myself have a paper pending review analyzing social-media-reported outcomes & themes from DIY community.
  • 2018ADA_Slide 19There are also multiple other posters about DIY outcomes here at #2018ADA:
  • 2018ADA_Slide 20 There are many topics of interest in DIY community we’d like to see studies on, and have data for. These include: “eating soon” (optimal insulin dosing for lesser post-prandial spikes); and variability in sensitivity for various ages, pregnancy, and menstrual cycle.
  • 2018ADA_Slide 21I’m also thrilled to announce funding will be awarded to OPEN (a new collaboration on Outcomes of Patients’ Evidence, with Novel, DIY-AP tech), a 36-month international collaboration assessing outcomes, QOL, further development, access of real-world AP tech, etc. (More to come on this soon!)

In summary: we don’t have a choice in living with diabetes. We *do* have a choice to DIY, and also to research to learn more and improve knowledge and availability of tools for us PWDs, more quickly. We would love to partner and collaborate with anyone interested in working with the DIY community, whether that is utilizing the #OpenAPS Data Commons for retrospective studies or designing prospective studies. If you take away one thing today: let it be the request for us to all openly share our tools, data, and insights so we can all make life with type 1 diabetes better, faster.

2018ADA_Slide 222018ADA_Slide 23

 

 

 

 

A huge thank you as always to the community: those who have donated and shared data; those who have helped develop, test, troubleshoot, and otherwise help power the #OpenAPS and other DIY diabetes communities.

2018ADA_Slide 24

Presentation:
Improvements in A1c and Time-in-Range in DIY Closed-Loop (OpenAPS) Users

(full tweet thread available here; or a description of this presentation below)

#OpenAPS is an open and transparent effort to make safe and effective Artificial Pancreas System (APS) technology widely available to reduce the burden of Type 1 diabetes. #OpenAPS evolved from my first DIY closed loop system and our desire to openly share what we’ve learned living with DIY closed loops. It takes a small, off-the-shelf computer; a radio; and a battery to communicate with existing insulin pumps and CGMs. As a PWD, I care a lot about safety: the safety reference design is the first thing in #OpenAPS that was shared, in order to help set expectations around what a DIY closed loop can (and cannot) do.

ADA2018_Slide 23ADA2018_Slide 24As I shared about my own DIY experience, people questioned whether it would work for others, or just me. At #2016ADA, we presented an outcomes study with data from 18 of the first 40 DIY closed loop users. Feedback on that study included requests to evaluate CGM data, given concerns around accuracy of self-reported outcomes.

This 2018 #OpenAPS outcomes study was the result. We performed a retrospective cross-over analysis of continuous BG readings recorded during 2-week segments 4-6 weeks before and after initiation of OpenAPS.

ADA2018_Slide 26For this study, n=20 based on the availability of data that met the stringent protocol requirements (and the limited number of people who had both recorded that data and donated it to the #OpenAPS Data Commons in early 2017).  Demographics show that, like the 2016 study, the people choosing to #OpenAPS typically have lower A1C than the average T1D population; have had diabetes for over a decade; and are long-time pump and CGM users. Like the 2016 study, this 2018 study found mean BG and TIR improved across all time categories (overall, day, and nighttime).

ADA2018_Slide 28ADA2018_Slide 29ADA2018_Slide 30ADA2018_Slide 31ADA2018_Slide 32

Overall, mean BG (mg/dl) improved (135.7 to 128.3); mean estimated HbA1c improved (6.4 to 6.1%). TIR (70-180) increased from 75.8 to 82.2%. Overall, time spent high and low were all reduced, in addition to eAG and A1c reduction. Overnight (11pm-7am) had smaller improvement in all categories compared to daytime improvements in these categories.

Notably: although this study primarily focused on a 4-6 week time frame pre-looping vs. 4-6 weeks post-looping, the improvements in all categories are sustained over time by #OpenAPS users.

ADA2018_Slide 33 ADA2018_Slide 34

ADA2018_Slide 35Conclusion: Even with tight initial control, persons with T1D saw meaningful improvements in estimated A1c, TIR, and a reduction in time spent high and low, during the day and at night, after initiating #OpenAPS. Although this study focused on BG data from CGM, do not overlook additional QOL benefits when analyzing benefits of hybrid closed loop therapy or designing future studies! See these examples shared from Sulka Haro and Jason Wittmer as example of quality of life impacts of #OpenAPS.

A huge thank you to the community: those who have donated and shared data; those who have helped develop, test, troubleshoot, and otherwise help power the #OpenAPS and other DIY diabetes communities.

And, special thank you to my co-authors, Scott Swain & Tom Donner, for the collaboration on this study. Lewis_Donner_Swain_ADA2018