How do you generate evidence faster so that tech transfer can take 17 months instead of 17 years? By using the FASTER Framework. A new approach to quickly generating research evidence for technology-based interventions in AT and Rehab Tech using the FASTER Framework with our guests Drs. Pooja Viswanathan and Rosalie Wang.
Host: Dr. Mary Goldberg, Co-Director of the IMPACT Center at the University of Pittsburgh
Guests: Dr. Pooja Viswanathan, CEO of BRAZE Mobility Inc and Dr. Rosalie Wang, Assistant Professor in the Department of Occupational Science and Occupational Therapy, University of Toronto
FASTER Collective | Website
AGE-WELL NCE | Website, Twitter, Facebook, YouTube
BRAZE Mobility Inc | Website, Facebook, YouTube, Twitter
IMPACT Center | Website, Facebook, LinkedIn, Twitter
FASTER Publication | https://pubmed.ncbi.nlm.nih.gov/33992634/
Wang, R. H., Kenyon, L. K., McGilton, K. S., Miller, W. C., Hovanec, N., Boger, J., Viswanathan, P., Robillard, J. M., & Czarnuch, S. M. (2021). The Time Is Now: A FASTER Approach to Generate Research Evidence for Technology-Based Interventions in the Field of Disability and Rehabilitation. Archives of physical medicine and rehabilitation, 102(9), 1848–1859. https://doi.org/10.1016/j.apmr.2021.04.009
AGE-WELL Workshop Position Paper | Smart Wheelchairs in Assessment and Training (SWAT): State of the Field
Full Episode Transcript | PDF
Timestamps:
01:15 Pooja Background & Intro
01:51 Rosalie Background & Intro
03:13 Barriers to Commercialization in Rehab Tech
08:32 Technology Based Interventions
09:57 Current Approaches for Technology-Based Interventions
16:24 Diving Deeper into the FASTER Framework
23:54 What would success look like?
SPEAKERS
Pooja Viswanathan, Rosalie Wang, Mary Goldberg
Mary Goldberg 00:04
The IMPACT Center at the University of Pittsburgh supported by the National Institute of Disability, Independent Living and Rehabilitation Research, proudly present ImpacTech. On today's episode "17 months instead of 17 years: Using FASTER to accelerate CRT and other assistive technologies to market." We continue our chat with Dr. Pooja Viswanathan and we are also joined by her colleague Dr. Rosalie Wang, assistant professor at the University of Toronto. Recorded remotely from my soundproof bedroom closet in Pittsburgh, PA, this is your host, Dr. Mary Goldberg, and welcome to our 10th episode of the ImpacTech podcast series Pooja, we're delighted to have you back and really excited to have Rosalie join us this time. But for any listeners who did not have privilege of hearing our last conversation Pooja, do you mind briefly reintroducing yourself to the audience?
Pooja Viswanathan 01:15
Absolutely. So my name is Pooja Viswanathan. I am currently the CEO and co founder of Braze Mobility, we make blind spot sensors for wheelchairs that can detect obstacles and provide alert to the user through intuitive lights, sounds and vibrations. So that's the company and then my background is of being an academic. So I did my PhD and postdoctoral research on smart wheelchair technologies, much of which was actually in collaboration with with Dr. Rosalie Wang here, too. So very excited to be on here with her.
Mary Goldberg 01:46
Thank you, and welcome again. Rosalie, can you please also introduce yourself?
Rosalie Wang 01:51
Yes, so I'm an occupational therapist by training. And I'm an assistant professor at the University of Toronto Canada. And the work that I've been involved in is really looking at developing and evaluating technologies primarily for use by seniors and people who have disabilities, and their caregivers and also clinicians, and have a really, really strong interest in looking at things like methodologies for developing an evaluation of technologies. But also the other part of the work that I do is looking at enhancing equitable access to technologies. And so we have a national project that's looking at how we can enhance equity. And also looking at where the service gaps are.
Mary Goldberg 02:31
Thank you. Without even diving into your collaboration, I can already tell how your skill sets are complementary to one another and completely understand why the two of you work together. So I'm interested in learning a little bit more today and sharing with our audience about what you see as some of the major barriers to commercialization specifically in the rehabilitation technology space. This is in part I would say a recap of our last discussion also Pooja so maybe I could ask this first question of you. And then Rosalie, please feel fill in any gaps based on your clinical perspective.
Pooja Viswanathan 03:13
Yeah, I sort of like where to start, right. So many barriers. And very interesting read for some of the listeners, here might be one of our reports that we published, that was a workshop that Rosalie and I co organized back in 2014. And we looked at a very specific application, which was around smart wheelchairs in assessment and training. But what we actually found through that workshop, where a lot of themes and barriers in commercialization that I think are broadly applicable to a lot of assistive technologies, and, you know, the process of our workshop, and itself was quite interesting, because we all came together using a consensus model where the objective was to come to consensus on, you know, perhaps some projects and some grants we could apply to by the end of the workshop. But along the way, we realized that there were actually so many hidden tensions among the people in the room, because we had quite a multidisciplinary group there. We had engineers and clinicians, and you know, people from industry as well. And it was just so interesting to see some of the conflicting objectives that people in the room had. And so it was really interesting, because we came to a point where after the first day, we were looking at Agenda for day two, and we said, maybe before we can start talking about consensus statements, we need to take a step back and actually address the elephant in the room, which was, what are the tensions and the points of disagreement between everyone before we could actually come together and agree on anything, and that was a very useful exercise and the results of that are in the paper on the AGE-WELL NCE website. So if we can make that link available, that would be great. But you know, some of the themes that came up there were around language barriers. There were talks about the valley of death and how you know, oftentimes we get to the point of These controlled experiments in lab. And then that's basically as far as we go. And we really don't have funding to move beyond that. We talked about some of the challenges with randomized control trials with our heterogeneous populations. So that was really, I think, the starting point for us to unveil a lot of challenges around commercialization. So those those are a few biting themes, there's a lot more there.
Mary Goldberg 05:24
You certainly describe the consensus challenge well. I'm curious, did there ultimately end up being agreement on the tensions? Or were there also disagreements on what those tensions were?
Pooja Viswanathan 05:38
It was, it was really interesting, actually, because I think just encouraging people to talk about those tensions really changed the dynamic in the room. And I think it's because before that everyone was trying to be polite, and kind of holding back when they were, you know, hearing things that perhaps were not in line with what their own objectives were, but I think it actually moved us closer to a place of consensus, because we were in a space where some of those differences were being acknowledged. And we were actually eventually able to come to a point where we were able to find some high level consensus statements, perhaps not as low level as we had wanted. But you know, we were able to agree on certain things, like, for example, we need to balance a pie in the sky ideas with low hanging fruit, right? Because researchers are all about the pie in the sky. And clinicians are about well, what can I get my hands on today. And we, you know, we did come to consensus that we need to find a balance, not quite sure how we get there. But it's something that we need to do.
Rosalie Wang 06:35
I just wanted to add, when Pooja talks about tensions, it's a really interesting sort of phenomenon. As we were engaging in this conversation, there were tensions, but I think it's also like, it really did unveil a lot of different complexities within the whole sort of realm of developing and evaluating technologies for the populations that we're working with. And so people with disabilities, and, you know, of all ages, and thinking about, we did come up with, you know, a list of consensus statements related to smart wheelchairs and their applications. But even I think some of the things that came out of that whole workshop, and all of these discussions, were really the complexities around how we come up with our ideas to develop technologies, and what are the complexities in trying to create a meaningful evaluation. And also, you know, what happens once we have something created, we maybe have a proof of concept, and all of the things that happen after that, do we get funding to do more testing, what kind of testing is really the most appropriate, and essentially, we came up with a huge list of different things that potentially could be barriers for how we might be able to, you know, have some form of technology to the users to the clinicians, who can potentially prescribe them. And so it uncovered so many different things just related to smart wheelchairs, but then it was way bigger than we imagined.
Mary Goldberg 08:06
And so central to this whole idea is technology based interventions, and making sure those who are intended to use those technologies are able to access them, and then go through that intervention. So let's take a step back. First, could you please define technology based interventions? And who specifically may use these?
Rosalie Wang 08:32
Yeah, so I'm thinking about technology based interventions, we often look at it from the perspective of assistive technologies, rehabilitative technologies, and so technologies that potentially can enhance somebody's capacity through rehabilitation therapy. So technology's involved in that. And we've also included in our definition, service delivery types of technologies. And so kinds of the types of technologies might be what we might use to support telerehabilitation, for example, and, you know, apps that might deliver a service or our potential clients, and so our clients, essentially, it could be anybody of all ages of any ability, who may benefit from, you know, use of technology in all of those sorts of functions. So assistive to compensate for different abilities or changes in abilities, and also you have rehabilitative and service delivery. So we tried to keep it really broad, just, you know, at the stage that we are kind of looking at broad concepts, we've tried to keep that definition rather broad.
Mary Goldberg 09:33
So the key to seeing the expected change would be having good confidence that the intervention that we're looking at is going to work. And so could you describe some of the current approaches and some of what is described in the paper that are used to gather this evidence that that it does, in fact work?
Rosalie Wang 09:57
So I think just taking Step back in terms of you know, how we develop our technologies, and one of the things that we're really promoting is to be able to evaluate along the way. And so not using a process where, you know, you see a lot of people developing and developing until they get to a concept where they think it's, it's perfect, and then take it for testing. And what we're really trying to do is look at it from a very iterative approach. So thinking about, you know, can we test out this element of this intervention, and test it out with real users in real environments, and to get a lot of feedback before moving on to more development. And so some of the things that we would be concerned about sort of at every step of the way. So in our FASTER model, we have our framework, we have three phases. And each of the phases, we're really, really hoping that testing that's involved throughout at different stages, there will be testing, looking at safety for one thing, whether or not people are potentially going to make use of it. So acceptance, adoption, anticipating whether or not this could be a value to people. So looking at what people's goals are evaluating what the needs are, and being able to assess some of those preliminary things at early stages, but also looking at potential mechanisms for how things might work. So whether or not they might be efficacious, and testing those aspects of it as well. In the lock sort of longer range, once you have something that once we have something that can be potentially deployed, we're looking at things like efficacy and effectiveness, long range goals, as well as short term goals, if there are any potential, you know, harms to us over the long term that might not be able to be identified at earlier stages. And we're constantly looking at things like whether or not the intervention is going to fit the person's environment, the contexts of use, and sort of looking at the scenarios where they might be using it. So we are looking at evaluation sort of along the way for all the phases that we envision, so.
Mary Goldberg 12:04
Why do you think that that type of iterative development and evaluation is so critical in rehab or assistive tech specifically?
Rosalie Wang 12:15
So I'm going to give an answer and then I'm going to hand it off to Pooja because I know she has a lot of thoughts in terms of her work and development to the startup company in this area as well. I think from my clinical perspective, it really is a matter of trying to best match our solutions to what our clients need, and what they would like. And I think being able to have small scale testing really, really helps in refining that. And, you know, we can envision an intervention that we think theoretically works really well. But once we start evaluating with real users and clinicians, we may find out a lot of information that we hadn't even thought of. And so I think having that kind of iterative sort of feedback really, really helps to get our interventions to where they really should be for them to be clinically integrated and implementable.
Pooja Viswanathan 13:06
Yeah. And I learned that iterative prototyping and testing was critical the hard way. Because, you know, I didn't start that way. My research, I had a very typical kind of what's called waterfall methodology and development, you know, where you just develop and develop and develop, and you get it to a stage where you think it's finally ready to be tested. And then you get all this feedback that you're like, oh, man, I wish I had had all this when I started, because maybe I would have built something different. So I did go through a non iterative process, and then switched over to a far more iterative process in my postdoctoral work, where it really helped us very quickly test different types of interventions that were not necessarily fully developed. So I am a huge advocate for Wizard of Oz style of prototyping. So for those who are not familiar with Wizard of Oz studies, it's the way that I like to describe it as fake it till you make it. And so we often using Wizard of Oz would simulate the kind of system that we were hoping to build. And in doing that, we're kind of offering the end user have similar experiences they would have that was the real technology, but without all of that investment of actually building the technology. So in our case, in smart wheelchairs, for example, it meant that we had a hidden researcher that was remote controlling the chair, you know, pretending that it was a smart wheelchair. So people got to experience what a smart wheelchair and all the different interventions and the ways in which a smart wheelchair might function. They got, they got to experience that, but we didn't spend years and years and years developing and we're able to get such great insights from that, that it kind of directed, you know, a lot of our future research and then also directed where I took the company and the kind of product that I built. It's really, really key because you really don't know what the user needs and wants and often times a user might not know what they need and want until they experience it. And so it's one thing to ask someone, you know, how would you like this, this sort of technology. And it's another thing to actually have them experience it. Taking that iterative approach, evaluating early, getting feedback early and often is just such, it's so critical, because it also saves so many resources. And this is kind of also the basis for lean startup methodology, which Hint Hint, we will be talking about more in our future publications. But I think there's some great lessons to be learned from the Lean Startup model. And that whole kind of way of kind of doing problem interviews and solution interviews and getting all of that feedback early on to really inform what is the product that you really need to build and what's going to be the product, but also someone is going to pay for it, which is often the biggest question.
Mary Goldberg 15:56
So, Pooja had you had this framework from the beginning, you would have truly gotten through each stage, the development, the feedback, the formal testing, and of course, most importantly, getting the product into the hands of the user faster. Right. And so, with that segue, I'm hoping we can dive a little bit more into the framework, the main focus of this episode with a brief definition. And why don't we start with you for that? Rosalie.
Rosalie Wang 16:24
Yeah, so we came up with the FASTER, the Framework for Accelerated Systematic Technology-based intervention Evaluation Research, if I get that correct.
Mary Goldberg 16:37
It's a mouthful, but yeah, that that's right.
Rosalie Wang 16:40
Yes. And we're really happy that the acronym actually is so much easier to say. So with FASTER, we wanted to be able to outline, potentially a different process that developers and could be academic researchers, clinical developers, or individuals working in different areas in industry potentially might be able to use as a guide to in some ways, of course, you know, everybody has a lot of different expertise. And we're really, really hoping that the paper starts a lot of this discussion as well, because we said in the paper as well, that we haven't got all of the components in there. And it's a good starting point, but it's nowhere near as comprehensive as we would like. And some of it is opening up for a lot of discussion. So some of the things that we wanted to really incorporate in FASTER is some critical elements based on our discussions from our smart wheelchair workshop, but also from our sort of collective experiences. And if you can see from the list of authors, we're really, really privileged to have so many great minds involved in this process. And so, you know, individuals who've been working in sort of the assistive technology world for many, many years, creating different kinds of interventions and being involved in that realm for a really long time as well. So lots and lots of expertise. So we're really happy to have our team and yeah, being able to come up with from our collective experiences, things like what are some of the principles that we have to be sure to incorporate within all of the development and evaluation processes? And what are some of the key features that we'd like to see. So we talked about Iterative development, we, you know, prefer, in a rapid prototyping, to be able to test things quickly be able to, you know, check out our hypotheses and test some things with users to get their feedback. We're also thinking, you know, other facets would be being able to incorporate heterogeneous users within the study population. Because what we do know from doing clinical studies we have often, we'd like to be able to control as many factors as possible. But that really isn't the reality that we have working with populations are often small, have very hetero genius characteristics, and also need a lot of customization for their interventions to work really well. And so if we take an approach that proactively tries to include people, we might be in a better position to create the interventions that would be more likely to be integrated. And so some of those ideas we wanted to embed in that and it is quite a divergence from what we think of as traditional or conventional clinical research, which really is trying to ultimately get to the stage where we do our pilot studies, do our feasibility studies, but then get to the ultimate randomized control trial that will tell you the answers. And we just recognize that it's sometimes what we get from a randomized control trials. We're not necessarily asking the questions that will be the most beneficial for translation into practice, but we're also not necessarily getting the information that we need to be able to improve the technologies or the interventions that we would like and then we are So exclude a whole set of people who might potentially be able to benefit. And they're not able to be involved in the studies because they need to manage the heterogeneity. And so it leads to a lot of different areas of potential exclusion, where our solution might not be catered towards some of those folks, but also that we don't have data for how well it might work for those folks either. And so we wanted to make sure that we proactively included people who might benefit within that process right at the start, and, you know, towards the deployment testing stage, so and then we also wanted to include sort of four guiding principles as well within the framework. So thinking about ethical development, which is also always a primary area. But once we kind of dissect what all of the elements of ethical development might be, there's a lot of different dimensions that can be included as well. We also wanted to make sure that we have explicitly a transdisciplinary and trans, sectoral approach. So again, with the idea of being very inclusive, we're proactively involving clinicians, engineers, computer scientists, psychologists, anybody who might be involved in this process, and of course, consumers, so users and clinicians and industry and policy makers or people who might be funding the technologies to be able to come together and be involved in creating what would be impacting them at some point with the interventions. And yeah, some of the other sort of dimensions around this is looking at sort of process evaluation as well. So thinking about how we evaluate at each stage of our testing, whether or not we are conforming to different aspects of how well we're actually conducting our studies, in terms of are we sort of including everybody that we wanted to include and be able to judge whether or not the studies were carried out the way that we would like, and of course, because there's a lot of potential, we're sort of working with small samples, and a lot of different factors involved. Transparency is really, really critical in this in the soundstage, just because we know that we want to reduce sort of risks for bias, because, you know, for working with small samples, we the biggest question that will come up is how do you know that your intervention is effective, but we want to make sure that biases are revealed and discussed explicitly. So that element of transparency is really, really critical within, you know, the process evaluation component of it. And there's another principle that I'm missing right now, user engagement, so of course!
Mary Goldberg 22:43
Which is right, so key, and something that you both stressed early on, I think in your examples of what is absolutely critical, and the user engagement piece, I think, above all other principles, right, it is so critical for AT. You do such a nice job of explaining the framework. The the paper itself is fairly dense. For those of you who are interested in learning more or reading the paper, it's called The Time Is Now: A FASTER Approach to Generate Research Evidence for Technology-Based Interventions in the Field of Disability and Rehab. And it is in the archives of Physical Medicine and Rehabilitation. And so, in this framework, it sounds so helpful, and potentially something that I think could result in a paradigm shift. And so I'm curious, in your words, I guess both of you, again, wearing the different hats that you wear as an academic Rosalie and as somebody working in industry or a small business Pooja, what would success look like for you? If this were to be implemented by both academics and industry alike?
Pooja Viswanathan 23:54
Yeah, I think adoption of this framework, it would in itself be a success, because hopefully, we would see the acceleration and have things move faster. You know, that's really the point of all of this. Right now, I think that's the numbers are like 17 years, or something to get from the lab to the real world. And so this is really of urgency right now. Because there's so much investment in research and development. And we're really not seeing the impact of a lot of this technology because of a lot of these barriers. And so we obviously need to do something about it. And this framework, hopefully is a starting point. And certainly not like Rosalie mentioned doesn't cover all the aspects, I think we still have a lot of other aspects to cover like regulatory side and actual details around the development process and, and how people can actually develop in ways to get things out there faster. So success for me really looks like getting more and more innovative technologies that are actually being used and not collecting dust in research labs. And for me, I think I do want to also note because they're has been a lot of discussion in the world of CRT and complex rehab around, you know the difference between CRT and durable medical equipment. And I think this framework that we're talking about, again, lends itself really well to evaluating technologies in CRT, where that customization piece and the fact that you know, every product that's used by every consumer might be unique. And so offering this alternative to RCTs. And shifting away from this notion of standardization, and really looking at Case Study approaches, and single subject design is going to be so key in also generating evidence for CRT. So I think this is very, very relevant for my specific area. And I think a lot of other stakeholders in CRT would be, I think, quite interested in this and it would be great to see more uptake, especially in the CRT world where it's so relevant.
Mary Goldberg 25:53
Thanks, Pooja. And how about you? Rosalie?
Rosalie Wang 25:56
Yeah, I think a long term success is very much what Pooja had said. Ultimately, if we're able to expedite our being able to develop and get good evidence for our technologies and our technology based interventions, that would be, you know, the ultimate goal and the ultimate sort of success. In the short term, I think, something as simple as the realization. And of course, the processes have been discussed a lot in different areas as well like randomized control trials, or not working in a lot of different areas. But it's almost like the, when we say the time is now it's time to really sort of liberate ourselves from that paradigm of thinking that we have to ultimately get to the randomized control trial to find the definitive evidence for our interventions, and then follow that path for systematic reviews and things like that. But I think if somebody reading this feels like this paradigm is not working for us, and it's ultimately not serving our purpose, can we let it go and try something different? And I think that that sort of in the short term would already be a success and sort of my view, because it's really kind of shifting our thinking to something that potentially might be more useful in the long run. And of course, it remains to be shown how useful that might be in the future, but hoping people will take a chance with it. Yeah.
Mary Goldberg 27:23
Thanks so much, Rosalie. So listeners, I hope we've helped you digest the framework a bit and motivate you to personally use FASTER and I personally can't wait to see what is next for the both of you, what products come out of your respective labs next, but also how else you can help to implement this FASTER framework with some of the proposed approaches that you've nicely detailed in the paper. So I will definitely be keeping the pulse on this and encourage our listeners to as well. And so where can our listeners learn more? And how can they stay up to date.
Pooja Viswanathan 28:01
So we've actually have launched a website called faster-collective.com. And you can find our publications there and all of our future work will be updated on the website as well and there's a Contact Us page so that would be a great place to look up all of our work around the FASTER framework.
Mary Goldberg 28:21
Thank you so much for your time, both Rosalie and Pooja.
Rosalie Wang 28:25
Thank you.
Pooja Viswanathan 28:26
Thank you so much for having us.
Mary Goldberg 28:28
Thanks for tuning into our episode. We will be taking a short break for the holidays but we will return again in January 2022. So please stay tuned as we continue our tour of our friends and colleagues in Canada to catch up with doctors Alex Mihailidis and Tilak Dutta from the research arm of the Toronto Rehabilitation Institute KITE. Find out more about where Pooja got her start in our next episode "Collaboration is key for KITE." Until then, if you like ImpacTech, please review us on Apple podcasts or wherever you listen to podcasts. Thank you again for tuning in and continue to make an impact in whatever you do. A quick note from our sponsors. IMPACT initiatives are being developed under a grant from the National Institute on Disability Independent Living and Rehabilitation Research. NIDILRR is a center within the Administration for Community Living Department of Health and Human Services. IMPACT initiatives do not necessarily represent the policy of NIDILRR, ACL or HHS, and you should not assume endorsement by the federal government and the same goes for the University of Pittsburgh. We would like to thank our ImpacTech guests and our production team led by Dr. Michelle Zorrilla at the University of Pittsburgh Department of Rehabilitation Science and Technology