It’s a mystery as to why it has taken so long to create a device that makes power wheelchairs smart and sensor-based, but we have the first one now, and it’s called LUCI. In this podcast, we tell the family story behind LUCI and talk about the technology that will improve the safety of persons with disabilities across the globe. Guest: Jered Dean, LUCI’s cofounder and CTO.
Podcast Transcription
Steve Konick: Hi and welcome to The Digital Thread, Protolabs’ podcast that looks at new trends in manufacturing technologies and strategies, cool products, and companies that are pushing boundaries with innovative ideas. Along the way we’ll also give you some design tips to improve how you and your manufacturer work together.
Jered Dean: I really do say. You know this is an area that if somebody is coming in and willing to learn and willing to listen, they will find an amazing community of people and they will find support.
Steve Konick: I'm your host, Steve Konick, thanks for giving us a listen. The first motorized wheelchairs date back to 1916, but believe it or not, the first electric wheelchairs only emerged in 1953. In those early days, battery technology was a big problem, so the wheelchairs didn't travel very far. Over time, though, batteries improved and that meant added freedom for people who use them. With that freedom came exposure to dangers, the kinds of things that nondisabled people encounter every day and don't even think twice about. Joining us to talk about LUCI's cutting edge wheelchair technology is their co-founder and CTO Jered Dean. Welcome, Jered.
Jered Dean: Thanks for having me.
Steve Konick: So, I've been reading up a lot on LUCI, and I understand it's really a family story. Can you tell us about that?
Jered Dean: Yeah, it really is. LUCI started as a project for my niece, Katherine. So, my brother Barry called me and one of our friends who also uses a power wheelchair, had an accident and her power wheelchair that really hurt her. And she just misjudged a curb a little bit and her wheelchair flipped. It doesn't take a whole lot, actually, to flip these wheelchairs and they weigh like 300 pounds, some of them. And so she was using the same kind of power wheelchair as my niece Catherine. And Barry wanted to add some sort of safety features to Catherine's wheelchair because he was worried about her. And that's really what started LUCI trying to find a way to bring Catherine's wheelchair into the modern world with some safety and connectivity and so that she could be a little more independent and Barry and Jennifer could have the peace of mind that she was OK.
Steve Konick: So, what exactly is LUCI, it had to have more from something very rudimentary in your eyes to something very complex, as I understand it.
Jered Dean: It did. LUCI helps prevent power wheelchair users from collisions and falling off ramps or curbs. It's a one-of-its-kind of hardware and software product and it attaches to the power wheelchair to provide security, stability, and connectivity through cloud and sensor fusion technologies. Basically, by attaching LUCI to a new or existing power, wheelchair users gain safety features that we expect on a motor vehicle and connectivity with the modern world.
Steve Konick: These days, safety is intimately tied to communication, so when cars crash, some cars will instantaneously communicate to get help. It seems like you've worked that tech into LUCI, too.
Jered Dean: That's right. LUCI will actually reach out and send an alert to loved ones and caregivers, the user sets up in LUCI that, hey, something's happened, I want to go check on them and even where they're at.
Steve Konick: And like a lot of the greatest inventions, the original concept for LUCI was drawn out on a paper tablecloth at a restaurant. Is that true? And do you still have the early sketches?
Jered Dean: Yeah, that's actually true. Back in 2017 and I had built sort of an initial prototype of what was to become LUCI. It was like Spark Fun and Arduinos and Raspberry Pis and just off the shelf stuff we could cobble together with some Tetrix aluminum building materials. It looked super rough, but it worked! It would stop the chair before Katherine ran into things and it would stop the chair before she drove off of stairs and all. And I had flown into Nashville to install and test this really first prototype. And it happened that there was a major wheelchair equipment convention in town in Nashville there. And so Barry and I decided we'd go to the convention and figure out what we needed to get for Katherine because we were 100 percent sure we'd find safety products, all sorts of options for Katherine at this convention. But there wasn't anything. In fact, we met with a couple of higher-ups, a couple of companies there, and talk to them about collision avoidance and tipping. And they were both a little dismissive. So, I guess it was that night before I headed to the airport, we had dinner at one of those places with the paper tablecloths. And that's sort of when a pet project for my niece turned into LUCI. We started sketching out both the technical side and the business side of what would be needed to make LUCI a thing. And we decided we needed to make this product for Katherine and for her friends and other wheelchair writers weren't going to see this technology in their lifetimes if someone didn't step up and make it happen. And we felt like we were the people that needed to make it happen. So that was that was really a turning point. We kept that sketch really close for a long time until I actually don't know where it is now.
Steve Konick: You don't know where it is now?
Jered Dean: I don't. I hope we could find it because it'd probably be funny to look back at how simple we thought it was going to be at that point.
Steve Konick: The naiveté of youth. OK, so I just I'm stunned that there wasn't competition instantly and that nobody had thought of this before. It seems to me like that's just that that's a given that this sort of technology should be applied to people who are in wheelchairs.
Jered Dean: Yeah. And I'm not sure that it's the thinking of it that was the biggest challenge. Honestly, I think it's kind of a no brainer for power wheelchairs that they should have safety features and something that costs as much as a Tesla should have modern technology on board. I think it's more of an issue of like will and skill. Barry and I had a very personal reason to do this. So, no matter how hard it was, we've got family. And that was sort of coupled with an unusual set of circumstances and blessings that allowed us to follow through on the idea. I've got an engineering background, Barry's got a marketing background, and we sought out these amazing advisors who had revolutionized agriculture and automotive technology. We got connected and found clinicians and users outside our family and had sort of this crazy group of people that then supported an equally unique group of engineers and programmers and roboticists that we put together. So, we were able to make LUCI because we were able to assemble this team of experts and skills. It's sort of a rare thing. There's a cliche, I think, about success is hard work residue. There was a lot of hard work.
Steve Konick: You mentioned all of these people you worked with, they sort of became an extended family. I hear your team has won a lot of design awards along the way. What can you show off for us?
Jered Dean: It's been really amazing and honestly humbling how much recognition LUCI's received in the press. One of our hopes was that we bring awareness to the topics of accessibility and technology access to a broader set of people. And that's been probably the coolest part of these awards. LUCI has been recognized by Popular Science. That one meant a lot to me—in the engineering category, even. Time Magazine Best Inventions. At CES we were an Innovation Honoree and we won one of the Tech for Change Awards there. And then Fast Company just announced that we're on the finalist list for World-Changing Ideas. So it's exciting. And honestly, it's a lot to live up to for the whole LUCI team.
Steve Konick: Well, awards are great and so is finding great people to work with, but what's been the response from the people who matter the most—the customers?
Jered Dean: It's a real game changer for some of our users, there are people succeeding in driving a power wheelchair that have relied on others to push them around for years. You and I take for granted how you know how important independent mobility is. But for many LUCI users, LUCI has given them independent mobility for the first time, or it's allowed them greater freedom on where they can go. Sometimes that increased freedom is coming from just peace of mind and confidence that LUCI is watching their back. It's really emotional to be around some of these customers honestly, and as they're using LUCI. I was just yesterday over a customer's house and she was telling me, “I wasn't allowed to go out by myself. Someone always had to be with me all the time. And I've always had this babysitter telling me how to drive and where to drive. Now I'm more independent.” And that's what it's all about. That's true for my niece, and I hope that's true for all our customers.
Steve Konick: That must be an extraordinary feeling for you.
Jered Dean: It is. It's amazing. You know, they keep us honest, too, though, they're always pushing us to improve LUCI, and that's great. So, it's a blessing.
Steve Konick: And I definitely want to talk to you about the future in just a little bit, because there are so many directions this can go. Getting to the technology itself. My cellphone probably has more sensors in it than I know about it. Even has lied are on it. I mean, it is a powerful little device. Why does technology like that end up in cellphones before it gets added to a product that can so directly help persons with disabilities?
Jered Dean: Yeah, that's a great question on the one hand, the fact that your phone and car have those sensors is what's driving the broad availability of the technology that's so needed here. But the truth is, while I don't know the overall answer to that, there are a couple of factors that really come into play in this market. The first is just a general lack of awareness of the disability community. I mean, honestly, if it weren't for our family members and those we know who use power wheelchairs, I wouldn't be aware of the disparity in technology access. And once people are aware of the community, they tend to be really blown away. From our experience, the disability community is super welcoming and open. After all, one of our advisers says it’s the only group that anyone may join at any time for a whole host of reasons. So, awareness is definitely a factor. And I hope that our efforts bring some of these issues into the mainstream and help with that awareness. But the second thing that really stands out in this industry is the funding system and the industry structure and how that's been built. The fact is that power wheelchair users like my niece aren't treated like consumers or customers.
Jered Dean: They're often told what they can have by equipment providers. And limitations are justified by funding restrictions, which are typically from insurance or government funding sources that measure whether users need technology to—and they use a quote—“Complete the activities of daily living in their home.” So those two things, sort of the structure and the funding compound to really limit technology access, freedom and independence. To be clear, wheelchair users are being told that going to school, going to the grocery store, going to work are not considered crucial or part of their activities of daily living. So, they can't get access to technology that supports those things. By that definition, an iPhone is never going to get funded, right? Cell coverage? Why would you need that? You only need to be supported at home. It doesn't need to work when you go to the store. Netflix? That's not activity of daily living. You don't need that. So, if consumer technology we're running through this ecosystem, through these distributors and funding mandates, we'd all be using flip phones that you could only call for medical assistance from your house. It's a real problem.
Steve Konick: And have you found that insurance companies aren't willing to pay for the addition of LUCI to an existing wheelchair, and has that been tough for your customers?
Jered Dean: I think getting access to technology for customers is difficult in this market. That said, we've had some really early positive signs. We've had some state Medicaids and some insurance companies go, hey, this actually is important. This is something we're going to fund and we hope to have more of those as we as we go forward, right. And not to come across as too negative here. Like, there are so many excuses in this industry. But the truth is there's a whole lot of hunger for more access by users and by their loved ones and by clinicians, and there are some amazing people working really hard on all this stuff outside the industry and in to get people from the technology and equipment they deserve for independent mobility inside their home and in the community anywhere they go.
Steve Konick: It's about fairness and making sure that everybody has access to be able to do the things that they need to do. Sounds like to me. I pulled us slightly off track. Tell me how LUCI itself works, where are the sensors located and what do they do to ensure that a person stays upright in a wheelchair.
Jered Dean: Yeah. LUCI adds a combination of stereo vision cameras, radars, and ultrasonic sensors to the base of the power wheelchair. And then there's a radar that goes on the footplate. So those mount down around the base of the wheelchair and then our computer fuses all of that information into a map of the world around the power wheelchair. And then we listen to the input from the user with the joystick or the head array, or however they're driving. And then LUCI will slow the wheelchair down before a collision with a detected object or before it goes off of the detected step to help keep the wheelchair user safe and to help them navigate the world.
Steve Konick: Everyone is talking about LIDAR these days, but LUCI is a bit more strategic when it comes to sensor technology. Why not use LIDAR?
Jered Dean: Basically, at this phase in the technology LIDAR is too power hungry and LIDAR is also not cost effective. And so we use cameras, radar and ultrasonics and fuse those instead of sort of the single fancy LIDAR approach to doing the same thing. It's also hard to get LIDAR to look around the person. And so we used a different approach on our sensors. I guess at the highest level, we take those three different sensor types, feeds from 360 degrees around the chair, and then we fuse that into one map of the world. Those sensors are running very fast. So, we have an asynchronous system and those sensors are running anywhere from 100 hertz to 12 hertz while you're driving around. And then 10 times a second, LUCI is taking that map and listening to the user input and making a decision on whether the chair needs to slow down before it hits something or drives off an unsafe curve. And so that's at the highest level how LUCI works. We do have GPS onboard and we can track location so that that can be shared out. But because we don't have accurate maps of the places that LUCI drives, we actually have to do this mapping and decision-making in real time with sort of no preconceived notions because we have to work in your living room within a couple of centimeters of things and we have to work in the parking lot in the middle of the day in the wide open. We have this really unique and interesting technical challenge of making good decisions on real-time data with really no context.
Steve Konick: It must be very tough to locate sensors in the front of a wheelchair when there are legs in the way and there are, you know, foot holders, what do you call them? Footpads.
Jered Dean: Yeah. Footpads or footplates.
Steve Konick: Yeah. How did you work around that?
Jered Dean: That's one of the many challenges. You know, if you think about an autonomous vehicle or a warehouse robot, you tend to have nice exterior surfaces that you can mount all your sensors on that are away from whatever the people, passengers or the cargo. And in this case, you know, the user is the important part of the entire equation, right? And so there's no big fender out front that would get in the way. And so you've got to weave these sensors into the chair. Underneath and around the user in a way that they can see enough of the world for us to create this 360 degree map, but where they don't stop you all the time because they're seeing the person. And so that was definitely a challenge for our cameras. We've got some really interesting software onboard that basically recognizes people and ignores them. And then our radars that we developed do a great job of being able to be in places where people are too, and ignore the people.
Steve Konick: How does installation of LUCI affect the wheelchair as a whole, since it's it's an it's an option. It's like an aftermarket option for your for your wheelchair. What does it do to the wheelchair in terms of weight and structure? Just weight distribution?
Jered Dean: Yeah, our system is actually really light and small, so you've got to keep in mind that the base wheelchair weighs anywhere from 300 to 450 pounds without a person in it. These are very, very heavy, bulky pieces of equipment and so LUCI is sort of negligible in that scale, from a weight standpoint. What we really had to focus on more than that was making sure we use as little battery as possible. If you take an autonomous vehicle that's got a supercomputer in the trunk, we can't strap a supercomputer on the wheelchair and use all the battery. And so we needed to be able to do this at a very low power, very efficient. And so we're using ARM processor and really focused on efficiency of our system and then also making sure that we don't limit the seating assembly. So, some of these power wheelchairs, some of the seats will raise or tilt or recline, the leg rests move. So, making sure that the user is still get in all the positions they need to get into to navigate the world without our system limiting that. Those were really the things we were focused on and were challenging.
Steve Konick: When did Protolabs enter the picture, did we start with you at the prototyping stage or did it come later?
Jered Dean: Yeah, Protolabs entered really early on. I don't know the exact date, but I can tell you that I have SLA components from very early prototypes of LUCI that came from Protolabs. And all of our prototype sheet metal has come through Protolabs through the entire project to this day. We continue to work with Protolabs on prototyping, but we also use you guys for our production injection molded components.
Steve Konick: And what were some of the takeaways that you had from the prototyping stage, I'm guessing there were quite a few.
Jered Dean: There were quite a few. Yeah, we actually have a wall in our shop that says “Fail Early” on it. It has all sorts of failed and broken prototypes mounted all over it. And we've tried to build sort of this culture of, OK, we're going to do internal rapid iteration and figure these things out. And that's a lot of prototyping. And so, yeah, we built our system early as a modular system so that we kind of have these sensor pods and a main computer module and then all that is connected by sheet metal. And so we there's I can't even tell you how many rounds of sheet metal and such we went through trying to optimize. How do you put the sensors in the right places and how do you make sure that they're out of the user's way, but they can still see things around the user. And those sorts of tradeoffs were really hard. And so we used particularly sheet metal as a way to move fast and prototype different geometries and layouts really fast.
Steve Konick: It’s pretty typical that when you move from prototyping to production, you’ll go through multiple iterations of sheet metal and 3D printed parts. So, can you talk more about how things flowed and what you learned at each step?
Jered Dean: Yeah, absolutely. So early on, we decided we were going to do this modular structure for LUCI so that we'd have a series of sensor pods and a computer module and those were going to be housed in plastic housings and then those would be positioned around the chair with sheet metal. And so we needed to prototype both of those things. The sheet metal was really easy to prototype through Protolabs, and so we did a lot of that, the housings went through stages, so we started out with SLA models because we could get high precision, rapid parts to physically play with our CAD. And we really found that—You know, we've got a low res printer here on site and that's great if we're just kind of generally checking if something's going to fit. But to really look at molded components, we needed high res prototypes and so we would use SLA and then for mechanical testing of those housings, we would use SLS prototypes so that we had a closer to our production material that we could test with. And so we used both of those sets of prototyping tools to do early verification on our housing for those sensor pods. At a certain point that transitioned over to—and this is a really hard point to know—I think in any project we need to cut metal and actually make it out of the real material with the real service finishes and see how the plastic is really going to flow and all that. But that was sort of the process of we tried to stay in those high res prototypes with using SLS for structural testing, SLA for nonstructural testing. And so we just had to go, OK, now it's time to go ahead and cut some metal.
Steve Konick: I heard from our engineers that the housings for the sensors had to be put together just right and what was so critical about those parts?
Jered Dean: Absolutely, it specifically as we started this journey, we found that off-the-shelf sensors weren't really cutting it. And so of our three sensor types, two of them are ones we had to develop in-house. One of them was a millimeter wave radar. And as we developed our millimeter wave radar system. We found that it was really, really critical how the housing worked. We needed the housing to be invisible to the radar, but we needed to deal with heat dissipation and lots of things. And so we worked with Protolabs engineers. They were great on back and forth on getting the right material for the housings so that it was invisible to the radar. But also things—just the sort of minor details in the mold design—like making sure our ejector pin locations and our gate locations were in the right places to not be in the field of view of our radars. And so there were several parts where they kind of had to do a couple of back flips for us so that we could keep the plastic just right for our radars in the system.
Steve Konick: Now there's the hardware side of LUCI, but all the sensors in the world won't do anything for you if you don't have good code, right?
Jered Dean: We have a team of about a dozen software developers, roboticists, and experts here on-staff that their whole focus now that the hardware and the sensors have been developed, is developing the software on LUCI. So, our software is really cutting-edge and unique in the way that it is a human-in-the-loop robotic system. So, we actually are not trying to replace the human. We're trying to make the human the best driver in the world and assist them. And so it would almost in some ways and in some settings be easier to have a fully autonomous wheelchair. But that's not what my niece is looking for. That's not what the wheelchair users that we work with are looking for. They're looking for independence, not an autonomous wheelchair. There are things they'd like the chair to do, autonomous features or automated features, but that doesn't mean they want to give up control of where they go, the wheelchairs, their legs.
Steve Konick: And just as our brains send signals based on surroundings, the sensors and software have to work together to create a complete virtual environment.
Jered Dean: Yeah, the maps that LUCI uses are. Oh, I'm trying to decide—it's hard because I can geek out on you, but like, there's a fine line here between what people actually are interested in and how geeky I get on this stuff, but… If you think about all these sensors looking out, right. So, imagine the you and I are looking in opposite directions and we don't get to look in the other person's direction and all we get to do is yell out what we see. Right? And then we got a third person that's listening to those and going, they have to have their eyes closed. They're going, huh? Well, that probably means that there's an elephant over there, right? That that's what our software is doing, right? We're listening to multiple feeds from sensors going, “Hey, I see something over here!” And then we have to translate that into what is that relative to the user of the wheelchair. And then we have to calculate what is the wheelchair going to do? And then we need to take action and we need to do this 10 times a second. And so that's the trick to our software. And it's one thing to do that when the robot is just going, “OK, I'm going to drive in a straight line.” It's another thing to do that when Katherine is saying I'm trying to drive around the curve of this sidewalk safely. And so, our software is doing sort of all that ten times a second.
Steve Konick: So LUCI must have gone through extensive testing to ensure that the sensor system worked.
Jered Dean: Well, and that's the other interesting thing is one sensor early testing showed us that one sensor is not enough. Every sensor can be tricked. And so if you look at our sensor coverage, we're overlapping cameras, radars, ultrasonics strategically. Because we need to be able to pick up. As much as anything you throw at it, right, and what you're driving next to, which is an impossible task. There's no way you could ever design a system that could see and detect everything. But by overlapping radar, which is not sonic, ultrasonic, which is sound and vision. Sensors, we've got all these different spectrums. And sensors that look at the world in a different way and we're fusing those into one map so that we try and get the best of all worlds to get the best map we can get for LUCI users.
Steve Konick: Is the onboard software and firmware updatable or is it as is and set?
Jered Dean: It's 100% updatable, so LUCI's always getting better thanks to over the air updates. It's actually one of those places where we see LUCI as a platform for future innovation. We're excited about what we're doing now, but we're going to continue to go further. And, you know, we say here, if it's not good enough for Katherine, my niece, it's not good enough. And so even in our over-the-air update process, our team kind of went a little overboard, probably. But our over-the-air update process is the same kind of process that Airbus uses and Siemens uses for mission-critical hardware. We do this really cool dual A/B partition update process so that if anything ever happens, the user still rolling and it doesn't affect them. Most of our users won't care to geek out on this sort of stuff, but our team cares about it, so.
Steve Konick: It's important, yeah, you don't want to be rolling around and then suddenly an update happens and you're stuck.
Jered Dean: That's right.
Steve Konick: Was there ever a moment in the development of LUCI where you started to feel discouraged that it wasn't coming together for you and you weren't sure this was going to make it?
Jered Dean: Yeah, there have been many of those moments. Honestly, this turned out to be a lot harder project than I gave it credit for when we started. Just as an example, those sensors that I talked about that we had to develop. I told Barry when we first started this project, it was probably that dinner where we were sketching on the napkin. If we have to develop our own sensors, we are doing something horribly wrong. And then we got to a point where we couldn't find sensors that could work in all the environments, you know, if you think about everywhere you go throughout a week, our sensors have to work in all those places, we don't get stripes down the middle of the road, we don't get maps. And so it turned out to be a really challenging problem. And there were moments where, you know, we would have a prototype and it would be it would be pretty good, but it wouldn't be good enough. And you'd have to sit down and and sort of start with fresh eyes, and those were those were tough and there were several rounds of that over the several years it's taken to develop this.
Steve Konick: We keep talking about how LUCI’s development started as a family affair, but it looks like it quickly advanced beyond that—and even created your own growth industry with a lot of caring people on your team.
Jered Dean: Yeah, as far as sort of understanding the problem, we had an ace in the hole, if you will, in that we have more than one actually family member in power wheelchairs. We talk a lot about Katherine, but having those family members and friends that use power wheelchairs is where we started. So, sort of family connections, first-hand experience. And then from that standpoint on, we reached out to local clinics and started working with them and I really do say, you know, this this is an area that if somebody is coming in and willing to learn and willing to listen, they will find an amazing community of people and they will find support. And there's a groundswell, I think, of support here for changing this industry and bringing technology access to this community. And in fact, it sort of feels like accessibility is coming to the forefront right now. And I think that'll change the structure of distribution and funding. And we'll see these things really shift positively. I hope we're part of that. And I'd say I really appreciate, if you look at Microsoft and Google and Apple and a lot of others, they're making accessibility part of technology. And I think that's going to continue to empower and push great change and things that will help the people we love and care about.
Steve Konick: What's the future for LUCI, you've just come out with—It's so unfair of me to ask you this because you just come out with the first LUCI and by the way, is L-U-C-I, folks. It's not L-U-C-Y. It's an “I” at the end? And where do you see it going from here?
Jered Dean: We're just getting started. We've only been for sale for a couple of months and we just released our first major software update already, that improves LUCI's performance in a bunch of scenarios and cuts the power usage, among other things. The truth is that idea that our current product was designed to be the platform is the basis. We had to do the hardware, so we had a canvas to work with and the current software just barely scratches what is possible with LUCI, we're testing all sorts of stuff in-house here that I'd love to tell you about, but I'm not supposed to tell you about. But we'll be announcing new models soon, and there might even be some completely new things coming soon that Protolabs hypothetically is involved in. So, there's lots more to come. But that's probably about all I should say.
Steve Konick: I promise you. I promise I will not tell anyone, OK.
Jered Dean: OK, yeah.
Steve Konick: Jered, it's been a pleasure to have you here today. I've really enjoyed the conversation. And I wish you the best of luck. LUCI sounds like an amazing product.
Jered Dean: Thanks so much. Appreciate it.
Steve Konick: And that's this edition of The Digital Thread, I want to thank our guest, Jered Dean from LUCI for hanging out with us. And don't forget to subscribe to future Digital Thread podcasts through one of our host sites, Apple, Google, or Spotify. Or, you can listen on our website. The Digital Thread is produced by Protolabs, an international manufacturing company with locations across North America, Europe, and Japan. For more information, go to Protolabs.com.