Louis Rosenberg is the founder of Immersion, a company that pioneered the development of Force Feedback technology. This interview was conducted by Henner Thomsen as part of the research for a podcast episode.

Stay Forever: You worked on virtual and augmented reality applications; why did you decide to turn Force Feedback technology into a gaming peripheral (via Immersion), instead of other parts of your research involving VR and AR?

Louis Rosenberg: You are correct, my early research at Stanford, NASA, and the U.S. Air Force focused on developing the first interactive augmented reality system – the Virtual Fixtures platform. It allowed users to reach out and interact with a mixed reality of real and virtual objects. When I pitched the concept to the Air Force in 1991, the example application was augmented surgery – to enable surgeons to overlay “virtual fixtures” around the patient that could help them increase their dexterity. A key requirement was that these virtual fixtures feel real – so real, that when a real scalpel hits a virtual fixture, the doctor could slide along its contours and guide their motion with extra precision. That required sophisticated 3D haptic models aligned with the real physical world, creating a convincing mixed reality for the first time. So, my early research showed me the importance of force feedback – it has a major impact on „suspension of disbelief“ (making things seem real) and can greatly enhance human performance in dexterous tasks when added correctly.

So… when I founded Immersion Corporation in 1993, the focus of the company was virtual reality and the mission was to bring immersive technologies to real-world markets. Our early work focused mostly on developing high-end medical simulators for teaching doctors to perform surgeries in VR. And my philosophy from the beginning was that to do it right, you need realistic haptic feedback. So we developed a wide range of advanced technologies.

Early on, I developed our first haptic joystick, mostly as a development tool for working on medical software. But when we gave people demos of the joystick, they loved it. (This old TV clip from 1996 is an example: https://youtu.be/PUTNHCGB7ik) So, as we built our team, we had a major development project to make force feedback joysticks that were viable for consumer markets. We had our first reference design in 1995, I believe and began showing that reference design around. It included a full API and eventually included a custom chip for generating haptic sensations without burdening the PC. I believe it was shown for the first time at Siggraph and Game Developer Conference in 1995. We had interest from many manufacturers and began licensing the design and API. We also developed a reference design for a haptic steering wheel around that time.

The very first force feedback joystick to hit the market was the Force FX from CH Products which I believe shipped in 1996. This was followed by joysticks and steering wheel products that the company licensed to other manufacturers including Logitech, Thrustmaster, Advanced Gravis, Saitek, ACT Labs, Kensington, SC&T, InterAct, Belkin, and eventually Microsoft. And because just about every manufacturer was using our technology and our reference designs and our drivers, we pushed to get support for those peripherals into DirectX. We also put out a full-sized book for game developers on how to program haptic joysticks and wheels and worked closely with all major gaming companies. The Logitech Wingman Force Feedback was probably the highest quality force feedback joystick that was ever on the market – it employed our most advanced technologies. The Logitech Force Feedback Steering wheel was also really well made.

Stay Forever: Why are these products not still popular today?

Louis Rosenberg: The market shifted from PC gaming to console gaming, which meant hand controllers instead of joysticks. At the same time, people moved their gaming habits from sitting at a desk (where you can have a force feedback joystick sitting on a fixed surface) to sitting on a couch where you really can’t have a force feedback joystick in a convenient place. Wheels too were impacted by this shift. Basically, the culture of gaming changed and large joysticks and wheels fell out of favor in the 2000’s. It’s a bummer because people really liked those products and they offered real value, making gaming more realistic and compelling. As VR gets more popular, full-force feedback systems will find a new life.

As VR gets more popular, full-force feedback systems will find a new life.

Stay Forever: You once predicted that „feel will one day be as much a part of computing as sight or sound“, how do you think about this prediction today? Do you think this could still happen?

Louis Rosenberg: Yes, I still believe that haptic feedback will ultimately be a universal part of computing. When I founded Immersion Corporation in 1993, one of my goals was to bring the power of haptic feedback to all computing, both in virtual and augmented worlds, as well as in traditional productivity applications. This is why I pushed so hard to invent the haptic computer mouse, which was much harder than joysticks and wheels. We developed a reference design for two different mice – one that had full-force feedback sensations in two dimensions. And one that used vibration feedback in the z-axis. We showed these haptic mice at CES in 1998 and were overwhelmed with interest. Both of our reference designs were licensed exclusively by Logitech because they were the largest mouse maker in the world at the time. The full-force feedback mouse was produced as the Wingman Force Feedback Mouse and was aimed at gaming. It was really effective for first-person games, giving directional sensations when you were hit from any direction or bumped into things. The smaller mouse was called the iFeel Mouse and was aimed at general productivity. It was fully integrated into the Windows desktop, allowing users to feel menus and icons and windows and buttons and sliders for the very first time. For more details, I wrote an article about these mice on the 25-year anniversary: https://medium.com/@louis_rosenberg/haptics-at-ces-25-years-ago-ba7046778c9a

Of all the work we did in the early 1990s, it was the invention of the haptic mouse that I believe was most consequential. We developed not just the hardware, but the first haptic GUI, creating all the basic techniques for feeling menus and sliders and icons. We also invented the first haptic web pages in 1996, enabling contours and textures. We also conducted the first research studies to show that adding haptics to GUIs could increase performance – users worked faster. To me, those were fundamental milestones in computing and while they’ve not yet permeated mainstream computing – they will. In fact, the haptic GUI methods we pioneered almost 30 years ago are becoming commonplace in virtual worlds, not with mice but with VR handheld controllers. I believe as VR and AR take off, the haptic GUI will become ubiquitous. It just makes sense. This brings me back to where I started, developing mixed reality for the US Air Force back in 1992: when interacting in immersive worlds, haptic feedback is critical for realism and performance.