Mobile Virtual Reality for Ophthalmic Image Display and Diagnosis
Luo Luo Zheng, BS1, Lingmin He, MD, MS2, Charles Qian Yu, MD3,4
1New York Medical College, Westchester, NY; 2Wilmer Eye Institute, Johns Hopkins University, Baltimore, MD; 3Department of Ophthalmology, Weill Cornell Medical College, New York, NY; 4Illinois Eye and Ear Infirmary, University of Illinois Chicago, Chicago, IL
Corresponding author: charlesy@uic.edu
Supplemental Video 1: Video capture of virtual display showing rhegmatogenous retinal detachment. The clinician feels that he or she is within the eye.
Journal MTM 4:3:35–38, 2015
We here present the use of mobile technology (Samsung Gear VR) in the field of ophthalmology for the display of retinal imaging and demonstrate its use in diagnosis of ocular pathology. In this study a trained retina specialist used the Samsung Gear VR device to view ten wide field photos of retinal pathology, generating a diagnosis for each photograph. Another ophthalmologist then reviewed the ten photos in the traditional manner on a computer. Their diagnoses were compared to the known diagnoses. There was 100% concordance between the device and the traditional method as well as the known diagnosis. The increased display size and realistic presentation of the virtual reality display have the potential to improve diagnosis of ocular disease.
Case details
Virtual reality (VR) has been an emerging technology in the past years. In particular Oculus, a company acquired by Facebook (Menlo Park, CA), has been producing development kits but has not yet released a consumer version. A recent study demonstrated the use of an Oculus Rift device in adjunctive pain control.1 In partnership Oculus and Samsung (Seoul, South Korea) recently released the Gear VR headset to consumers. This device consists of an Oculus-powered head mount into which a Samsung Galaxy Note 4 smartphone is inserted. [Figure 1] The phone provides the computing power and display (2560 x 1440 resolution, 515 pixels per inch), while the head mount incorporates precise accelerometers to track head movements to update the display. The head mount also includes a touchpad on the right side to allow for user interface. The device can be used to display 3D video and audio in an immersive 360-degree manner without any attachments to a computer.
Figure 1: The Gear VR device, showing attached smart phone (A) and oculars (B).
Due to the clarity of the visual axis photographs are very often used clinically in ophthalmology.2 They allow for disease documentation, education, and can aid in making diagnoses, especially of retinal conditions such as macular degeneration and diabetic retinopathy. In particular, the Optomap (Marlborough, MA) system captures ultra wide field (270 degrees) images of the retina.3 These images are currently viewed on a computer monitor or printed out and are distorted in order to display a three dimensional object, the human retina, on a flat surface. (Figure 2)
Figure 2: Wide field retinal photograph. The pathology here depicted is a rhegmatogenous retinal detachment. A 40“ x 20” photograph size incorporating a 8″ black border on each horizontal side allowed display of the image with minimal distortion within the device.
To study the use of virtual reality for display of retinal imaging, we collected ten publically available de-identified wide field photographs of common retinal pathologies with a known diagnosis. [(Table 1)] We then uploaded them into the 360-degree photograph mode of the Gear VR. We found that a 40″x20″ photograph size incorporating a 8″ black border on each horizontal side accounted for the areas of the eye not included in the wide-field photograph and makes for minimal distortion. (Figure 2) Once the ophthalmologist (LH) put the device on, she could turn her head left, right, up or down to examine closely the photograph of the eye. (Supplemental video 1, Figure 3) She was asked to make a primary diagnosis of each retinal pathology pictured. Another ophthalmologist (CQY) then examined the photographs in a traditional manner on a computer monitor and made a primary diagnosis. These were then both compared to the known diagnosis. (Table 1)
Figure 3: (A) and (B) traditional method of viewing retinal images on the computer. (B) and (C) using the virtual display greatly magnifies the image and reduces distortion.
Table 1: Depicted Pathology
The ophthalmologist was able to view all the photographs successfully using the native device interface. There was no significant learning curve and she was able to navigate through the photographs using the touchpad after a few minutes of instruction. The device was comfortable to wear and could be easily adjusted to fit properly. The battery life was more than adequate for extended periods of use. The images were displayed in native resolution without noticeable compression and without blur or color distortion. There was minimal screen door effect (visible pixel borders), which did not hinder use of the device. She did not experience nausea or motion sickness or eye strain.
She was able to see pathology including exudate, drusen, hemorrhage, neovascularization, retinal pigmented epithelium atrophy, retinal tear, retinal detachment, pigmented lesion, and bone spicules. She was able to identify the correct diagnosis in all ten VR displayed photographs as detailed in (Table 1). Therefore, there was 100% concordance between the diagnosis made by the virtual reality display with the traditional method of computer monitor display and the known diagnoses.
Discussion
Our study demonstrates the proof of concept that VR display technology can allow retina photographs to be examined in a more three dimensional manner than with a computer monitor display. There is reduced distortion since the image of the concave retina does not need to be flattened for display. Also the image on the virtual display is an order of magnitude larger, surrounding the viewer’s entire field of view in multiple directions instead of restricted within a monitor. With this technology the ophthalmologist is able to, for the first time, virtually go “inside the eye.”
The virtual display was not in this study superior in diagnostic ability to the traditional method, but this was not unexpected given the clear primary diagnosis demonstrated in each retinal photograph. However we speculate that in cases of subtle findings and multiple diagnoses, the enlarged and more natural viewing provided by a VR display may prove superior. This will need to be confirmed with additional study. In this study the photographs were two-dimensional images converted back to a 360-degree configuration. A more ideal method would be a camera designed specifically to take stereoscopic wide field photos of the eye. This would take full advantage of the Oculus display technology and provide an unprecedented level of fidelity and detail.
A few small barriers remain to widespread adoption of VR technology in the field of retina image display. The device takes time to put on and take off–though in a manner similar to the indirect ophthalmoscope that is commonly used. It is not integrated with the image acquisition systems and therefore is not currently convenient to use in every day clinical practice. Lastly the clinician must also be careful to turn his or her head in order to examine all areas of the photograph so as not to miss pathology that is not apparent in one field of view.
There is no way currently to easily manipulate the image or zoom in on a particular area, which is important for examination of subtle details. We hope there will be development of virtual reality input devices such as gloves or controllers that will allow for improved manipulation and magnification of displayed data. This could allow the device to not only display spherical images but also volumetric data such as optical coherence tomography or computed tomography scans.
New retinal imaging technologies are generating ever higher resolution images of the human eye. Adaptive optics now allows for visualization of the retina on the cellular level, providing images too large to be practically displayed using a computer monitor.4 This enhanced visualization at a cellular level may also accelerate nanotechnology applications, which act on the molecular level of single cellular structures. We feel the use of virtual displays for such applications has great potential for use in research and in the clinic. The future of virtual reality technology in medicine in general is very bright. It is clear to the authors that VR technology is coming to fruition and will find numerous uses. Among many possible uses, it could make telemedicine more personal and practical, could restore quality of life and perhaps virtual mobility to the disabled, and will certainly have uses in the training of physicians especially in surgery.
Conclusion
This use of a virtual display for ophthalmic photographs is the first demonstration of mobile VR technology in the field of eye care. We show that VR technology can already be a useful tool in the eye clinic. With further improvement and development VR technology has the potential to revolutionize many different aspects of medicine.
Disclosures
All authors have completed the Unified Competing Interest form at www.icmje.org/coi_disclosure.pdf (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous 3 years; no other relationships or activities that could appear to have influenced the submitted work.
Acknowledgements
None
References
1. Hoffman HG, Meyer WJ, 3rd, Ramirez M, et al. Feasibility of articulated arm mounted Oculus Rift Virtual Reality goggles for adjunctive pain control during occupational therapy in pediatric burn patients. Cyberpsychol Behav Soc Netw 2014;17:397–401. 
2. Keane PA, Sadda SR. Retinal Imaging in the Twenty-First Century: State of the Art and Future Directions. Ophthalmology 2014;121:2489–500.
3. Gramatikov BI. Modern technologies for retinal scanning and imaging: an introduction for the biomedical engineer. Biomed Eng Online 2014;13:52.
4. Carroll J, Kay DB, Scoles D, et al. Adaptive optics retinal imaging–clinical opportunities and challenges. Curr Eye Res 2013;38:709–21.
