An ARCore Based User Centric Assistive Navigation System for Visually Impaired People - MDPI
Page content transcription
If your browser does not render page correctly, please read the page content below
applied sciences Article An ARCore Based User Centric Assistive Navigation System for Visually Impaired People Xiaochen Zhang 1 , Xiaoyu Yao 1 , Yi Zhu 1,2 and Fei Hu 1, * 1 Department of Industrial Design, Guangdong University of Technology, Guangzhou 510006, China; email@example.com (X.Z.); firstname.lastname@example.org (X.Y.); email@example.com or firstname.lastname@example.org (Y.Z.) 2 School of Industrial Design, Georgia Institute of Technology, GA 30332, USA * Correspondence: email@example.com Received: 17 February 2019; Accepted: 6 March 2019; Published: 9 March 2019 Featured Application: The navigation system can be implemented in smartphones. With affordable haptic accessories, it helps the visually impaired people to navigate indoor without using GPS and wireless beacons. In the meantime, the advanced path planning in the system benefits the visually impaired navigation since it minimizes the possibility of collision in application. Moreover, the haptic interaction allows a human-centric real-time delivery of motion instruction which overcomes the conventional turn-by-turn waypoint finding instructions. Since the system prototype has been developed and tested, a commercialized application that helps visually impaired people in real life can be expected. Abstract: In this work, we propose an assistive navigation system for visually impaired people (ANSVIP) that takes advantage of ARCore to acquire robust computer vision-based localization. To complete the system, we propose adaptive artificial potential field (AAPF) path planning that considers both efficiency and safety. We also propose a dual-channel human–machine interaction mechanism, which delivers accurate and continuous directional micro-instruction via a haptic interface and macro-long-term planning and situational awareness via audio. Our system user-centrically incorporates haptic interfaces to provide fluent and continuous guidance superior to the conventional turn-by-turn audio-guiding method; moreover, the continuous guidance makes the path under complete control in avoiding obstacles and risky places. The system prototype is implemented with full functionality. Unit tests and simulations are conducted to evaluate the localization, path planning, and human–machine interactions, and the results show that the proposed solutions are superior to those of the present state-of-the-art solutions. Finally, integrated tests are carried out with low-vision and blind subjects to verify the proposed system. Keywords: assistive technology; ARCore; user centric design; navigation aids; haptic interaction; adaptive path planning; visual impairment; SLAM 1. Introduction According to statistics presented by the World Health Organization in October 2017, there are more than 253 million visually impaired people worldwide. Compared to normally sighted people, they are unable to access sufficient visual clues of the surroundings due to weakness in visual perception. Consequently, visually impaired people face challenges in numerous aspects of daily life, including when traveling, learning, entertaining, socializing, and working. Visually impaired people have a strong dependency on travel aids. Self-driving vehicles have achieved SAE (Society of Automotive Engineers) Level 3 long back, which allows the vehicle to make Appl. Sci. 2019, 9, 989; doi:10.3390/app9050989 www.mdpi.com/journal/applsci
Appl. Appl. Sci. Sci.2019, 2019,9, 9,x989 FOR PEER REVIEW 2 2ofof16 15 decisions autonomously regarding the machine cognition. Autonomous robots and drones have also been dispatched decisions for unmanned autonomously regarding tasks. Obviously, the machine the advances cognition. in robotics, Autonomous robotscomputer and drones vision, have GIS also (Geographic been dispatched Information for unmanned System),tasks.andObviously, sensors allow for integrated the advances smart computer in robotics, systems to perform vision, GIS mapping, (Geographic positioning, Information and decision-making System), and sensors while allowexecuting in urban for integrated smart areas. systems to perform mapping, Human and positioning, beings have the ability decision-making to interpret while executingtheinsurrounding urban areas.environment using sensory organs. Over Human 90% of the information beings have the transmitted to the brain ability to interpret is visual, and the surrounding the brain processes environment images using sensory tens organs. of thousands Over 90% of theof times faster than information texts. This transmitted explains to the why brain is human visual, andbeings the brainareprocesses called visual creatures; images tens of when traveling, thousands visually of times fasterimpaired than texts. people have to face This explains whydifficulties human beings imposed by their are called visual visual impairment creatures; when . traveling, visually impaired people have to face difficulties imposed by their visual impairment . Traditional Traditional assistive solutions solutions forfor visually visually impaired impaired people people include include white white canes, canes, guide guide dogs, dogs, and and volunteers. volunteers. However, However, each each ofof these these solutions solutions hashas its its own own restrictions. restrictions. They They either either work work only only under under certain certain situations, situations, with with limited limited capability, capability, or are expensive in terms of extra manpower. manpower. Modern Modern assistive assistivesolutions solutionsfor forvisually visuallyimpaired impairedpeople peopleborrow borrowpowerpowerfromfrommobile mobilecomputing, computing, robotics, andautonomous robotics, and autonomoustechnology. technology. TheyThey are implemented are implemented in various in various forms forms such such asterminals, as mobile mobile terminals, portable computers, portable computers, wearable sensorwearable sensor stations, andstations, and indispensable indispensable accessories.accessories. Most of these Most of these devices use devices computer usevision computer vision orto or GIS/GPS GIS/GPS understandto understand the surroundings, the surroundings, acquire aacquire real-timea real-time location,location, and use and use turn-by-turn turn-by-turn commands commands to guideto the guide user.the user. However, However, turn-by-turnturn-by-turn commands commands are for are difficult difficult users for users to follow. to follow. In In this this work, work,we wepropose proposean anassistive assistivenavigation navigationsystem systemfor forvisually visuallyimpaired impairedpeople people(ANSVIP, (ANSVIP, see see Figure Figure 1.)1) using ARCore area learning; we introduce an adaptive adaptive artificial artificial potential potential field field path path planning planning mechanism mechanism that that generates generates smooth smooth and and safe safe paths; paths; wewe design design aa user-centric user-centric dual dual channel channel interaction interaction that that uses uses haptic haptic sensors sensors to todeliver deliverreal-time real-timetraction tractioninformation informationto tothe theuser. user. To To verify verify the the design design ofof the the system, system, wewe havehave the the proposed proposed system systemprototype prototypeimplemented implementedwith withfull fullfunctionality, functionality, and and have have the the prototype prototype tested tested with with blind blind folded and blind subjects. Figure1. Figure Thecomponents 1.The componentsof ofthe theproposed proposed ANSVIP ANSVIP system. system. 2. Related Works 2. Related Works 2.1. Assistive Navigation System Frameworks 2.1. Assistive Navigation System Frameworks Recent advances in sensor technology support the design and integration of portable assistive navigation. advances Recent in sensoran Katz  designed technology support assistive device theaids that design and integration ofand in macro-navigation portable assistive micro-obstacle navigation. Katz  designed an assistive device that aids in macro-navigation and micro-obstacle
Appl. Sci. 2019, 9, 989 3 of 15 avoidance. The prototype was on a backpacked laptop employed with a stereo camera and audio sensors. Zhang  proposed a hybrid-assistive system with a laptop with a head-mounted web camera and a belt-mounted depth camera along with IMU (Inertial Measurement Unit). They used a robotics-operating system to connect and manage the devices and ultimately help visually impaired people when roaming indoors. Ahmetovic  used a smartphone as the carrier of the system, but a considerable number of beacons had to be deployed prior to support the system. Furthermore, Bing  used Project Tango Tablet with no extra sensor to implement their proposed system. The system allowed the on-board depth sensor to support area learning, but the computational power burden was heavy. Zhu  proposed and implemented the ASSIST system on a Project Tango smartphone. However, due to the presence of more advanced Google ARCore, the Google Project Tango introduced in 2014 has been depreciated  since 2017, and smartphones with the required capability are no longer available. To the best of our knowledge, the proposed ANSVIP system is the first assistive human–machine system using an ARCore-supported commercial smartphone. 2.2. Positioning and Tracking Most indoor positioning and tracking technologies were borrowed from autonomous robotics and computer vision. Methods using direct sensing and dead reckoning  are no longer qualified options. Yang  proposed a Bluetooth RSSI-based sensing framework to localize users in large public venues. The particle filter was applied to localize the subject. Jiao  used an RGB-D camera to reconstruct a semantic map to support indoor positioning. They used an artificial neural network on reflectivity to improve the accuracy of 3D localization. Moreover, Xiao [10,11] and Zhang [3,12] used hybrid sensors to carry out fast visual odometry and feature-based loop closure in localization, while Zhu  and Bing [5,13] used area learning (a pattern recognition method) to bond subjects with areas of interest. 2.3. Path Planning As the most popular path planning in robotics, A* is also extensively used by assistive systems. Xiao [10,11] and Zhang  used A* to connect areas of interest. Bing  applied greedy path planning in the label-intensive semantic map. Meanwhile, Zhao  suggested a potential field as a potential candidate for local planning. Paths in Reference [15,16] were planned on well-labeled maps using global optimal methods such as Dijkstra  and its varieties. Most existing path-planning methods generate sharp turn-by-turn paths connecting corner or feature anchors. These paths are good for robots, but bring unbearable experiences to humans. 2.4. Human–Machine Interaction Most present systems use audio to deliver turn-by-turn directional instructions [5,11,18]. However, the human brain has its own understanding for positioning, direction, and velocity [19,20], which is not robot-styled. Some recent works proposed using haptic interfaces in obstacle avoidance [1,3,6,10,13,21–23]. However, due to the restriction of turn-by-turn path planning, the haptic interaction is unlikely to be used in a continuous path-following interface in assistive systems. Fernandes  used perceptual 3D audio as a solution; however, the learning was not easy and the accuracy in real complex scenes needs to be improved. Ahmetovic  conducted a data-driven analysis, which pointed out that turn-by-turn audio instructions have considerable drawbacks due to latency in interaction and limited information per instruction. Guerreiro  stated that turn-by-turn instructions may confuse visually impaired people’s navigation behaviors, and result in, for example, deviating from the intended path. These behaviors lead to errors, confusion and longer recovery times back to the right track. Such behaviors also emphasize that more effective real-time feedback interfaces are necessary. Ahmetovic  studied the factors that cause rotation error in turn-by-turn navigation. Rotation errors accompanying audio instructions significantly affect user experiences in navigation. Rector  compared the accuracy of three different human guidance interfaces and provided insights into the design of multimodal feedback mechanisms.
Appl. Sci. 2019, 9, 989 4 of 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 4 of 16 3. Design of ANSVIP 3. Design of ANSVIP 3.1. Information 3.1. Information Flow Flow in in System System Design Design Most tasks Most tasks in in real real life life are are difficult difficult to to accomplish accomplish using using aa single single sensor sensor or or functional functional unit. unit. Instead, Instead, they require collaboration (cooperation, competition, or coordination) from multiple they require collaboration (cooperation, competition, or coordination) from multiple functional units functional units or or sensing sensing agents agents in intelligent in intelligent systems systems to maketothemakemost the most final favorable favorable finalInsynthesis. synthesis. In such a such a collaborative collaborative context, each context, functionaleachunit functional has its unit ownhas dutyits and owncooperates duty and cooperates via thechannel, via the agreed agreed channel, thereby thereby maximizing the effectiveness of shared resources maximizing the effectiveness of shared resources to achieve the goal. to achieve the goal. Specifically, an Specifically, an assistive assistive navigation navigation system system is is composed composed of of two two parts: parts: the the cognitive cognitive system system andand the guidance system. The cognitive system aims to understand the world, including the guidance system. The cognitive system aims to understand the world, including the micro-scale the micro-scale surroundingsand surroundings andthethemacro-scale macro-scalescene; scene;thethe guidance guidance system system aims aims to properly to properly deliver deliver the micro- the micro-scale scale guidance guidance command, command, the macro-scale the macro-scale plan, plan, as wellas as well as semantic semantic scenescene understanding, understanding, to thetouser. the user. The The collaboration of the two allows the machine to understand the scene, and collaboration of the two allows the machine to understand the scene, and then the user acquires then the user acquires understanding from understanding from the the machine, machine, asas shown shown in in Figure Figure2. 2. Figure 2. The Figure 2. The information information flow flow in in the the assistive assistive system: system: The The assistive assistive system system core core aims aims to to understand understand the the world world and and translate translate the the essential essential understanding understanding to tothe theuser. user. The The proposed proposed ANSVIP ANSVIP uses uses anan ARCore-supported ARCore-supported smartphone smartphone as as the the major major carrier carrier and and uses uses ARCore-based ARCore-based SLAM (Simultaneous Localization and Mapping) to track motion so as to create SLAM (Simultaneous Localization and Mapping) to track motion so as to create aa scene scene understanding understanding along along with with mapping. mapping. The Thehuman-scale human-scaleunderstanding understandingofofmotion motion and and space space is is processed to produce a short and safe path towards the goal. The processed to produce a short and safe path towards the goal. The corresponding micro-motioncorresponding micro-motion guidance guidance is is delivered delivered to to the the user user using using haptic haptic interaction, interaction, while while the the macro-path macro-path cluesclues are are delivered delivered using audio interaction. using audio interaction. Based Based on onthetheinformation information flow in aninassistive flow navigation an assistive system,system, navigation we design we the ANSVIP design the structure ANSVIP as follows: structure as follows: Firstly, Firstly,the thesystem systemshould shouldbe befully fullyaware awareof ofthe the information information related relatedto to the the user’s user’s location location during during navigation. navigation. Unlike GPS-based solutions that are commonly used outdoors, our system has Unlike GPS-based solutions that are commonly used outdoors, our system has to to use use computer computer vision-based vision-based SLAMSLAM sincesince indoor indoor GPS GPS signals signals are are unreliable. unreliable. SLAM SLAM is is based based on on the the Google Google ARCore, ARCore, which which integrates integratesthethevision visionandandinert inertsensor sensorhardware hardwareto tosupport supportareaarealearning. learning. Secondly, Secondly, the system should be capable of conveying the abstracted systemic cognition the system should be capable of conveying the abstracted systemic cognition to to the the user. Unlike the conventional exclusive audio interaction, we propose a user. Unlike the conventional exclusive audio interaction, we propose a haptic-based cooperativehaptic-based cooperative mechanism. mechanism. This This allows allows usus to to replace replace thethe popular popular turn-by-turn turn-by-turn guidance guidance withwith aa more more continuous continuous motion guidance. motion guidance. The The working working logiclogic among among the the ANSVIP ANSVIP components components is is shown shown in in Figure Figure 3.3. Details Details ofof the the major major components are discussed in the following components are discussed in the following subsections: subsections:
Appl. Appl. Sci. 2019, 9, Sci. 2019, 9, 989 5 of of 15 Appl. Sci. 2019, 9, xx FOR FOR PEER PEER REVIEW REVIEW 55 of 16 16 Figure 3. Figure 3. The 3. The working working logic logic among among the among the ANSVIP the ANSVIP components: ANSVIP components: The components: The physical physical components components and and soft soft soft components are components are components shown are shown on shown on the on the left the left hand left hand side hand side and side and right and right hand right hand side, hand side, respectively. side, respectively. respectively. 3.2. 3.2. Real-Time 3.2. Real-Time Area Real-Time Area Learning-Based Area Learning-Based Localization Learning-Based Localization Localization The The system The systemrelies system relieson relies onexisting on existing existingindoor scenario indoor indoor scenario scenarioCAD maps, CAD CAD which maps, maps, which whichare available as escape are available are available mapsmaps as escape as escape near maps elevators near (as requested near elevators elevators (as by fire (as requested requested bydepartments). by fire TheThe fire departments). departments). mapmap The we we map useuse we in this use study in this in this is presented study study in in is presented is presented Figure in 4. Figure Figure We 4. label We the label area the of area interest of on interest the on map the mapso as so to as allow to allowthe thesystem system to to understand understand navigation navigation 4. We label the area of interest on the map so as to allow the system to understand navigation requests requests requests and and to and to plan to plan the plan the path the path accordingly. path accordingly. accordingly. Figure Figure 4. The digital 4. The digital CAD CAD map map before (left) and before (left) (left) and after and after (right) after (right) being (right) being labeled. being labeled. labeled. Google Google ARCore ARCore is is used used to used to track to track the track the pose the pose of pose of the of the system the system in system in navigation. in navigation. The navigation. The sparse The sparse features sparse features are features are are collected collected and andstored in stored an in area an description area dataset description and datasetsubsequently and used subsequentlyin collected and stored in an area description dataset and subsequently used in re-localization. re-localization. used in Specifically, re-localization. aSpecifically, normal-sighted Specifically, person has toperson aa normal-sighted normal-sighted build the person has sparse has to build to build map theof the the indoor sparse sparse ofscenario map of map by running the indoor the indoor scenariothe scenario by ARCore by running running SLAM the in ARCore advance. SLAM Then, in the advance.assistive Then, system the is capable assistive to system re-localize is capable itself to on the re-localize the ARCore SLAM in advance. Then, the assistive system is capable to re-localize itself on the pre-built itself map on theafter pre- pre- entering built map the scenario. after enteringBy observing the scenario. and By recognizing observing the and labeled recognizingtraceable the objects, labeled the traceable built map after entering the scenario. By observing and recognizing the labeled traceable objects, the system is objects,able the to re-localize system is system is able itself able to after roaming to re-localize re-localize itself in the itself after system after roaming roaming in map. in the However, the system system map. the mapping map. However, However, the between the mappingpoints mapping between in the between system points feature-based in the system map and those feature-based mapon the and scenario those on CAD the map has scenario to CAD be points in the system feature-based map and those on the scenario CAD map has to be obtained. obtained. map has to be obtained. We use We use aa singular singular value value decomposition decomposition method method toto find find the the transportation transportation matrix A .. Two matrix A Two groups of corresponding feature point sets are used for finding the homogeneous transformation groups of corresponding feature point sets are used for finding the homogeneous transformation matrix: matrix:
Appl. Sci. 2019, 9, 989 6 of 15 We use a singular value decomposition method to find the transportation matrix A. Two groups of corresponding feature point sets are used for finding the homogeneous transformation matrix: " # cos(θ ) − sin(θ ) t x R 2×2 t 2×1 A= = sin(θ ) cos(θ ) ty . (1) 0 1 0 0 1 Let ln = [ xn , yn ] T denote the point sets in the feature map, and let pn = [in , jn ] T denote the corresponding point set on the scenario CAD map. We use the least squares method to find the rotation R and translation t as follows: N ( R, t) ← arg min ∑ k R × pi + t − li k2 . (2) R,t i =1 N N 1 1 Denote l = N ∑ li , p = N ∑ pi , Equation (2) can be written as i =1 i =1 N 2 R ← arg min ∑ R × pi − l i . (3) R i =1 For M x = b, x1 − y1 1 0 y1 x1 0 1 M= .. .. .. .. . . . . , (4) −yn xn 1 0 yn xn 0 1 x = [cos(θ ), sin(θ ), t x , ty ] T , (5) b = [i1 , j1 , ..., in , jn ] T . (6) Using SVD to decompose M, we find M N ×4 = U N × N S N ×4 V T 4×4 , (7) where U denotes the feature matrix of MM T , S denotes the diagonal matrix with eigenvalue δi , V is the eigenvector matrix of M T M, and we have A in (1) calculated by −1 x = (Vdiag(δ1−1 , ..., δ4−1 )U T ) b. (8) 3.3. Area Learning in ANSVIP ARCore is an augmented reality framework for smartphones with Android operating system. It is an advanced substitute of the deprecated Project Tango. Without the extra depth sensor, an ARCore-powered cell phone is able to track its pose and build a map of the surroundings in real time. Besides, ARCore also enhances the area of interest detection by estimating the average illumination intensity, which helps area segmentation during semantic mapping. The smartphone is a remarkable feat of engineering. It integrates a great number of sensors like a gyroscope, camera, and GPS into a small slab. Specifically, in our work, a HUAWEI P20 with Kirin 970 CPU, Gravity sensor, Ambient light sensor, Proximity sensor, Gyroscope Compass is used.
Appl. Sci. 2019, 9, 989 7 of 15 3.4. Adaptive Artificial Potential Field-Based Path Planning In indoor navigation for the visually impaired, the path planning has to consider both efficiency and safety. Specifically, our path planning considers the issues that follow. 1. The path should be planned to be away from obstacles and risks: Where the conventional robot path planning prefers the shortest path, the assistive system has more in-depth requirements. For visually impaired users, the path should be away from obstacles and risks such as walls, pillars, and uneven steps, which may cause falls . 2. The path and guidance shall be updated in real time: Unlike autonomous robot systems, the assistive system cannot expect visually impaired users to proceed along the planned path accordingly. When the user deviates from the planned path, there should be a corresponding real-time path evolution instead of asking the user to return to the planned track. 3. The mechanism shall be flexible to scale up with new elements: The path-planning algorithm should be able to easily expand with new elements, such as dynamic obstacle avoidance, functional unit integration, step warning, and extreme case re-planning. 4. The path shall be planned in a human-friendly manner: Unlike robots, visually impaired users are unable to grasp precise turning angles, and thus, it is difficult for them to follow conventional turn-by-turn paths . Qualitative direction guidance is more suitable. Users prefer continuous guidance in navigation and a generally smooth plan. The artificial potential field path planning is a suitable candidate for the above issues and challenges, since it has the characteristics of a simple structure, strong practicability, ease of implementation, and flexibility regarding expansion [14,17]. Therefore, we propose an adaptive artificial potential field path-planning mechanism for path generation. Specifically, the target (goal) is considered an attractive potential, while walls are repulsive. The potential fields are the combination of first-order attractive and repulsive potentials: U = Uatt + Urep , (9) Uatt ( Xcurrent ) = kρ( Xcurrent , Xtarget ), (10) 1 1 η ρ( Xcurrent ,Xobs ) − ρ0 if ρ( Xcurrent , Xobs ) ≤ ρ0 Urep ( Xcurrent ) = , (11) 0 if ρ( Xcurrent , Xobs ) > ρ0 where η denotes the repulsive factor, ρ denotes a distance function, and ρ0 denotes the effective radius. A path can be generated along with the potential gradients. However, local minimums may block the path or cause redundant travel costs. Thus, we refer to the local minimum immune A* algorithm path length to control ρ0 and solve this problem. ρ0 = ρ0 − ∆ρ until CAAPF > λCA∗ , where λ denotes the control factor, and CAAPF and CA∗ denote the path length of adaptive artificial potential field (AAPF) and A* from the current position, respectively. A sliding window is used to smooth the path to support and enhance the experience of motion guidance in human–machine interaction (Figure 5): 1 X (i ) = ( X (i + N ) + X (i + N − 1) + ... + X (i − N )). (12) 2N + 1 A case of a smoothed path is shown in Figure 5. Since the plan is discrete, the path (red dotted) is planned in taxicab style before smoothing. The sliding window described in equation (12) updates each point on the path by averaging its position with the nearest 2N points’ positions on the path. Consequently, the path (dark curve) is smoothed after the process.
Appl. Sci. 2019, 9, 989 8 of 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 8 of 16 Figure 5. A5.case Figure of aofsmoothed A case a smoothedpath pathby bysliding window:before sliding window: beforesmoothing smoothing (red (red dotted) dotted) versus versus afterafter smoothing (dark smoothing curve). (dark curve). 3.5. Dual-Channel Human–Machine Interaction A case of a smoothed path is shown in Figure 5. Since the plan is discrete, the path (red dotted) is plannedThein information taxicab styletransfer beforebetween the user smoothing. The and system sliding relies on window human–machine described in equation interactions (12) updates (HMIs). The HMI in an assistive navigation system has certain unique characters. each point on the path by averaging its position with the nearest 2N points’ positions on the First, the HMI doespath. not rely on visual cognition. Second, the HMI is highly Consequently, the path (dark curve) is smoothed after the process.task-oriented. Third, the different information has distinct delivery requirements in situations of urgency or depending on high accuracy. The most popular audio interaction for assistive navigation systems [16,26–28] suffers from the following aspects: 3.5. Dual-Channel Human–Machine Interaction Instruction delay: The instruction delivery is not instantaneous, and the latency becomes a The information bottleneck transfer when dealing between with urgent the user and interaction systemItrelies requests. onwhen is vital human–machine interactions dealing with urgency in navigation. (HMIs). The HMI in an assistive navigation system has certain unique characters. First, the HMI does Limited not rely on visualinformation: The amount cognition. Second, of information the HMI is highlyper message/second task-oriented. is very Third, limited and the different tends information to cause ambiguity, which makes accomplishing tasks with multiple semantics difficult. has distinct delivery requirements in situations of urgency or depending on high accuracy. The most Vulnerable to interference: The user may not be able to access multiple instructions simultaneously, popular audio interaction for assistive navigation systems [16,26–28] suffers from the following and environmental sounds may cause interference. aspects: Result-oriented instructions: The conventional graphical interaction provides many individual Instruction delay: The instruction delivery is not instantaneous, and the latency becomes a small tasks to users, allowing them to choose among different combinations to achieve their bottleneck goals. when Audiodealing with are instructions urgent interaction usually requests. goal-driven It is vital whenand and result-oriented, dealing they with urgency are weak in in navigation. procedure-oriented interaction tasks. Limited Thus,information: we design aThe amount hybrid of information haptic per message/second interaction mechanism as the major is very limited interface and tends to deliver to cause ambiguity, navigation which especially instructions, makes accomplishing micro-motion tasks with multiple instructions. Audio issemantics difficult. used to deliver less-sensitive macro-informative messages. Vulnerable to interference: The user may not be able to access multiple instructions After a path to the target is determined, simultaneously, and environmental sounds may motion causeguidance is generated as shown in Figure 6. interference. Result-oriented instructions: The conventional graphical interaction provides many individual small tasks to users, allowing them to choose among different combinations to achieve their goals. Audio instructions are usually goal-driven and result-oriented, and they are weak in procedure- oriented interaction tasks. Thus, we design a hybrid haptic interaction mechanism as the major interface to deliver navigation instructions, especially micro-motion instructions. Audio is used to deliver less-sensitive macro-informative messages. After a path to the target is determined, motion guidance is generated as shown in Figure 6.
Appl. Appl. Sci. 2019, 9, 989 999 of of 15 Appl. Sci. Sci. 2019, 2019, 9, 9, xx FOR FOR PEER PEER REVIEW REVIEW of 16 16 Figure 6. Figure The motion motion guidance is is generated intersecting the planned path and the awareness cycle. Figure 6. 6. The The motion guidance guidance is generated generated intersecting intersecting the the planned planned path path and and the the awareness awareness cycle. cycle. To To deliver the motion guidance in real time via haptic interaction, interaction, there is is a numerical solution. solution. To deliver deliver the the motion motion guidance guidance inin real real time time via via haptic haptic interaction, there there is aa numerical numerical solution. In In this work, we design haptic gloves as shown in Figure 7. In this this work, work, we we design design haptic haptic gloves gloves asas shown shown inin Figure Figure 7. 7. Figure Figure 7. The 7. The Figure7. design The design of design of haptic of haptic gloves. hapticgloves. gloves. The Theleft leftglove gloveguides guidesthe themotion, motion,andandthe theright rightglove glovewarns warnsof obstacles. Using of obstacles. Using the the middle middle finger finger as as the head front of the subject, the motion directional guidance can be instantaneously delivered the head front of the subject, the motion directional guidance can be instantaneously delivered to to the theuser useras assoon soonasasthe motion themotion plan motionplan is planis made. ismade. made. 4. 4. System System Prototyping 4. System Prototyping and Prototyping and Evaluation and Evaluation Evaluation 4.1. 4.1. System Prototyping 4.1. System System Prototyping Prototyping We We use the HUAWEI P20 as the ARCore-supported smartphone, use Arduino sensors to We use use the the HUAWEI HUAWEI P20 P20 as as the the ARCore-supported ARCore-supported smartphone, smartphone, useuse Arduino Arduino sensors sensors toto implement implement the haptic interactive glove, and use the BAIDU open API for audio recognition. The implement the the haptic haptic interactive interactive glove, glove, and and use use the the BAIDU BAIDU openopen API API for for audio audio recognition. recognition. The The application application is developed in Unity3D. Roberto Lopez Mendez ARCore SLAM is applied as the the base for application isis developed developed in in Unity3D. Unity3D. Roberto Roberto Lopez Lopez Mendez Mendez ARCore ARCore SLAM SLAM is is applied applied as as the base base visual for odometry and area learning. Bluetooth is used to connect the smartphone and the accessory. for visual visual odometry odometry and and area area learning. learning. Bluetooth Bluetooth isis used used toto connect connect the the smartphone smartphone and and the the The ready-to-work accessory. human–machine prototype isprototype shown in Figure 8. Figure 8. accessory. The The ready-to-work ready-to-work human–machine human–machine prototype is is shown shown in in Figure 8.
Appl. Sci. 2019, 9, 989 10 of 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 10 of 16 Appl. Sci. 2019, 9, x FOR PEER REVIEW 10 of 16 Figure 8. The implemented ANSVIP prototype with full functionality. Figure 8. The implemented ANSVIP prototype with full functionality. functionality. 4.2. 4.2. Localization Localization 4.2. Localization To To validate the localization accuracy and reliability, we compare compare the localization of area learning To validate validate the the localization localization accuracy accuracy and and reliability, reliability, we we compare the the localization localization ofof area area learning learning with with visual odometry in an indoor test. Two subjects wearing the system are asked to walk five times with visual visual odometry odometry in in an an indoor indoor test. Two subjects test. Two subjects wearing wearing thethe system system are are asked asked toto walk walk five five times times along along a path in in the the corridor, corridor, one subject with with area area learning learning and and the the other other with with visual visual odometry odometry(VO). along aa path path in the corridor, one one subject subject with area learning and the other with visual odometry (VO). (VO). The The results on Figure 9 are consistent with our expectations: The VO trials suffer from accumulative The results results on on Figure Figure 99 are are consistent consistent with with our our expectations: expectations: The The VOVO trials trials suffer suffer from from accumulative accumulative errors, errors, which cause localization drifts; meanwhile, in the area learning method, there are certain drifts errors, which cause localization drifts; meanwhile, in the area learning method, there are which cause localization drifts; meanwhile, in the area learning method, there are certain certain drifts drifts in in passing corners but the the system is is able to to swiftly correct correct the drift drift by recognizing recognizing learned areas. areas. in passing passing corners corners but but the system system is able able to swiftly swiftly correct the the drift by by recognizinglearned learned areas. Ground Truth Ground VO 1 Truth VO 21 VO VO 32 VO VO 43 VO VO 54 VO VO 5Learning Area Area Learning Figure 9. The ground truth and trajectory of test trails. Figure 9. The ground truth and trajectory of test trails. Figure 9. The ground truth and trajectory of test trails. 4.3. Path Planning 4.3. Path Planning 4.3. Path Planning Simulation comparisons on four different path planning mechanisms are conducted: the adaptive Simulation artificial potential comparisons field (AAPF), on thefour differentartificial path planning potentialmechanisms field withoutare conducted: the Simulation comparisons on fouradaptive different path planning mechanisms a sliding are window conducted: the adaptive (AAPF/S), artificial potential the artificial field potential (AAPF), the adaptive artificial potential field without a sliding adaptive artificial potential fieldfield without (AAPF), thea adaptive repulsive artificial force andpotential sliding window (AAPF/RS), field without and a sliding window the (AAPF/S), A* path planning.the artificial potential field without a repulsive force and sliding window window (AAPF/S), the artificial potential field without a repulsive force and sliding window (AAPF/RS), and On the map, the A* weA* path setpath planning.location as the starting position. Then, 100 random destinations the elevator’s (AAPF/RS), and the planning. On the are generated map, we outside set the elevator’s with alocation as 25 the startingcentered position. Then, 100 random destinations On the map, we setathe circle elevator’s radius of location as themeters starting position.atThen, the starting point, 100 random as shown destinations are in generated Figure 10. outside We use a circle the four with a radius candidate of 25 meters centered path-planning mechanisms at the to starting generate point, the as shown paths in for the are generated outside a circle with a radius of 25 meters centered at the starting point, as shown in Figure 10. start–target We use pairs. the four candidate path-planning mechanisms to generate the paths for the start– Figure 10. We use the four candidate path-planning mechanisms to generate the paths for the start– target pairs. target pairs.
Appl. Sci. 2019, 9, 989 11 of 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 11 of 16 Appl. Sci. 2019, 9, x FOR PEER REVIEW 11 of 16 Figure 10. The 100 generated destinations (star) and the starting position (pentagram) on the map. Figure 10. The 100 generated destinations (star) and the starting position (pentagram) on the map. In Figure 11, In 11, we compare compare the pathpath lengths generated generated by the the four mechanisms. mechanisms. Obviously, Obviously, the the In Figure Figure 11, wewe compare the the path lengths lengths generated by by the four four mechanisms. Obviously, the path path lengths of AAPF are always lower than those of AAPF/S and AAPF/RS because the sliding path lengths lengths of of AAPF AAPF are are always always lower lower than thanthose thoseofofAAPF/S AAPF/S andand AAPF/RS because the AAPF/RS because the sliding sliding window curves window curvesthethesharp sharpturns turnsononthe thepath pathinto intofilleted filleted turns. Therefore, the path length is shorter, window curves the sharp turns on the path into filleted turns. Therefore, the path length shorter, turns. Therefore, the path length is as is shorter, as expected. expected. The TheThe pathpath lengths lengths of of A* A* are always the lowest and are the best among the four. Because as expected. path lengths of are always A* are the lowest always and and the lowest are the arebest the among the four. best among Because the four. A* is Because A* is using using a a greedy greedy mechanism mechanism to to generate generate the theitpath, path, is it is guaranteed guaranteed to produceto produce the the shortest shortest length length when a A* is using a greedy mechanism to generate the path, it is guaranteed to produce the shortest length when aview global global is view is accessible. accessible. However, However, it is noted itthat is noted the thatlength path the path length differences differences between A*between and AAPFA* when a global view is accessible. However, it is noted that the path length differences between A* and is AAPF very is very limited. limited. and AAPF is very limited. 120 120 AAPF AAPF AAPF/S 110 AAPF/S 110 AAPF/RS AAPF/RS A* A* 100 100 90 90 80 80 70 70 60 60 50 50 40 40 0 10 20 30 40 50 60 70 80 90 100 0 10 20 30 40 50 60 70 80 90 100 Trial Trial Figure 11. Simulation Figure 11. Simulation results results on on path path planning planning cost. cost. Figure 11. Simulation results on path planning cost. In Figure 12, we collect the discrete distances from the path to obstacles along the paths. It is In Figure 12, we collect the discrete distances from the path to obstacles along the paths. It is shownIn that Figure 12, and AAPF we collect AAPF/S theproperly discretedeal distances from thetopath with distance to obstacles obstacles, along which is the paths. consistent with Itour is shown that AAPF and AAPF/S properly deal with distance to obstacles, which is consistent with our shown that AAPF and AAPF/S properly deal with distance to obstacles, which is consistent design: The repulsive forces of obstacles keep the path away. AAPF/RS and A* do not have such with our design: The repulsive forces of obstacles keep the path away. AAPF/RS and A* do not have such design: The repulsive forces of obstacles keep the path away. AAPF/RS and A* do not have such
Appl. Sci. 2019, Appl. Sci. 2019, 9, 9, 989 x FOR PEER REVIEW 12 of 12 of 16 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 12 of 16 repulsive forces, forces, and repulsive forces, thus, thus, aaa good and thus, good portion good portion of portion of the of the paths the paths is paths is close is close to close to obstacles. obstacles. This to obstacles. This is is not desired not desired in desired in in repulsive and This is not assistive navigation assistive navigation [6,11]. navigation [6,11]. [6,11]. assistive 5000 5000 0 0 5000 5000 0 0 5000 5000 0 0 2000 2000 1000 1000 0 0 0 1 2 3 4 5 6 7 8 9 10 11 12 0 1 2 3 4 5 6 7 8 9 10 11 12 Figure Figure 12. 12. Simulation Simulation results results on on distances distances to to obstacles. obstacles. Although the path lengths of A* are a bit shorter than those of AAPF, considering the fact that Although the path lengths of A* are a bit shorter than those of AAPF, considering the fact that subjects in navigation are prone totoexperience risks and panics if risky places areare close to the paths, the subjects in in navigation navigation are are prone prone to experience experience risks risks and and panics panics ifif risky risky places places are close close to to the the paths, paths, AAPF the outperforms AAPF A* outperforms in A*keeping in the keeping path the safer. path safer. the AAPF outperforms A* in keeping the path safer. 4.4. 4.4. Haptic Guidance Guidance 4.4. Haptic Haptic Guidance To To verify the directional guidance of the haptic device, we carry out unit tests of the haptic To verify verify the the directional directional guidance guidance of of the the haptic haptic device, device, we we carry carry out out unit unit tests tests of of the the haptic haptic guidance guidance glove. The guidance glove on the left hand and the Arduino joystick to be controlled on the the guidance glove. The guidance glove on the left hand and the Arduino joystick to be controlled on glove. The guidance glove on the left hand and the Arduino joystick to be controlled on the right right hand are are shown in Figure 13. A series series of programmed programmed guidance commands commands are stored and sent right hand hand are shown shown inin Figure Figure 13. 13. A A series of of programmed guidance guidance commands are are stored stored and and sent sent to to the glove so as to let the subject feel the guidance. A blind-folded subject is told to use the joystick to the glove so as to let the subject feel the guidance. A blind-folded subject is told to use the joystick the glove so as to let the subject feel the guidance. A blind-folded subject is told to use the joystick to to depict the directional instruction received. The joystick behaviors are recorded every half second. to depict depict the the directional directional instruction instruction received. received. The The joystick joystick behaviors behaviors are are recorded recorded every every half half second. second. Figure 13. (Left) Figure 13. (Left) Prototype Prototype of of haptic glove. (Right) haptic glove. (Right) Joystick Joystick for for test test purpose. purpose. In Figure 14, the input guidance commands are compared with with the joystick joystick records. Obviously, Obviously, In Figure 14, the input guidance commands are compared with the the joystick records. records. Obviously, there is a latency between the input and records. The latency is caused by three factors: the cognitive there is a latency between the input and records. The latency is caused by three factors: the cognitive delay of human haptic sensibility, the delay from understanding the guidance to controlling the delay of human haptic sensibility, the delay from understanding understanding the guidance to controlling the joystick, and the delay between joystick action and recording. The The average average delay is is less than than 0.4 s, s, joystick, and the delay between joystick action and recording. The average delay delay is less less than 0.4 0.4 s, which which is acceptable in most cases. Note that the delays in later trials are much less than those in earlier which is acceptable acceptable in in most most cases. cases. Note that the delays in later trials are much less than those in earlier trials. One of the reasons for this is that the subject is getting familiar with the haptic interaction. In trials. One of the reasons for this is that the subject is getting familiar with the haptic haptic interaction. interaction. In other words, the subject is capable of efficiently and quickly converting the data perceived by the other words, the subject is capable of efficiently and quickly converting the data perceived by the haptic interaction into their own perception after a few attempts. Thus, a cooperative cognition is built haptic interaction interaction into into their their own own perception perception after after aa few few attempts. attempts. Thus, Thus, aa cooperative cooperative cognition cognition is is between built the assistive system, haptichaptic interaction, and human perceptions. built between between the the assistive assistive system, system, haptic interaction, interaction, and and human human perceptions. perceptions.
Appl. Sci. 2019, 9, 989 13 of 15 Appl. Sci. 2019, 9, x FOR PEER REVIEW 13 of 16 Haptic Glove Guidance Joystick Trail 1 Joystick Trail 2 Joystick Trail 3 Joystick Trail 4 Haptic Glove Guidance Rocker Trail 1 Rocker Trail 2 Rocker Trail 3 Rocker Trail 4 Figure Haptic 14.14. Figure Hapticglove glove guidance versusjoystick guidance versus joystick records. records. 4.5. Integration TestTest 4.5. Integration To verify the prototype To verify the prototype system, we conduct system, we conduct target-oriented navigation target-oriented navigationtests testswith withthree threelow-vision low- vision subjects andsubjects and one one blind blind subject. subject. To evaluate To evaluate thethe human–machine interaction human–machine interactionthat occurs that in our occurs in our systems, we administrate navigation with two different interaction mechanisms. systems, we administrate navigation with two different interaction mechanisms. One with pure One with pure audioaudio instructions instructions   andand thethe otherwith other withhaptic haptic instructions. instructions. Experience Experience surveys are are surveys collected after after collected the tests. A 5-minute tutorial on the navigation instructions is given prior to the tests, and all of the the tests. A 5-minute tutorial on the navigation instructions is given prior to the tests, and all of the subjects are told that security personnel will interfere before any collision or risk is met. This gives subjects are told that security personnel will interfere before any collision or risk is met. This gives the the users peace of mind. users peace of mind. After the test, all four subjects believed they successfully followed the instructions to reach the After target the test, (5/5); mostallsubjects four subjects agreed believed they successfully that the instructions were veryfollowed the instructions easy to understand (4.5/5); to andreach all the targetsubjects (5/5); agreed most subjects that theiragreed cognitionthat the instructions of haptic instructions were verya short enhanced easy while to understand (4.5/5); after beginning the and all subjects agreed experiment that (5/5). their cognition Furthermore, of haptic all subjects agreed instructions enhanced that the haptic a short instructions werewhile after less likely to beginning cause hesitation than the experiment audio (5/5). instructions (5/5); Furthermore, some subjects all subjects agreedbelieved that thethat theyinstructions haptic feel safer than expected were less likely (3.75/5); to cause most believed hesitation than audio that instructions they had a better (5/5);experience with believed some subjects haptic instructions that they than audio than feel safer instructions expected (3.75/5); in most micro-guidance believed that (4.75/5); theyand hadall believed a better that audiowith experience instructions were indispensable haptic instructions than audio as macro-instructions (5/5). Two subjects believed the haptic glove would affect holding objects in instructions in micro-guidance (4.75/5); and all believed that audio instructions were indispensable as daily life and suggested migrating the haptic component to the arm or back of the hand. macro-instructions (5/5). Two subjects believed the haptic glove would affect holding objects in daily life and suggested migrating the haptic component to the arm or back of the hand.
Appl. Sci. 2019, 9, 989 14 of 15 5. Conclusions In this work, we propose a human-centric navigation system to assist people with visual impairment while travelling indoors. The system takes a commercial smartphone as the carrier and uses Google ARCore vison SLAM in positioning. Comparing with the conventional visual odometry-supported travel aids, the system achieves better mapping and tracking. An adaptive artificial potential field-based path planning has been proposed for the system; it keeps the path away from the obstacles so as to avoid risk and collision while generating a real-time smooth path. Finally, a dual-channel human-machine interaction mechanism is introduced in the system. The system user-centrically incorporates haptic interfaces to provide a fluent and continuous guidance superior to the conventional turn-by-turn audio-guiding method. The haptic interaction can be carried out via different candidate devices, but our proposed haptic gloves benefit from an affordable cost and the convenience to plug and play. Evaluation on field tests and simulations shows that the localization and path planning achieves the expected performance, and as such, the proposed ANSVIP system is welcomed by visually impaired subjects. Author Contributions: Conceptualization, X.Z.; Investigation, Y.Z.; Methodology, X.Z. and F.H.; Project administration, F.H.; Resources, Y.Z.; Software, X.Y.; Writing—original draft, X.Z. Funding: This work was funded by the Humanity and Social Science Youth foundation of the Ministry of Education of China, grant number 18YJCZH249, 17YJCZH275. Acknowledgments: The authors would like to thank Bing Li, Jizhong Xiao and Wei Wang for their insightful suggestions regarding this research. We thank LetPub for its linguistic assistance during the preparation of this manuscript. Conflicts of Interest: The authors declare no conflict of interest. References 1. Horton, E.L.; Renganathan, R.; Toth, B.N.; Cohen, A.J.; Bajcsy, A.V.; Bateman, A.; Jennings, M.C.; Khattar, A.; Kuo, R.S.; Lee, F.A.; et al. A review of principles in design and usability testing of tactile technology for individuals with visual impairments. Assist. Technol. 2017, 29, 28–36. [CrossRef] [PubMed] 2. Katz, B.F.G.; Kammoun, S.; Parseihian, G.; Gutierrez, O.; Brilhault, A.; Auvray, M.; Truillet, P.; Denis, M.; Thorpe, S.; Jouffrais, C. NAVIG: Augmented reality guidance system for the visually impaired. Virtual Reality 2012, 16, 253–269. [CrossRef] 3. Zhang, X. A Wearable Indoor Navigation System with Context Based Decision Making for Visually Impaired. Int. J. Adv. Robot. Autom. 2016, 1, 1–11. [CrossRef] 4. Ahmetovic, D.; Gleason, C.; Kitani, K.M.; Takagi, H.; Asakawa, C. NavCog: Turn-by-turn smartphone navigation assistant for people with visual impairments or blindness. In Proceedings of the 13th Web for All Conference Montreal, Montreal, QC, Canada, 11–13 April 2016; pp. 90–99. [CrossRef] 5. Bing, L.; Munoz, J.P.; Rong, X.; Chen, Q.; Xiao, J.; Tian, Y.; Arditi, A.; Yousuf, M. Vision-based Mobile Indoor Assistive Navigation Aid for Blind People. IEEE Trans. Mobile Comput. 2019, 18, 702–714. 6. Nair, V.; Budhai, M.; Olmschenk, G.; Seiple, W.H.; Zhu, Z. ASSIST: Personalized Indoor Navigation via Multimodal Sensors and High-Level Semantic Information. In Proceedings of the 2018 European Conference on Computer Vision, Munich, Germany, 8–14 September 2018; Volume 11134, pp. 128–143. [CrossRef] 7. Fernandes, H.; Costa, P.; Filipe, V.; Paredes, H.; Barroso, J. A review of assistive spatial orientation and navigation technologies for the visually impaired. In Universal Access in the Information Society; Springer: Berlin/Heidelberg, Germany, 2017. [CrossRef] 8. Yang, Z.; Ganz, A. A Sensing Framework for Indoor Spatial Awareness for Blind and Visually Impaired Users. IEEE Access 2019, 7, 10343–10352. [CrossRef] 9. Jiao, J.C.; Yuan, L.B.; Deng, Z.L.; Zhang, C.; Tang, W.H.; Wu, Q.; Jiao, J. A Smart Post-Rectification Algorithm Based on an ANN Considering Reflectivity and Distance for Indoor Scenario Reconstruction. IEEE Access 2018, 6, 58574–58586. [CrossRef]
Appl. Sci. 2019, 9, 989 15 of 15 10. Joseph, S.L.; Xiao, J.Z.; Zhang, X.C.; Chawda, B.; Narang, K.; Rajput, N.; Mehta, S.; Subramaniam, L.V. Being Aware of the World: Toward Using Social Media to Support the Blind with Navigation. IEEE Trans. Hum.-Mach. Syst. 2015, 45, 399–405. [CrossRef] 11. Xiao, J.; Joseph, S.L.; Zhang, X.; Li, B.; Li, X.; Zhang, J. An Assistive Navigation Framework for the Visually Impaired. IEEE Trans. Hum.-Mach. Syst. 2017, 45, 635–640. [CrossRef] 12. Zhang, X.; Bing, L.; Joseph, S.L.; Xiao, J.; Yi, S.; Tian, Y.; Munoz, J.P.; Yi, C. A SLAM Based Semantic Indoor Navigation System for Visually Impaired Users. In Proceedings of 2015 IEEE International Conference on Systems, Man, and Cybernetics, Kowloon, China, 9–12 October 2015. 13. Bing, L.; Muñoz, J.P.; Rong, X.; Xiao, J.; Tian, Y.; Arditi, A. ISANA: Wearable Context-Aware Indoor Assistive Navigation with Obstacle Avoidance for the Blind. In Proceedings of the 2016 European Conference on Computer Vision, Amsterdam, The Netherlands, 8–16 October 2016. 14. Zhao, Y.; Zheng, Z.; Liu, Y. Survey on computational-intelligence-based UAV path planning. Knowl.-Based Syst. 2018, 158, 54–64. [CrossRef] 15. Ahmetovic, D.; Oh, U.; Mascetti, S.; Asakawa, C. Turn Right: Analysis of Rotation Errors in Turn-by-Turn Navigation for Individuals with Visual Impairments. In Proceedings of the 20th International ACM Sigaccess Conference on Computers and Accessibility, Assets’18, Galway, Ireland, 22–24 October 2018; pp. 333–339. [CrossRef] 16. Balata, J.; Mikovec, Z.; Slavik, P. Landmark-enhanced route itineraries for navigation of blind pedestrians in urban environment. J. Multimodal User Interfaces 2018, 12, 181–198. [CrossRef] 17. Soltani, A.R.; Tawfik, H.; Goulermas, J.Y.; Fernando, T. Path planning in construction sites: Performance evaluation of the Dijkstra, A*, and GA search algorithms. Adv. Eng. Inform. 2002, 16, 291–303. [CrossRef] 18. Sato, D.; Oh, U.; Naito, K.; Takagi, H.; Kitani, K.; Asakawa, C. NavCog3 An Evaluation of a Smartphone-Based Blind Indoor Navigation Assistant with Semantic Features in a Large-Scale Environment. In Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility, Baltimore, MD, USA, 20 October–1 November 2017. [CrossRef] 19. Epstein, R.A.; Patai, E.Z.; Julian, J.B.; Spiers, H.J. The cognitive map in humans: Spatial navigation and beyond. Nat. Neurosci. 2017, 20, 1504–1513. [CrossRef] [PubMed] 20. Marianne, F.; Sturla, M.; Witter, M.P.; Moser, E.I.; May-Britt, M. Spatial representation in the entorhinal cortex. Science 2004, 305, 1258–1264. 21. Papadopoulos, K.; Koustriava, E.; Koukourikos, P.; Kartasidou, L.; Barouti, M.; Varveris, A.; Misiou, M.; Zacharogeorga, T.; Anastasiadis, T. Comparison of three orientation and mobility aids for individuals with blindness: Verbal description, audio-tactile map and audio-haptic map. Assist. Technol. 2017, 29, 1–7. [CrossRef] [PubMed] 22. Rector, K.; Bartlett, R.; Mullan, S. Exploring Aural and Haptic Feedback for Visually Impaired People on a Track: A Wizard of Oz Study. In Proceedings of the 20th International ACM Sigaccess Conference on Computers and Accessibility, Assets’18, Galway, Ireland, 22–24 October 2018. [CrossRef] 23. Papadopoulos, K.; Koustriava, E.; Koukourikos, P. Orientation and mobility aids for individuals with blindness: Verbal description vs. audio-tactile map. Assist. Technol. 2018, 30, 191–200. [CrossRef] [PubMed] 24. Guerreiro, J.; Ohn-Bar, E.; Ahmetovic, D.; Kitani, K.; Asakawa, C. How Context and User Behavior Affect Indoor Navigation Assistance for Blind People. In Proceedings of the 2018 Internet of Accessible Things, Lyon, France, 23–25 April 2018. [CrossRef] 25. Kacorri, H.; Ohn-Bar, E.; Kitani, K.M.; Asakawa, C. Environmental Factors in Indoor Navigation Based on Real-World Trajectories of Blind Users. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21–26 April 2018; pp. 1–12. [CrossRef] 26. Boerema, S.T.; van Velsen, L.; Vollenbroek-Hutten, M.M.R.; Hermens, H.J. Value-based design for the elderly: An application in the field of mobility aids. Assist. Technol. 2017, 29, 76–84. [CrossRef] [PubMed] 27. Mone, G. Feeling Sounds, Hearing Sights. Commun. ACM 2018, 61, 15–17. [CrossRef] 28. Martins, L.B.; Lima, F.J. Analysis of Wayfinding Strategies of Blind People Using Tactile Maps. Procedia Manuf. 2015, 3, 6020–6027. [CrossRef] © 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
You can also read
Next slide ... Cancel