Skip to page content
State of the Science Conference
September 15-16, 2005
Conference
Home
AudioMORPH: Automated Auditory Interface Adaptation
Slides On This Page
- AudioMORPH: Automated Auditory Interface Adaptation
- Access to Graphical User Interfaces in the Workplace
- Adapting a Graphical User Interface
- AudioMORPH: Automating Customization
- AudioMORPH Components
- AudioMORPH Process
- AudioMORPH Process
- Why not Macro Express?
- Example Application
- Without Customization
- Sample Path/Task
- AudioMORPH
-
A JAWS Script
- Future Work
- Contact Information
AudioMORPH: Automated Auditory Interface Adaptation
Melody Moore (Director)
Dan Ratanasit (Software Engineer)
GSU BrainLab, Computer Information Systems, Georgia State University
Access to Graphical User Interfaces in the Workplace
Web based systems are well supported by commercial screen readers…
But what about proprietary, non-web based business applications?
Adapting a Graphical User Interface
Commercial screen readers can be used to navigate any software application,
but they may be cumbersome without customization
Screen readers can be adapted for custom business applications via scripts
Script writing is usually performed by a rehabilitation engineer with
a programming background
Fully customizing a screen reader for a complex application can take weeks
AudioMORPH: Automating Customization
Goals of AudioMORPH:
Reduce dependence on outside professional customization
Automatically generate screen reader scripts based on real usage of a
system
Speed adaptation time for auditory interfaces
Remove barriers to employment
AudioMORPH Components
Automated Audio Interface Generation
- AudioMORPH allows common navigation paths to be captured and generates scripts
for screen reader customization
Target Domain Scripts
- Business information systems applications common to workplaces (databases,
payroll, and financial systems)
AudioMORPH Process
User: Domain Expert
- Reduce need for rehabilitation engineer to program the scripts
- Supervisor or co-worker familiar with target application
Automation
- Captures user’s keystrokes on target application
- Creates simple straightforward screen reader scripts based on captured data
- Allows arbitrary mapping for keyboard shortcuts
- Allows screen reader-specific commands to be added
AudioMORPH Process
Image: A flowchart shows how a domain expert and employee interact with
a computer application. The domain expert interacts with AudioMORPH, which
in turn interacts with the application, scripts, and key mappings. The employee
interacts with a screen reader, which interacts with the application, scripts,
and key mappings.
Why not Macro Express?
AudioMORPH can:
Add screen-reader specific commands
Handle context sensitivity
Automatically generate screen reader compatible scripts
Generate additional configuration files such as key mappings and documentation
files
Example Application
- Image: A screen capture image of a customer database shows an interface
window with:
- Areas for inputting or reviewing a customer’s name and address,
- An area for searching for a particular customer,
- An area for adding, deleting and editing customers,
- Buttons to bring up a customer’s order history or to place a
new order, and
- Links to other parts of the database, such as inventory.
Without Customization
Image: The same application is shown. Several points on the window are highlighted.
Although the application can be navigated with the use of keyboard controls,
such as the tab key for moving to the next field, sometimes for these applications,
it is necessary to skip around between various parts of the application screen.
Sample Path/Task
Add a new customer to the database
Without customization - requires 17 hits of TAB key
With customization - can reduce this to 1 shortcut
AudioMORPH
Image: A screen capture image of the AudioMORPH interface shows a window with
various Path Controls (Start Path, Rename Path, View Path, etc.), boxes for
assigning a name and keystroke shortcut, and buttons to start and stop recording
of the script.
A JAWS Script
Image: A screen capture of a script file shows multiple short scripts of about
6 lines each that announce which window a person has selected for some type
of media program. Each script includes lines of letters and Alt-character combinations,
in brackets, that represent various JAWS or Windows keyboard commands. A SayString
function is used in each script to provide spoken output of which window was
selected
Future Work
Currently completing AudioMORPH prototype
Fall 05 – perform extensive study in real work scenarios with domain
experts
Test usability of resulting scripts with workers with visual disabilities
Compare scripts to those generated by rehabilitation engineers
Contact Information
Melody Moore
melody@gsu.edu
404-463-7150
Dan Ratanasit
dratanas@cis.gsu.edu
404-463-7121
www.cis.gsu.edu/brainlab
Top of Page