1、The New COM API for Accessibility and Automation in Windows 7, Michael BernsteinPrincipal Software Design EngineerMicrosoft Corporation,Programmatic UI Access: A Key Element of Accessibility,Enables developers to write code that: Navigates between UI elements Gathers information about the UI Interac
2、ts with UI elements Receives notifications when the UI changes,2,UI Automation (UIA): Programmatic Access in Windows,Successor to Microsoft Active Accessibility (MSAA) Creates a universal object model for accessing UI Designed based on experience with MSAA usage Supported on Windows Vista, XP and Se
3、rver 2003 UIA COM Client API introduced in Windows 7,3,UI Automation Core,MSAA,UIA Managed Client API,UIA Provider API,UIA COM Client API,Whats new in Windows 7?,UIA Clients can now use COM instead of managed code New Accessibility properties are available Native clients can register custom proxies
4、to make existing UI more accessible Developers can define custom Accessibility properties and events beyond the official UIA list,Inspecting UIA Properties,demo,Context: Clients and Providers,Client-side API (C# or C+),Client,Provider,UIA Infrastructure,UIA,Provider-side interface definitions (C# or
5、 C+),Test Automation and Assistive Technology Manufacturers,Application and Control Developers,Taking advantage of the new COM Client API,Client,Provider,UIA,SAMPLE: Querying a windows password status,CComPtr spAutomation; CComPtr spElement; HRESULT hr; hr = spAutomation.CoCreateInstance(CLSID_CUIAu
6、tomation); if (SUCCEEDED(hr) hr = spAutomation-GetFocusedElement( ,Automation Elements,Just as in the managed API, an automation element represents a piece of UI, regardless of underlying implementation First, create an instance of CUIAutomation, the factory for the object model Next, get an element
7、: From an HWND: ElementFromHandle From a Point: ElementFromPoint From current focus: GetFocusedElement Each method returns an IUIAutomationElement,Basic Properties,IUIAutomationElement exposes properties that make sense for elements in general Properties are exposed in COM manner and are consistent
8、across all underlying UI,Three Key Properties,NameProperty String that a user would use to explain which control was being referred to (Customer: Accessibility Tools) ControlTypeProperty Primary identifier for what type of control the automation element represents. (Customers: Both) AutomationIdProp
9、erty String that uniquely identifies an element among its siblings in the tree (Customer: Test Automation),Control Patterns,Control Patterns allow access to a controls particular abilities InvokePattern: Push Me TransformPattern: Move Me ScrollPattern: Scroll Me SelectionItemPattern: Select Me Patte
10、rns can contain properties, methods and events,Find Requests,Find requests allow you to locate elements based on conditions Conditions are based on property matching They can include Boolean operators as well Requests can be restricted by scope Find can return the first match or all matches Faster a
11、lternative to manual searching,Visiting the Relatives: TreeWalker,All UIA elements are organized into a tree TreeWalkers allow navigation around the tree GetParentElement, GetFirstChildElement, GetNextSiblingElement, etc. Can filter by condition,A sample control tree:,Consuming Events,Enable clients
12、 to be notified when changes occur in the UI Client implements a listener interface Register your listener with CUIAutomation Clients can scope what parts of automation tree to listen for events on Applicable events are always associated with a reference to automation element where change occurredht
13、tp:/ your access: CacheRequests,Revisiting DumpUIATree,sample,New properties and patterns,Client,Provider,UIA,Patterns for Control Virtualization,Allows controls to have more children than are exposed in Automation tree at one time Parent container permits searching for children by property Children
14、 can be de-virtualized upon request Useful for: Containers with massive numbers of children,Other new properties and patterns,Building great UIA Providers,Client,Provider,UIA,When do you need a provider?,You dont always need to write a provider. Customizing beyond your framework Changing perceived p
15、roperties Annotation API may be sufficient Adding new abilities Full custom controls Creating new frameworks Silverlight had to implement Accessibility,Implementing the simple interface,Create a COM object that implements IRawElementProviderSimple Retrieve VARIANT properties Implement the most impor
16、tant ones for your control, especially Name and ControlType Retrieve a pattern implementation Connect your COM object to the outside world through WM_GETOBJECT,Implementing patterns,Control patterns are much like interfaces Can reside on same object or different object from Simple interface Implemen
17、t patterns that reflect your controls abilities,A basic custom-control UIA provider,sample,http:/ Simple Elements,Representing children Free support for mirroring HWND tree When you have non-window children Implement Fragment for all nodes in your tree Implement FragmentRoot for the root of your tre
18、e Firing Events If a user would notice a change in your control, you need to fire an event You may check for listeners first,Custom patterns and properties,Allows client and provider to exchange data beyond original Accessibility design Custom work must be done on both sides Properties and events: b
19、usiness as usual Methods: extra marshalling work required,Client,Provider,UIA,Custom data,Custom data,Custom proxy support,Enables a client to register a substitute provider for a UI that lacks a real provider Proxy implements UIA interfaces on behalf of UI Useful for legacy code that cannot be modi
20、fied but must be made accessible,Client,Provider,UIA,Custom Proxy,Non-UIA channel,Wrapping it up,Is this all of Accessibility?,Is your software usable with Programmatic access? Keyboard-only access? High contrast / high DPI? Slow reaction times? Accessibility is not a yes/no question: it is usabilit
21、y for a specific audience http:/ about managed clients?,The new COM API was designed with .NET interoperability in mind Pro: Fastest performance for programmatic access Con: Imported COM interfaces are not identical with older interfaces Con: Some Win32 controls have slight behavior differences,UI A
22、utomation Core,UIA Managed Client API,UIA Provider API,UIA COM Client API,Your Client Code; TlbImport Wrapper,Managed client option #2: Use the existing managed library,We continue to support the existing System.Windows.Automation namespace Pro: No change to existing code Con: May be slower when acc
23、essing MSAA applications Con: Cannot access new properties and patterns with current version,UI Automation Core,UIA Managed Client API,UIA Provider API,UIA COM Client API,Your Client Code,Call to Action,Use the new UI Automation Client API to interact with existing user interfaces Create UI Automati
24、on providers for new custom controls and UI frameworks Consider the Accessibility needs of your customers when designing applications, 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in t
25、he U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.,
