Here is an example that illustrates my question.
My question is:
How do I indicate to user that the command affects a subset of a multiple selection before clicking OK?
Can anyone cite examples of existing products that handle this scenario well?
I don’t know of any existing products that have dealt with this issue the way you’re trying to, but I think you’re taking the right approach. CorelDraw, for example, avoids the issue by not letting users multi-select a mix of editable and non-editable (“locked”) objects even for just viewing read-only information. Likewise, IronCAD won’t let you multi-select objects of certain different classes (e.g., a camera and a part). That seems like an unnecessary restriction to me. Windows XP simply ignores you if try to open a properties window for multiple items of different classes (e.g., My Computer and a pdf file). That could be confusing and frustrating. It seems to me you want to allow the user the greatest flexibility by allowing multi-selection of anything and do as much as possible on whatever is selected.
Here’s some ideas:
First, try to indicate the relevant information in the main/parent window so the user can guess that some actions won’t affect certain selected objects. For example, give read-only objects a distinct appearance, perhaps only on selection (I’m imagining little padlocks for handles for a CAD-type app). Each class of object should have a distinct appearance, perhaps by tagging each with a particular icon. This way, as the users multi-select, they can anticipate what commands do and don’t make sense (e.g., this item is a camera, so it can be moved but obviously can’t be resized).
If the app can’t tell what applies to what until the dialog is opened, then maybe change the appearance of selection of objects for which the action does not apply. For example, when the user changes a property value, all objects that lack that property take a “secondary” selection appearance on the parent window.
If the criteria for applying an action isn’t obvious, you may need some text cues. The menu item or button invoking the action may include in its caption how many items it affects or what it affects (e.g., “Sheet metal only”). In a Properties box, you can include a column beside the column of properties that says many selected items each property applies to or can be changed for.
If that is too space intensive, and all you need to do is indicate that only some objects are affected (not the exact number, identity, or proportion), then maybe you can use a footnote in the dialog. Create a symbol that means “partial,” maybe a half-filled circle? (Don’t use an asterisk –that too often means “required.”) Put this symbol by any control that affects a subset of the selection. At the bottom of the dialog, show the symbol with the text “= applies to only some selected items.”
You are correct that you should indicate that the action only partially applies before the user commits that action, but as a fallback, you could provide some feedback after the action. In general, you want to make the effect of any action visually apparent in your objects in the parent window, and that may be sufficient. However, if you’re still worried about users being confused (e.g., by objects that have scrolled out of view that they later noticed weren’t changed), then maybe you can provide a text notification (self-dismissing) that says “[actioned] [n] of [m] selected items” when the action is completed. Maybe provide a Help link in case the user doesn’t understand why.
It seems you’re cutting a new UI design trail. It’ll be a good idea to test out whatever you decide on users to see if it’ll actually work.