In this paper, we propose a new type of user interface using palm-silhouette, which realizes intuitive interaction on ubiquitous displays located at far from user’s location or interfered in direct operation. In the area of augmented reality based user interface, besides the interface which allows users to operate virtual objects directly, the interface which lets users to interact remotely from a short-distant location is necessary, especially in the environment where multi-displays are shared in public areas. We propose two gesture-related functions using the palm-silhouette interface: grasp-and-release operation and wrist rotating operation which represent “selecting” and “adjustment” respectively. The usability of proposed palm-silhouette interface was evaluated by experiment comparison with a conventional arrowhead pointer. We studied and concluded the design rationale to realize a rotary switch operation by utilizing pseudo-haptic visual cue.