The Java Message Service (JMS) is a specification that provides a consistent Java API for accessing message-oriented middleware services. This paper presents a test harness that automates the testing of JMS implementations (providers) for correctness and performance. Since the JMS specification is expressed in informal language, a formal model for JMS behaviour is developed, based on the I/O automata used in other group communication systems. The test harness has been successfully used to test a number of JMS implementations. This paper contains a descriptive presentation of the formal model, the full details are found in a technical report