API 1163 is a recognized standard for in-line inspection system qualification. An extensive part of this standard deals with creation and validation of tool performance specifications. These specifications are required to identify a suitable inspection technology and tool for specific inspection goals. API 1163 knows different validation levels which can be distinguished by the quality they provided and the effort they require. The validation level should be chosen according to the risk profile of a pipeline and the demanded confidence. This paper describes this performance specification creation and validation processes in general and by example of the ROSEN ultrasound crack detection service. The validation includes field verifications to confirm the specified tool performance.