When Form implements InputFilterProviderInterface and validators for an element are defined in method getInputFilterSpecification, only these validators should be added to the InputFilter of the Form for the element.
The same behavior should be for Fieldsets and it works fine. Basically, validators defined in getInputFilterSpecification should override default validators of the Element.
Originally posted by @michalbundyra at zendframework/zend-form#82