At the moment, when creating a floatingippools with dnsRecordSet declared in the spec, the DNS record set gets created with all the spec.ips.
This is an issue, because even though the IP from the list is not assigned to any node, it will be part of the DNS resolution, serving the traffic to that IP, resulting in connection timeout/failure.
IMHO the DNS record set should be managed in a way that only the assigned IPs (the ones with status active) should be added to the DNS record set.
When IPs are detached (status becomes unassigned), they should be removed from the DNS resolution.