servertest: avoid duplicates

Commit 351a0ce3, which switched from
QList to QVector, also switched a QSet to QVector: this gives as result
duplicate items for the connection results.

Since having duplicate items does not make sense, then switch the
internal storage back to QSet, manually converting it to the QVector
that is emitted as parameter of the finished signal.