gnutls running in the FIPS140-2 mode does not allow MD5 to be
initialized, which breaks chronyd using MD5 to calculate reference ID
of IPv6 addresses. Specify a new hash algorithm for non-security MD5 use
and temporarily switch to the lax mode when initializing the hash
function.
For consistency and safety, change the CMC and HSH functions to accept
signed lengths and handle negative values as errors. Also, change the
input data type to void * to not require casting in the caller.
Tomcrypt, some NSS hash functions, and the internal MD5 require the
output buffer to be at least as long as the digest. To provide the same
hashing API with all four options, use an extra buffer for the digest
when necessary and copy only the requested bytes to the caller.
Some hash functions in the freebl3 library don't support truncated
digests and either return immediately with no update of the output
length, or ignore the length of the output buffer and always write whole
digest.
Initialize the return value to zero to get correct result with the
former.
This is triggered only in the hash unit test. chronyd always provides a
sufficient buffer for the digest.
This should reduce the number of possible memory leaks reported by
valgrind. The remaining reported leaks are sched tqe allocation, async
DNS instance allocation, cmdmon response/timestamp cell allocation, and
clientlog subnet allocation.