...
It is also difficult to check for math errors using errno because an implementation might not set it. For real functions, the programmer can test whether the implementation sets errno because in that case math_errhandling & MATH_ERRNO is nonzero. For complex functions, the C Standard C99 Section 7.3.2 simply states that setting errno is optional.
...
The System V Interface Definition, Third Edition (SVID3) provides more control over the treatment of errors in the math library. The user can provide a function named matherr that is invoked if errors occur in a math function. This function could print diagnostics, terminate the execution, or specify the desired return-value. The matherr() function has not been adopted by the C standardC99, so its use is not generally portable.
...