Stop retrying terminal subscription errors

[dlanov]

Fixes #834.

Summary

• classify gRPC subscription errors as retryable or terminal
• stop retrying terminal errors across Subscribe, SubscribeChan, and SubscribeStreamChan
• preserve retries for transient and unclassified errors for compatibility with targets that do not return canonical gRPC statuses
• distinguish locally initiated cancellation from a remotely returned CANCELED status
• add table-driven and behavioral tests for subscription retry handling

Retry behavior

The following canonical gRPC statuses stop the subscription retry loop:

• CANCELED
• INVALID_ARGUMENT
• NOT_FOUND
• ALREADY_EXISTS
• PERMISSION_DENIED
• RESOURCE_EXHAUSTED
• FAILED_PRECONDITION
• OUT_OF_RANGE
• UNIMPLEMENTED
• DATA_LOSS
• UNAUTHENTICATED

RESOURCE_EXHAUSTED is treated as terminal for subscriptions as requested in #834.

Transient statuses such as DEADLINE_EXCEEDED, ABORTED, and UNAVAILABLE remain retryable. UNKNOWN, INTERNAL, and non-gRPC errors also retain the existing retry behavior for compatibility with older targets and proxies.

Locally initiated cancellation remains silent, while a remote CANCELED status is reported once before the retry loop stops.

Testing

• go test ./target -count=1 -race
• go test ./... -count=1
• go vet ./...
• CGO_ENABLED=0 ./tests/run_tests.sh

[google-cla]

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.