What this usually means
The interceptor is registered but never reaches the execution path. Common patterns: the interceptor is passed to `grpc.NewServer` but the server is created before the interceptor is defined (order inversion). Or the interceptor is attached to an `UnaryServerInterceptor` slice but the server uses `grpc.ChainUnaryInterceptor` which expects a different signature. Or the interceptor is registered for unary but the call is streaming (or vice versa). Or the server is created with `grpc.NewServer()` and then the interceptor is set via an option after server creation — that option is ignored. In Go's gRPC, interceptors are compile-time wired: once the server is created, you can't add interceptors.
The first ten minutes — establish facts before touching code.
- 1Check the order of `grpc.NewServer` and interceptor definition — ensure interceptor variable is defined before server creation
- 2Print the server's interceptors via reflection: `reflect.ValueOf(server).Elem().FieldByName("opts").Elem().FieldByName("unaryInt")`
- 3Add a time-based panic in interceptor to see if it's invoked (e.g., `time.Sleep(1*time.Second); panic("test")`)
- 4Use `grpc.Server`'s `GetServiceInfo()` to list registered services and confirm server is serving
- 5Test with a simple unary RPC using grpcurl or a client to ensure the server is actually handling requests
The specific files, logs, configs, and dashboards that usually own this bug.
- search`main.go` or server startup file — check `grpc.NewServer` call and any options passed
- searchInterceptor registration line — ensure interceptor is passed as a `grpc.ServerOption`
- searchTest files — compare interceptor registration pattern between tests and production code
- search`go.mod` — verify gRPC version; older versions had different interceptor signatures
- searchServer logs — look for any gRPC-related warnings about interceptor registration failures
- searchProto-generated code — ensure service methods match unary vs stream expected by interceptor
Practical causes, not theory. These are the things you will actually find.
- warningInterceptor variable is defined after `grpc.NewServer` call (Go variable hoisting doesn't apply to function calls)
- warningInterceptor is passed as a nil value (e.g., `grpc.NewServer(interceptor)` where interceptor is nil)
- warningUsing `grpc.ChainUnaryInterceptor` but passing a single interceptor without wrapping in a slice
- warningStream interceptor registered but unary RPCs are used (or vice versa)
- warningServer is created with `grpc.NewServer()` and interceptor is added later via `server.AddService` or similar — that's not supported
- warningInterceptor returns `nil, nil` prematurely due to a condition that always fails
Concrete fix directions. Pick the one that matches your root cause.
- buildMove interceptor definition before `grpc.NewServer` call, or use a variable declared at package level
- buildFor multiple interceptors, use `grpc.ChainUnaryInterceptor(int1, int2)` correctly — pass as individual arguments, not a slice
- buildRegister both unary and stream interceptors if your server handles both types of RPCs
- buildIf using `grpc.NewServer(opts...)`, ensure opts slice includes the interceptor option
- buildAdd a `defer` recovery in interceptor to catch panics and log them
- buildUse `grpc.UnaryServerInterceptor` type explicitly to avoid signature mismatches
A fix you cannot prove is a guess. Close the loop.
- verifiedAdd a unique log line at the start of interceptor and verify it appears in server output on first request
- verifiedUse `grpcurl -plaintext localhost:50051 list` to verify services are registered
- verifiedSend a test RPC and check if interceptor's metadata modifications are reflected in handler
- verifiedWrite a unit test that creates a server with the interceptor and calls `server.Serve` then a client RPC
- verifiedCheck server's `GetServiceInfo()` for the service name and method list to confirm server is running
Things that make this bug worse or harder to find.
- warningDo not declare interceptor inline inside `grpc.NewServer` argument if it depends on external state that isn't ready
- warningAvoid using `grpc.UnaryServerInterceptor` as a variable name if you also import the package — name collision
- warningDo not assume `grpc.NewServer()` with no options will accept interceptors later; they are immutable post-creation
- warningDo not register interceptors on the client side when debugging server interceptors — they are separate
- warningAvoid calling `server.Serve` before interceptors are fully constructed; defer Serve until after all setup
The Silent Skip: Auth Interceptor Never Called
Timeline
- 09:00Deployed new auth interceptor to production
- 09:15Metrics show all requests passing without auth — interceptor is supposed to reject unauthenticated calls
- 09:20Checked server logs: no logs from interceptor
- 09:25Verified server startup code — interceptor variable declared after grpc.NewServer
- 09:30Moved interceptor declaration before NewServer, redeployed
- 09:35Metrics show auth failures for invalid tokens — interceptor is now executing
- 09:40Confirmed fix with grpcurl test
I had just added a JWT auth interceptor to our gRPC microservice. The interceptor was a closure that validated tokens from metadata. I wrote it, tested it locally with a unit test that created a server with the interceptor, and it worked fine. But after deployment, our metrics showed zero auth failures — every request was passing through. My first thought was that the interceptor was too permissive, but I added a panic at the top and still nothing happened. The interceptor wasn't being called at all.
I ssh'd into a pod and checked the server logs. No errors, no interceptor logs. I listed registered services with grpcurl and the server was serving. Then I looked at main.go: I had declared the interceptor variable after the call to grpc.NewServer. In Go, variables are hoisted but not initialized before function calls. So when I passed the interceptor to NewServer, it was nil, silently ignored. The server started without any interceptors.
The fix was trivial: move the interceptor variable declaration above the NewServer line. But the lesson was painful — gRPC doesn't warn you when you pass a nil interceptor. It just ignores it. Now I always define interceptors at package level or at least before any server creation. I also added a startup check that prints the number of registered interceptors to stderr.
Root cause
Interceptor variable declared after `grpc.NewServer()` call, causing `nil` to be passed as the interceptor option, which gRPC silently ignores.
The fix
Moved interceptor variable declaration before `grpc.NewServer()` call. Added a startup log printing the number of interceptors.
The lesson
Always define interceptors before creating the gRPC server, and add a sanity check that logs interceptor count at startup.
In Go's gRPC, interceptors are passed as `grpc.ServerOption`s to `grpc.NewServer`. The server stores them in an internal options struct. Once the server is created, these options are immutable — you cannot add interceptors later via any setter. This is by design for thread safety and performance.
When you call `grpc.NewServer(opt1, opt2)`, each option is processed in order. If you pass a nil interceptor (e.g., a nil variable of type `grpc.UnaryServerInterceptor`), it is silently ignored. There is no validation or error. The server will start without that interceptor.
gRPC Go distinguishes between unary and stream interceptors. `grpc.UnaryServerInterceptor` handles unary RPCs, while `grpc.StreamServerInterceptor` handles streaming RPCs. If your server uses streaming RPCs but you only register a unary interceptor, the stream interceptor won't execute. Conversely, if your service is unary-only but you register a stream interceptor, it won't be called either.
To handle both, you need to pass both options: `grpc.NewServer(grpc.UnaryInterceptor(myUnaryInt), grpc.StreamInterceptor(myStreamInt))`. Or use `grpc.ChainUnaryInterceptor` and `grpc.ChainStreamInterceptor` for multiple.
When using `grpc.ChainUnaryInterceptor`, interceptors are executed in the order they are provided. The first interceptor in the chain is the outermost (called first on request, last on response). Mixing `grpc.UnaryInterceptor` with `grpc.ChainUnaryInterceptor` in the same `NewServer` call can lead to unexpected order or one being overridden.
The `grpc.UnaryInterceptor` option sets a single interceptor, while `ChainUnaryInterceptor` appends to a chain. If you use both, the chain takes precedence? Actually, the behavior is that `UnaryInterceptor` sets the interceptor, and `ChainUnaryInterceptor` adds to it. But if you call `UnaryInterceptor` after `ChainUnaryInterceptor`, it may replace the chain. Always use one pattern consistently.
The most reliable way to test interceptors is to create a test gRPC server with the interceptor and a test service, then call it from a test client. Use `grpc.NewServer` with the interceptor, register a simple service, and serve on an in-memory listener (`bufconn`). This validates the full stack.
Avoid testing interceptors in unit tests that only check the interceptor function directly — you miss the registration step. Always include the server creation in your test to catch nil-passing bugs.
gRPC server reflection (often enabled for tools like grpcurl) does not affect interceptor execution. However, if you enable reflection, it registers a service on your server. That service's RPCs will also pass through your interceptors. This can be surprising if you think reflection is internal — it's just another service.
If your interceptor has side effects (like rate limiting), reflection requests might trigger them. Be aware of this during debugging.
Frequently asked questions
Why does my interceptor work in unit tests but not in production?
In unit tests, you likely create the server and interceptor in the same function, ensuring the interceptor is non-nil. In production, the interceptor variable might be declared after the server creation call, or its initialization depends on runtime config that hasn't loaded yet. Double-check the order of operations in your main function.
Can I add an interceptor after creating the server?
No. gRPC server options are immutable after `grpc.NewServer` returns. You must pass all interceptors at creation time. If you need to add interceptors dynamically, you'll need to recreate the server or use a wrapper pattern that delegates to a dynamic interceptor chain.
What happens if I pass a nil interceptor?
gRPC silently ignores nil interceptors. The server starts without them. No error, no warning. Always ensure your interceptor variable is initialized before passing it to `grpc.NewServer`.
How do I debug if my interceptor is registered but not called?
Add a panic at the very start of the interceptor (e.g., `panic("interceptor called")`). If the server panics on the first RPC, the interceptor is registered. If not, it's not. Alternatively, use reflection to inspect the server's internal options: `reflect.ValueOf(server).Elem().FieldByName("opts")`.
Do I need separate interceptors for unary and stream?
Yes, if your server handles both types. Unary interceptors only apply to unary RPCs, stream interceptors only to streaming RPCs. Register both using `grpc.UnaryInterceptor` and `grpc.StreamInterceptor` (or their chain counterparts) if your service uses both patterns.