📖 Guide
Protobuf & gRPC Reference
Comprehensive reference for Protocol Buffers (proto3) and gRPC — message definitions, services, streaming, and common patterns.
95 commands across 14 categories
Proto3 BasicsScalar TypesMessage DefinitionsEnumsNested MessagesOneofMapsRepeated FieldsDefault ValuesService Definition (RPC)StreamingOptions & ExtensionsWell-Known TypesgRPC Metadata & Errors
Proto3 Basics
| Command | Description |
|---|---|
syntax = "proto3";e.g. syntax = "proto3"; | Declare proto3 syntax at the top of every .proto file |
package mypackage;e.g. package com.example.api; | Define the package namespace to prevent naming conflicts between projects |
import "other.proto";e.g. import "google/protobuf/timestamp.proto"; | Import definitions from another .proto file |
import public "other.proto";e.g. import public "common/types.proto"; | Transitive import — anyone importing your file also gets this import |
option go_package = "path";e.g. option go_package = "github.com/example/api/v1"; | Set the Go package path for generated code |
option java_package = "com.example";e.g. option java_package = "com.example.api"; | Set the Java package for generated classes |
option java_multiple_files = true; | Generate a separate .java file for each top-level message/enum/service |
protoc --proto_path=SRC --xxx_out=DST file.protoe.g. protoc --proto_path=. --go_out=./gen --go-grpc_out=./gen api.proto | Compile .proto files with protoc; --xxx_out sets the target language |
Scalar Types
| Command | Description |
|---|---|
doublee.g. double latitude = 1; | 64-bit floating point (maps to float64 in Go, double in Java/C++) |
floate.g. float score = 2; | 32-bit floating point (maps to float32 in Go, float in Java/C++) |
int32e.g. int32 age = 3; | Variable-length encoding, inefficient for negative numbers — use sint32 instead |
int64e.g. int64 user_id = 4; | Variable-length 64-bit integer, inefficient for negatives — use sint64 |
uint32e.g. uint32 count = 5; | Variable-length unsigned 32-bit integer |
uint64e.g. uint64 big_count = 6; | Variable-length unsigned 64-bit integer |
sint32e.g. sint32 temperature = 7; | Uses ZigZag encoding — efficient for negative values |
sint64e.g. sint64 offset = 8; | ZigZag-encoded signed 64-bit integer |
fixed32e.g. fixed32 ip_v4 = 9; | Always 4 bytes — more efficient than uint32 if values are often > 2^28 |
fixed64e.g. fixed64 file_size = 10; | Always 8 bytes — efficient for large unsigned values |
sfixed32e.g. sfixed32 delta = 11; | Always 4 bytes, signed |
sfixed64e.g. sfixed64 big_delta = 12; | Always 8 bytes, signed |
boole.g. bool is_active = 13; | Boolean value (true/false) |
stringe.g. string name = 14; | UTF-8 encoded or 7-bit ASCII text, max 2^32 length |
bytese.g. bytes avatar = 15; | Arbitrary byte sequence, max 2^32 length |
Message Definitions
| Command | Description |
|---|---|
message Name { ... }e.g. message User {
string name = 1;
int32 age = 2;
string email = 3;
} | Define a message type — the fundamental unit in protobuf |
field_type name = field_number;e.g. string first_name = 1; | Each field has a type, name, and unique number (1-536870911, skip 19000-19999) |
reserved 2, 15, 9 to 11;e.g. reserved 2, 15, 9 to 11;
reserved "foo", "bar"; | Reserve field numbers to prevent reuse after deleting fields |
reserved "field_name";e.g. reserved "old_field", "deprecated_field"; | Reserve field names to prevent reuse |
// Field numbers 1-15 use 1 byte | Optimization: assign 1-15 to frequently-used fields (1 byte tag), 16-2047 use 2 bytes |
optional type name = N;e.g. optional string nickname = 4; | Explicitly optional field — generates has_* method to check presence |
Enums
| Command | Description |
|---|---|
enum Name { ... }e.g. enum Status {
STATUS_UNSPECIFIED = 0;
STATUS_ACTIVE = 1;
STATUS_INACTIVE = 2;
} | Define an enumeration; first value MUST be 0 (default value) |
ENUM_UNSPECIFIED = 0;e.g. ROLE_UNSPECIFIED = 0; | Convention: first enum value should be UNSPECIFIED or UNKNOWN with value 0 |
option allow_alias = true;e.g. enum Priority {
option allow_alias = true;
NORMAL = 0;
DEFAULT = 0;
HIGH = 1;
} | Allow multiple enum values to share the same number |
reserved 2, 15, 9 to 11;e.g. reserved 2;
reserved "OLD_VALUE"; | Reserve enum values to prevent reuse |
Nested Messages
| Command | Description |
|---|---|
message Outer { message Inner { ... } }e.g. message SearchResponse {
message Result {
string url = 1;
string title = 2;
}
repeated Result results = 1;
} | Nest message definitions inside other messages for logical grouping |
Outer.Innere.g. SearchResponse.Result result = 1; | Reference a nested message from outside its parent using Parent.Child syntax |
message with nested enume.g. message Order {
enum Status {
STATUS_UNSPECIFIED = 0;
STATUS_PENDING = 1;
STATUS_SHIPPED = 2;
}
Status status = 1;
} | Enums can also be nested inside messages |
Oneof
| Command | Description |
|---|---|
oneof name { ... }e.g. oneof payload {
string text = 1;
bytes binary = 2;
Image image = 3;
} | At most one field in the group will be set; setting one clears the others |
oneof cannot contain repeated/map | Limitation: repeated fields and map fields cannot be inside a oneof |
WhichOneof / has_* / casee.g. // Go: switch msg.GetPayload().(type) { ... }
// Python: msg.WhichOneof('payload') | Generated code provides methods to check which oneof field is set |
oneof backwards compatibility | You can add/remove fields in a oneof but never move existing fields into/out of it |
Maps
| Command | Description |
|---|---|
map<key_type, value_type> name = N;e.g. map<string, int32> scores = 1; | Define a map field; key can be any integral/string type (not float/bytes), value can be anything except another map |
map ordering | Map iteration order is undefined in the wire format; do not depend on ordering |
map equivalent to repeated messagee.g. // Equivalent:
message MapEntry {
string key = 1;
V value = 2;
}
repeated MapEntry items = 1; | map<string, V> is sugar for repeated message with key/value fields |
map<string, Message> name = N;e.g. map<string, User> users_by_id = 1; | Maps can contain message types as values |
Repeated Fields
| Command | Description |
|---|---|
repeated type name = N;e.g. repeated string tags = 1; | Define an ordered list (array) of values; can be empty |
repeated message fielde.g. repeated Address addresses = 2; | Lists of nested messages |
[packed = true] | In proto3, scalar numeric repeated fields use packed encoding by default (more compact wire format) |
repeated field ordering | Order is preserved in the wire format; safe to depend on list order |
Default Values
| Command | Description |
|---|---|
string → "" | Default value for string fields is the empty string |
bytes → empty bytes | Default value for bytes fields is empty byte sequence |
bool → false | Default value for bool fields is false |
numeric → 0 | Default value for all numeric types (int32, float, double, etc.) is zero |
enum → first value (0) | Default value for enums is the first defined value (must be 0) |
message → null / not set | Default value for message fields is language-dependent (null, None, nil) |
repeated → empty list | Default value for repeated fields is an empty list |
Presence detection | In proto3, scalar fields set to default values are NOT serialized; use optional or wrapper types for presence tracking |
Service Definition (RPC)
| Command | Description |
|---|---|
service Name { ... }e.g. service UserService {
rpc GetUser (GetUserRequest) returns (User);
rpc CreateUser (CreateUserRequest) returns (User);
} | Define a gRPC service with one or more RPC methods |
rpc Method (Request) returns (Response);e.g. rpc GetUser (GetUserRequest) returns (GetUserResponse); | Unary RPC — single request, single response |
rpc Method (Request) returns (Response) { ... }e.g. rpc GetUser (GetUserRequest) returns (User) {
option (google.api.http) = {
get: "/v1/users/{id}"
};
} | RPC with options defined in the body |
message Request / Response patterne.g. message GetUserRequest {
string user_id = 1;
}
message GetUserResponse {
User user = 1;
} | Convention: each RPC should have dedicated Request/Response messages (not reuse) |
Empty messagee.g. import "google/protobuf/empty.proto";
rpc Ping (google.protobuf.Empty) returns (google.protobuf.Empty); | Use google.protobuf.Empty for RPCs with no request/response data |
Streaming
| Command | Description |
|---|---|
rpc Method (stream Request) returns (Response);e.g. rpc UploadChunks (stream Chunk) returns (UploadResult); | Client streaming — client sends a stream of messages, server responds once |
rpc Method (Request) returns (stream Response);e.g. rpc ListUsers (ListUsersRequest) returns (stream User); | Server streaming — client sends one request, server responds with a stream |
rpc Method (stream Request) returns (stream Response);e.g. rpc Chat (stream ChatMessage) returns (stream ChatMessage); | Bidirectional streaming — both client and server send streams of messages |
Stream lifecycle | Streams support Send, Recv, CloseSend (half-close), and error/status on completion |
Server-side streaming example (Go)e.g. func (s *server) ListUsers(req *pb.ListUsersRequest, stream pb.UserService_ListUsersServer) error {
for _, u := range users {
if err := stream.Send(u); err != nil {
return err
}
}
return nil
} | Server sends multiple responses then returns nil to close |
Options & Extensions
| Command | Description |
|---|---|
option optimize_for = SPEED;e.g. option optimize_for = CODE_SIZE; | File-level option: SPEED (default), CODE_SIZE, or LITE_RUNTIME |
option deprecated = true;e.g. int32 old_field = 6 [deprecated = true]; | Mark a field, message, enum, or service as deprecated |
option (custom_option) = value;e.g. extend google.protobuf.FieldOptions {
optional string json_name_override = 50000;
}
string name = 1 [(json_name_override) = "userName"]; | Custom options via extensions on descriptor types (FieldOptions, MessageOptions, etc.) |
json_namee.g. string user_name = 1 [json_name = "username"]; | Override the JSON field name (default is lowerCamelCase of the field name) |
Well-Known Types
| Command | Description |
|---|---|
google.protobuf.Timestampe.g. import "google/protobuf/timestamp.proto";
google.protobuf.Timestamp created_at = 1; | Represents a point in time (seconds + nanos since Unix epoch) |
google.protobuf.Duratione.g. import "google/protobuf/duration.proto";
google.protobuf.Duration timeout = 2; | Signed fixed-length span of time (seconds + nanos) |
google.protobuf.Anye.g. import "google/protobuf/any.proto";
google.protobuf.Any details = 1;
// Pack/unpack: any.PackFrom(msg); any.UnpackTo(&msg); | Embed an arbitrary serialized message with a type URL |
google.protobuf.Structe.g. import "google/protobuf/struct.proto";
google.protobuf.Struct metadata = 1; | Represents a JSON object — useful for dynamic/schemaless data |
google.protobuf.Valuee.g. google.protobuf.Value config = 1; | Represents a dynamically-typed value (null, number, string, bool, struct, list) |
Wrapper types (StringValue, Int32Value, etc.)e.g. import "google/protobuf/wrappers.proto";
google.protobuf.StringValue nickname = 1;
google.protobuf.Int32Value score = 2; | Nullable scalar wrappers — allow distinguishing between 'not set' and 'set to default' |
google.protobuf.FieldMaske.g. import "google/protobuf/field_mask.proto";
message UpdateUserRequest {
User user = 1;
google.protobuf.FieldMask update_mask = 2;
} | Specify a subset of fields for partial updates (PATCH semantics) |
google.protobuf.Emptye.g. import "google/protobuf/empty.proto";
rpc Ping (google.protobuf.Empty) returns (google.protobuf.Empty); | Empty message — used for RPCs with no input/output |
gRPC Metadata & Errors
| Command | Description |
|---|---|
metadata.MD (Go) / Metadata (Java/Python)e.g. // Go client:
md := metadata.Pairs("authorization", "Bearer token")
ctx := metadata.NewOutgoingContext(ctx, md) | Key-value pairs sent as headers/trailers alongside RPC calls |
status.Errorf / StatusRuntimeExceptione.g. // Go:
return nil, status.Errorf(codes.NotFound, "user %s not found", id) | Return gRPC errors with a status code and message |
codes.OK (0) | Success — no error |
codes.Cancelled (1) | The operation was cancelled by the caller |
codes.InvalidArgument (3)e.g. status.Errorf(codes.InvalidArgument, "email format invalid") | Client sent an invalid argument (do NOT use for missing resource) |
codes.DeadlineExceeded (4) | The deadline expired before the operation completed |
codes.NotFound (5)e.g. status.Errorf(codes.NotFound, "order %s not found", id) | Requested entity was not found |
codes.AlreadyExists (6) | The entity already exists (e.g., duplicate create) |
codes.PermissionDenied (7) | Caller does not have permission (authenticated but unauthorized) |
codes.ResourceExhausted (8) | Resource quota exceeded (rate limiting, disk full, etc.) |
codes.Unimplemented (12) | Method not implemented by the server |
codes.Internal (13) | Internal server error — generic catch-all for server bugs |
codes.Unavailable (14) | Service unavailable — transient, client should retry with backoff |
codes.Unauthenticated (16) | Request lacks valid authentication credentials |
errdetails / Status.WithDetailse.g. // Go:
st, _ := status.New(codes.InvalidArgument, "bad request").WithDetails(
&errdetails.BadRequest{FieldViolations: [...]})
return nil, st.Err() | Attach rich error details (BadRequest, ErrorInfo, RetryInfo, etc.) to gRPC errors |
Interceptors / Middlewaree.g. // Go server:
grpc.NewServer(
grpc.UnaryInterceptor(loggingInterceptor),
grpc.StreamInterceptor(streamLoggingInterceptor),
) | Unary and stream interceptors for cross-cutting concerns (auth, logging, metrics) |
Deadlines / Timeoutse.g. // Go client:
ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
resp, err := client.GetUser(ctx, req) | Always set deadlines on RPC calls to avoid hanging forever |
📖 Free, searchable command reference. Bookmark this page for quick access.