aboutsummaryrefslogtreecommitdiff
path: root/proto/core.rst
diff options
context:
space:
mode:
Diffstat (limited to 'proto/core.rst')
-rw-r--r--proto/core.rst500
1 files changed, 500 insertions, 0 deletions
diff --git a/proto/core.rst b/proto/core.rst
new file mode 100644
index 00000000..9c765729
--- /dev/null
+++ b/proto/core.rst
@@ -0,0 +1,500 @@
+Go Protocol buffers
+===================
+
+.. _proto_library: https://docs.bazel.build/versions/master/be/protocol-buffer.html#proto_library
+.. _default Go plugin: https://github.com/golang/protobuf
+.. _common plugins: #predefined-plugins
+.. _Go providers: /go/providers.rst
+.. _GoLibrary: /go/providers.rst#golibrary
+.. _GoSource: /go/providers.rst#gosource
+.. _GoArchive: /go/providers.rst#goarchive
+.. _Gazelle: https://github.com/bazelbuild/bazel-gazelle
+.. _Make variable substitution: https://docs.bazel.build/versions/master/be/make-variables.html#make-var-substitution
+.. _Bourne shell tokenization: https://docs.bazel.build/versions/master/be/common-definitions.html#sh-tokenization
+.. _gogoprotobuf: https://github.com/gogo/protobuf
+.. _compiler.bzl: compiler.bzl
+
+.. role:: param(kbd)
+.. role:: type(emphasis)
+.. role:: value(code)
+.. |mandatory| replace:: **mandatory value**
+
+rules_go provides rules that generate Go packages from .proto files. These
+packages can be imported like regular Go libraries.
+
+.. contents:: :depth: 2
+
+-----
+
+Overview
+--------
+
+Protocol buffers are built with the three rules below. ``go_proto_library`` and
+``go_proto_compiler`` may be loaded from ``@io_bazel_rules_go//proto:def.bzl``.
+
+* `proto_library`_: This is a Bazel built-in rule. It lists a set of .proto
+ files in its ``srcs`` attribute and lists other ``proto_library`` dependencies
+ in its ``deps`` attribute. ``proto_library`` rules may be referenced by
+ language-specific code generation rules like ``java_proto_library`` and
+ ``go_proto_library``.
+* `go_proto_library`_: Generates Go code from .proto files using one or more
+ proto plugins, then builds that code into a Go library. ``go_proto_library``
+ references ``proto_library`` sources via the ``proto`` attribute. They may
+ reference other ``go_proto_library`` and ``go_library`` dependencies via the
+ ``deps`` attributes. ``go_proto_library`` rules can be depended on or
+ embedded directly by ``go_library`` and ``go_binary``.
+* `go_proto_compiler`_: Defines a protoc plugin. By default,
+ ``go_proto_library`` generates Go code with the `default Go plugin`_, but
+ other plugins can be used by setting the ``compilers`` attribute. A few
+ `common plugins`_ are provided in ``@io_bazel_rules_go//proto``.
+
+The ``go_proto_compiler`` rule produces a `GoProtoCompiler`_ provider. If you
+need a greater degree of customization (for example, if you don't want to use
+protoc), you can implement a compatible rule that returns one of these.
+
+The ``go_proto_library`` rule produces the normal set of `Go providers`_. This
+makes it compatible with other Go rules for use in ``deps`` and ``embed``
+attributes.
+
+Avoiding conflicts
+------------------
+
+When linking programs that depend on protos, care must be taken to ensure that
+the same proto isn't registered by more than one package. This may happen if
+you depend on a ``go_proto_library`` and a vendored ``go_library`` generated
+from the same .proto files. You may see compile-time, link-time, or run-time
+errors as a result of this.
+
+There are two main ways to avoid conflicts.
+
+Option 1: Use go_proto_library exclusively
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can avoid proto conflicts by using ``go_proto_library`` to generate code
+at build time and avoiding ``go_library`` rules based on pre-generated .pb.go
+files.
+
+Gazelle generates rules in this mode by default. When .proto files are present,
+it will generate ``go_proto_library`` rules and ``go_library`` rules that embed
+them (which are safe to use). Gazelle will automatically exclude .pb.go files
+that correspond to .proto files. If you have .proto files belonging to multiple
+packages in the same directory, add the following directives to your
+root build file:
+
+.. code:: bzl
+
+ # gazelle:proto package
+ # gazelle:proto_group go_package
+
+rules_go provides ``go_proto_library`` rules for commonly used proto libraries.
+The Well Known Types can be found in the ``@io_bazel_rules_go//proto/wkt``
+package. There are implicit dependencies of ``go_proto_library`` rules
+that use the default compiler, so they don't need to be written
+explicitly in ``deps``. You can also find rules for Google APIs and gRPC in
+``@go_googleapis//``. You can list these rules with the commands:
+
+.. code:: bash
+
+ $ bazel query 'kind(go_proto_library, @io_bazel_rules_go//proto/wkt:all)'
+ $ bazel query 'kind(go_proto_library, @go_googleapis//...)'
+
+Some commonly used Go libraries, such as ``github.com/golang/protobuf/ptypes``,
+depend on the Well Known Types. In order to avoid conflicts when using these
+libraries, separate versions of these libraries are provided with
+``go_proto_library`` dependencies. Gazelle resolves imports of these libraries
+automatically. For example, it will resolve ``ptypes`` as
+``@com_github_golang_protobuf//ptypes:go_default_library_gen``.
+
+Option 2: Use pre-generated .pb.go files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You can also avoid conflicts by generating .pb.go files ahead of time and using
+those exclusively instead of using ``go_proto_library``. This may be a better
+option for established Go projects that also need to build with ``go build``.
+
+Gazelle can generate rules for projects built in this mode. Add the following
+comment to your root build file:
+
+.. code:: bzl
+
+ # gazelle:proto disable_global
+
+This prevents Gazelle from generating ``go_proto_library`` rules. .pb.go files
+won't be excluded, and all special cases for imports (such as ``ptypes``) are
+disabled.
+
+If you have ``go_repository`` rules in your ``WORKSPACE`` file that may
+have protos, you'll also need to add
+``build_file_proto_mode = "disable_global"`` to those as well.
+
+.. code:: bzl
+
+ go_repository(
+ name = "com_example_some_project",
+ importpath = "example.com/some/project",
+ tag = "v0.1.2",
+ build_file_proto_mode = "disable_global",
+ )
+
+A note on vendored .proto files
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+By default, Bazel assumes imports in .proto files are relative to a repository
+root directory. This means, for example, if you import ``"foo/bar/baz.proto"``,
+that file must be in the directory ``foo/bar``, not
+``vendor/example.com/repo/foo/bar``.
+
+To deal with this, use the `strip_import_prefix` option in the proto_library_
+for the vendored file.
+
+API
+---
+
+go_proto_library
+~~~~~~~~~~~~~~~~
+
+``go_proto_library`` generates a set of .go files from a set of .proto files
+(specified in a ``proto_library`` rule), then builds a Go library from those
+files. ``go_proto_library`` can be imported like any ``go_library`` rule.
+
+Providers
+^^^^^^^^^
+
+* GoLibrary_
+* GoSource_
+* GoArchive_
+
+Attributes
+^^^^^^^^^^
+
++---------------------+----------------------+-------------------------------------------------+
+| **Name** | **Type** | **Default value** |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`name` | :type:`string` | |mandatory| |
++---------------------+----------------------+-------------------------------------------------+
+| A unique name for this rule. |
+| |
+| By convention, and in order to interoperate cleanly with Gazelle_, this |
+| should be a name like ``foo_go_proto``, where ``foo`` is the Go package name |
+| or the last component of the proto package name (hopefully the same). The |
+| ``proto_library`` referenced by ``proto`` should be named ``foo_proto``. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`proto` | :type:`label` | |mandatory| |
++---------------------+----------------------+-------------------------------------------------+
+| Points to the ``proto_library`` containing the .proto sources this rule |
+| should generate code from. Avoid using this argument, use ``protos`` instead. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`protos` | :type:`label` | |mandatory| |
++---------------------+----------------------+-------------------------------------------------+
+| List of ``proto_library`` targets containing the .proto sources this rule should generate |
+| code from. This argument should be used instead of ``proto`` argument. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`deps` | :type:`label_list` | :value:`[]` |
++---------------------+----------------------+-------------------------------------------------+
+| List of Go libraries this library depends on directly. Usually, this will be |
+| a list of ``go_proto_library`` rules that correspond to the ``deps`` of the |
+| ``proto_library`` rule referenced by ``proto``. |
+| |
+| Additional dependencies may be added by the proto compiler. For example, the |
+| default compiler implicitly adds dependencies on the ``go_proto_library`` |
+| rules for the Well Known Types. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`importpath` | :type:`string` | |mandatory| |
++---------------------+----------------------+-------------------------------------------------+
+| The source import path of this library. Other libraries can import this |
+| library using this path. This must be specified in ``go_proto_library`` or |
+| inherited from one of the targets in ``embed``. |
+| |
+| ``importpath`` must match the import path specified in ``.proto`` files using |
+| ``option go_package``. The option determines how ``.pb.go`` files generated |
+| for protos importing this proto will import this package. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`importmap` | :type:`string` | :value:`""` |
++---------------------+----------------------+-------------------------------------------------+
+| The Go package path of this library. This is mostly only visible to the |
+| compiler and linker, but it may also be seen in stack traces. This may be |
+| set to prevent a binary from linking multiple packages with the same import |
+| path, e.g., from different vendor directories. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`embed` | :type:`label_list` | :value:`[]` |
++---------------------+----------------------+-------------------------------------------------+
+| List of Go libraries that should be combined with this library. The ``srcs`` |
+| and ``deps`` from these libraries will be incorporated into this library when it |
+| is compiled. Embedded libraries must have the same ``importpath`` and |
+| Go package name. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`gc_goopts` | :type:`string_list` | :value:`[]` |
++---------------------+----------------------+-------------------------------------------------+
+| List of flags to add to the Go compilation command when using the gc |
+| compiler. Subject to `Make variable substitution`_ and `Bourne shell tokenization`_. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`compiler` | :type:`label` | :value:`None` |
++---------------------+----------------------+-------------------------------------------------+
+| Equivalent to ``compilers`` with a single label. |
++---------------------+----------------------+-------------------------------------------------+
+| :param:`compilers` | :type:`label_list` | :value:`["@io_bazel_rules_go//proto:go_proto"]` |
++---------------------+----------------------+-------------------------------------------------+
+| List of rules producing `GoProtoCompiler`_ providers (normally |
+| `go_proto_compiler`_ rules). This is usually understood to be a list of |
+| protoc plugins used to generate Go code. See `Predefined plugins`_ for |
+| some options. |
++---------------------+----------------------+-------------------------------------------------+
+
+Example: Basic proto
+^^^^^^^^^^^^^^^^^^^^
+
+Suppose you have two .proto files in separate packages: foo/foo.proto and
+bar/bar.proto. foo/foo.proto looks like this:
+
+.. code:: proto
+
+ syntax = "proto3";
+
+ option go_package = "example.com/repo/foo";
+
+ import "google/protobuf/any.proto";
+ import "bar/bar.proto";
+
+ message Foo {
+ bar.Bar x = 1;
+ google.protobuf.Any y = 2;
+ };
+
+In foo/BUILD.bazel, we need to declare a ``proto_library`` rule that lists
+foo.proto in its ``srcs`` attribute. Since we import some other protos, we
+also need a label in ``deps`` for each imported package. We will need to
+create another ``proto_library`` in bar/BUILD.bazel, but we can use an
+existing library for any.proto, since it's one of the Well Known Types.
+
+.. code:: bzl
+
+ proto_library(
+ name = "foo_proto",
+ srcs = ["foo.proto"],
+ deps = [
+ "//bar:bar_proto",
+ "@com_google_protobuf//:any_proto",
+ ],
+ visibility = ["//visibility:public"],
+ )
+
+In order to these this proto in Go, we need to declare a ``go_proto_library``
+that references to ``proto_library`` to be built via the ``proto`` attribute.
+Like ``go_library``, an ``importpath`` attribute needs to be declared.
+Ideally, this should match the ``option go_package`` declaration in the .proto
+file, but this is not required. We also need to list Go packages that the
+generated Go code imports in the ``deps`` attributes. Generally, ``deps``
+in ``go_proto_library`` will correspond with ``deps`` in ``proto_library``,
+but the Well Known Types don't need to be listed (they are added automatically
+by the compiler in use).
+
+.. code:: bzl
+
+ load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
+
+ go_proto_library(
+ name = "foo_go_proto",
+ importpath = "example.com/repo/foo",
+ proto = ":foo_proto",
+ visibility = ["//visibility:public"],
+ deps = ["//bar:bar_go_proto"],
+ )
+
+This library can be imported like a regular Go library by other rules.
+
+.. code:: bzl
+
+ load("@io_bazel_rules_go//go:def.bzl", "go_binary")
+
+ go_binary(
+ name = "main",
+ srcs = ["main.go"],
+ deps = ["//foo:foo_go_proto"],
+ )
+
+If you need to add additional source files to a package built from protos,
+you can do so with a separate ``go_library`` that embeds the
+``go_proto_library``.
+
+.. code:: bzl
+
+ load("@io_bazel_rules_go//go:def.bzl", "go_library")
+
+ go_library(
+ name = "foo",
+ srcs = ["extra.go"],
+ embed = [":foo_go_proto"],
+ importpath = "example.com/repo/foo",
+ visibility = ["//visibility:public"],
+ )
+
+For convenience, ``proto_library``, ``go_proto_library``, and ``go_binary``
+can all be generated by Gazelle_.
+
+Example: gRPC
+^^^^^^^^^^^^^
+
+To compile protos that contain service definitions, just use the ``go_grpc``
+plugin.
+
+.. code:: bzl
+
+ load("@io_bazel_rules_go//proto:def.bzl", "go_proto_library")
+
+ proto_library(
+ name = "foo_proto",
+ srcs = ["foo.proto"],
+ visibility = ["//visibility:public"],
+ )
+
+ go_proto_library(
+ name = "foo_go_proto",
+ compilers = ["@io_bazel_rules_go//proto:go_grpc"],
+ importpath = "example.com/repo/foo",
+ proto = ":foo_proto",
+ visibility = ["//visibility:public"],
+ deps = ["//bar:bar_go_proto"],
+ )
+
+go_proto_compiler
+~~~~~~~~~~~~~~~~~
+
+``go_proto_compiler`` describes a plugin for protoc, the proto compiler.
+Different plugins will generate different Go code from the same protos.
+Compilers may be chosen through the ``compilers`` attribute of
+``go_proto_library``.
+
+Several instances of this rule are listed in `Predefined plugins`_. You will
+only need to use this rule directly if you need a plugin which is not there.
+
+Providers
+^^^^^^^^^
+
+* GoProtoCompiler_
+* GoLibrary_
+* GoSource_
+
+Attributes
+^^^^^^^^^^
+
++-----------------------------+----------------------+-----------------------------------------------------+
+| **Name** | **Type** | **Default value** |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`name` | :type:`string` | |mandatory| |
++-----------------------------+----------------------+-----------------------------------------------------+
+| A unique name for this rule. |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`deps` | :type:`label_list` | :value:`[]` |
++-----------------------------+----------------------+-----------------------------------------------------+
+| List of Go libraries that Go code *generated by* this compiler depends on |
+| implicitly. Rules in this list must produce the `GoLibrary`_ provider. This |
+| should contain libraries for the Well Known Types at least. |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`options` | :type:`string_list` | :value:`[]` |
++-----------------------------+----------------------+-----------------------------------------------------+
+| List of command line options to be passed to the compiler. Each option will |
+| be preceded by ``--option``. |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`suffix` | :type:`string` | :value:`.pb.go` |
++-----------------------------+----------------------+-----------------------------------------------------+
+| File name suffix of generated Go files. ``go_proto_compiler`` assumes that |
+| one Go file will be generated for each input .proto file. Output file names |
+| will have the .proto suffix removed and this suffix appended. For example, |
+| ``foo.proto`` will become ``foo.pb.go``. |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`valid_archive` | :type:`bool` | :value:`True` |
++-----------------------------+----------------------+-----------------------------------------------------+
+| Whether code generated by this compiler can be compiled into a standalone |
+| archive file without additional sources. |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`import_path_option` | :type:`bool` | :value:`True` |
++-----------------------------+----------------------+-----------------------------------------------------+
+| When true, the ``importpath`` attribute from ``go_proto_library`` rules |
+| using this compiler will be passed to the compiler on the command line as |
+| ``--option import_path={}``. |
++-----------------------------+----------------------+-----------------------------------------------------+
+| :param:`plugin` | :type:`label` | :value:`@com_github_golang_protobuf//protoc-gen-go` |
++-----------------------------+----------------------+-----------------------------------------------------+
+| The plugin to use with protoc via the ``--plugin`` option. This rule must |
+| produce an executable file. |
++-----------------------------+----------------------+-----------------------------------------------------+
+
+Predefined plugins
+------------------
+
+Several ``go_proto_compiler`` rules are predefined in
+``@io_bazel_rules_go//proto``.
+
+* ``go_proto``: default plugin from github.com/golang/protobuf.
+* ``go_grpc``: default gRPC plugin.
+* ``go_proto_validate``: validator plugin from
+ github.com/mwitkow/go-proto-validators. Generates ``Validate`` methods.
+* gogoprotobuf_ plugins for the variants ``combo``, ``gofast``, ``gogo``,
+ ``gogofast``, ``gogofaster``, ``gogoslick``, ``gogotypes``, ``gostring``.
+ For each variant, there is a regular version (e.g., ``gogo_proto``) and a
+ gRPC version (e.g., ``gogo_grpc``).
+
+Providers
+---------
+
+Providers are objects produced by Bazel rules and consumed by other rules that
+depend on them. See `Go providers`_ for information about Go providers,
+specifically GoLibrary_, GoSource_, and GoArchive_.
+
+GoProtoCompiler
+~~~~~~~~~~~~~~~
+
+GoProtoCompiler is the provider returned by the ``go_proto_compiler`` rule and
+anything compatible with it. The ``go_proto_library`` rule expects any rule
+listed in its ``compilers`` attribute to provide ``GoProtoCompiler``. If the
+``go_proto_compiler`` rule doesn't do what you need (e.g., you don't want to
+use protoc), you can write a new rule that produces this.
+
+``GoProtoCompiler`` is loaded from ``@io_bazel_rules_go//proto:def.bzl``.
+
+``GoProtoCompiler`` has the fields described below. Additional fields may be
+added to pass information to the ``compile`` function. This interface is
+*not final* and may change in the future.
+
++-----------------------------+-------------------------------------------------+
+| **Name** | **Type** |
++-----------------------------+-------------------------------------------------+
+| :param:`deps` | :type:`Target list` |
++-----------------------------+-------------------------------------------------+
+| A list of Go libraries to be added as dependencies to any |
+| ``go_proto_library`` compiled with this compiler. Each target must provide |
+| GoLibrary_, GoSource_, and GoArchive_. This list should include libraries |
+| for the Well Known Types and anything else considered "standard". |
++-----------------------------+-------------------------------------------------+
+| :param:`compile` | :type:`Function` |
++-----------------------------+-------------------------------------------------+
+| A function which declares output files and actions when called. See |
+| `compiler.bzl`_ for details. |
++-----------------------------+-------------------------------------------------+
+| :param:`valid_archive` | :type:`bool` |
++-----------------------------+-------------------------------------------------+
+| Whether the compiler produces a complete Go library. Compilers that just add |
+| methods to structs produced by other compilers will set this to false. |
++-----------------------------+-------------------------------------------------+
+
+Dependencies
+------------
+
+In order to support protocol buffers, rules_go declares the external
+repositories listed below in ``go_rules_dependencies()``. These repositories
+will only be downloaded if proto rules are used.
+
+* ``@com_google_protobuf (github.com/google/protobuf)``: Well Known Types and
+ general proto support.
+* ``@com_github_golang_protobuf (github.com/golang/protobuf)``: standard
+ Go proto plugin.
+* ``@com_github_mwitkow_go_proto_validators
+ (github.com/mwitkow/go-proto-validators)``: validator plugin.
+* ``@com_github_gogo_protobuf (github.com/gogo/protobuf)``: gogoprotobuf
+ plugins.
+* ``@org_golang_google_grpc (github.com/grpc/grpc-go``: gRPC support.
+* gRPC dependencies
+
+ * ``@org_golang_x_net (golang.org/x/net)``
+ * ``@org_golang_x_text (golang.org/x/text)``
+ * ``@org_golang_google_genproto (google.golang.org/genproto)``
generated by cgit v1.2.3 (git 2.25.1) at 2024-05-27 14:32:46 +0000