diff options
Diffstat (limited to 'docs/get_started/upstream.rst')
| -rw-r--r-- | docs/get_started/upstream.rst | 462 |
1 files changed, 462 insertions, 0 deletions
diff --git a/docs/get_started/upstream.rst b/docs/get_started/upstream.rst new file mode 100644 index 000000000..136443048 --- /dev/null +++ b/docs/get_started/upstream.rst @@ -0,0 +1,462 @@ +.. _docs-get-started-upstream: + +============================================= +Get Started With Upstream Pigweed Development +============================================= +This guide will walk you through the typical upstream development workflow. + +.. note:: + + This documentation and the `sample project + <https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md>`_ + show how to use Pigweed as a library in your existing project. Using Pigweed + as the foundation for *new* projects is our intended use case, but you may + need more guidance than this documentation provides to do that right now. + + We're happy to help you get your project setup; just drop in our `chat room + <https://discord.gg/M9NSeTA>`_ or send a note to the `mailing list + <https://groups.google.com/forum/#!forum/pigweed>`_. + +Express setup +============= +If you'd like to skip the detailed explanations, below is the shorter version +of getting setup for Pigweed. If you run into trouble, look at the more +in-depth guide below, starting at :ref:`prerequisites`. The express setup +configures Pigweed's watcher for three targets to give a taste of Pigweed: + +#. **Host** - Mac, Linux, or Windows. Builds and runs tests +#. **Device/STM32F429** - Build only; Optionally, the STM32F429I-DISC1 kit to + follow along later in the guide to run tests directly on said device(s) +#. **Docs** - Builds the Pigweed docs + +To get setup: + +#. Make sure you have Git and Python installed and on your path. + +#. Clone Pigweed and bootstrap the environment (compiler setup & more). **Be + patient, this step downloads ~1GB of LLVM, GCC, and other tooling**. + + .. code-block:: bash + + $ cd ~ + $ git clone https://pigweed.googlesource.com/pigweed/pigweed + ... + $ cd pigweed + $ source ./bootstrap.sh (On Linux & Mac) + $ bootstrap.bat (On Windows) + ... + +#. Configure the GN build. + + .. code-block:: bash + + $ gn gen out + Done. Made 1047 targets from 91 files in 114ms + +#. Start the watcher. The watcher will invoke Ninja to build all the targets + + .. code-block:: bash + + $ pw watch + + ▒█████▄ █▓ ▄███▒ ▒█ ▒█ ░▓████▒ ░▓████▒ ▒▓████▄ + ▒█░ █░ ░█▒ ██▒ ▀█▒ ▒█░ █ ▒█ ▒█ ▀ ▒█ ▀ ▒█ ▀█▌ + ▒█▄▄▄█░ ░█▒ █▓░ ▄▄░ ▒█░ █ ▒█ ▒███ ▒███ ░█ █▌ + ▒█▀ ░█░ ▓█ █▓ ░█░ █ ▒█ ▒█ ▄ ▒█ ▄ ░█ ▄█▌ + ▒█ ░█░ ░▓███▀ ▒█▓▀▓█░ ░▓████▒ ░▓████▒ ▒▓████▀ + + 20200707 17:24:06 INF Starting Pigweed build watcher + 20200707 17:24:06 INF Will build [1/1]: out + 20200707 17:24:06 INF Attaching filesystem watcher to $HOME/wrk/pigweed/... + 20200707 17:24:06 INF Triggering initial build... + ... + +#. **Congratulations, you're ready to go!** Now take Pigweed for a spin by + making a test fail. + +#. With the watcher running in a separate window, edit + ``pw_status/status_test.cc`` to make an expectation fail; for example, add + ``EXPECT_EQ(0, 1);`` in a test. + +#. Save the file. Observe the watcher rebuild & retest, and fail. Restore the + test if you feel like it. + +#. Open the generated docs in ``out/docs/gen/docs/html/index.html`` in your + browser. + +#. Edit ``docs/getting_started.rst`` (this file!) and make any change. Save. + See the watcher rebuild the docs. Reload your browser, and see the changes. + +See below for equivalent Windows commands, and for more details on what each +part does. + +**Note:** After running bootstrap once, use ``source ./activate.sh`` (or +``activate.bat`` on Windows) to re-activate the environment without +re-bootstrapping. + +.. _prerequisites: + +Prerequisites +------------- +**Linux** + +Most Linux installations should work out of box, and not require any manual +installation of prerequisites beyond basics like ``git`` and +``build-essential`` (or the equivalent for your distro). + +.. inclusive-language: disable + +To flash devices using OpenOCD, you may need to extend your system udev rules +at ``/etc/udev/rules.d/``. The OpenOCD repository has a good +`example udev rules file <https://github.com/openocd-org/openocd/blob/master/contrib/60-openocd.rules>`_ +that includes many popular hardware debuggers. + +.. inclusive-language: enable + +**macOS** + +To start using Pigweed on MacOS, you'll need to install XCode. Download it +via the App Store, then install the relevant tools from the command line. + +.. code-block:: none + + $ xcode-select --install + +If you get SSL certificate errors, try installing +`Python 3.11.3 <https://www.python.org/downloads/release/python-3113/>` +and then running ``/Applications/Python\ 3.11/Install\ Certificates.command``. + +**Windows** + +To start using Pigweed on Windows, you'll need to do the following: + +* Install `Git <https://git-scm.com/download/win>`_. Git must be installed to + run from the command line and third-party software or be added to ``PATH``. + Also, ensure that the **Enable symbolic links** option is selected. +* Install `Python <https://www.python.org/downloads/windows/>`_. +* Ensure that `Developer Mode + <https://docs.microsoft.com/en-us/windows/apps/get-started/enable-your-device-for-development>`_ + is enabled. +* Enable long file paths. This can be done using ``regedit`` or by running the + following command as an administrator: + + .. code-block:: bat + + REG ADD HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem /v LongPathsEnabled /t REG_DWORD /d 1 /f + + +If you plan to flash devices with firmware, you'll need to install OpenOCD and +ensure it's on your system path. + +Bootstrap +========= + +Once you satisfied the prerequisites, you will be able to clone Pigweed and +run the bootstrap that initializes the Pigweed virtual environment. The +bootstrap may take several minutes to complete, so please be patient. + +**Linux & macOS** + +.. code-block:: bash + + $ git clone https://pigweed.googlesource.com/pigweed/pigweed ~/pigweed + $ cd ~/pigweed + $ source ./bootstrap.sh + +**Windows** + +.. code-block:: batch + + :: Run git commands from the shell you set up to use with Git during install. + > git clone https://pigweed.googlesource.com/pigweed/pigweed %HOMEPATH%\pigweed + > cd %HOMEPATH%\pigweed + > bootstrap.bat + +Below is a real-time demo with roughly what you should expect to see as output: + +.. image:: https://storage.googleapis.com/pigweed-media/pw_env_setup_demo.gif + :width: 800 + :alt: build example using pw watch + +Congratulations, you are now set up to start using Pigweed! + +.. _activate-pigweed-environment: + +Activate your Pigweed Environment +================================= +After going through the initial setup process, your current terminal will be in +the Pigweed development environment that provides all the tools you should need +to develop on Pigweed. If you leave that session, you can activate the +environment in a new session with the following command: + +**Linux & macOS** + +.. code-block:: bash + + $ source ./activate.sh + +**Windows** + +.. code-block:: batch + + > activate.bat + +Some major changes may require triggering the bootstrap again, so if you run +into host tooling changes after a pull it may be worth re-running bootstrap. + +Build Pigweed for Host +====================== +Pigweed's primary build system is GN/Ninja based. There are CMake and Bazel +builds in-development, but they are incomplete and don't have feature parity +with the GN build. We strongly recommend you stick to the GN build system. + +GN (Generate Ninja) just does what it says on the tin; GN generates +`Ninja <https://ninja-build.org/>`_ build files. + +The default GN configuration generates build files that allow you to build host +binaries, device binaries, and upstream documentation all in one Ninja +invocation. + +Run GN as seen below: + +.. code-block:: bash + + $ gn gen out + +.. note:: + ``out`` is simply the directory the build files are saved to. Unless + this directory is deleted or you desire to do a clean build, there's no need + to run GN again; just rebuild using Ninja directly. + +.. warning:: + Unless your build directory (the ``out`` in ``gn gen out``) is exactly one + directory away from the project root directory (the Pigweed repo root in this + case), there will be issues finding source files while debugging and while + generating coverage reports. This is due an issue in upstream LLVM reordering + debug and coverage path mappings (tracked by + `b/278898014 <https://issuetracker.google.com/278898014>`_ and + `b/278906020 <https://issuetracker.google.com/278906020>`_). **We recommend + sticking to simple, single directory build directories for the time being.** + +Now that we have build files, it's time to build Pigweed! + +Now you *could* manually invoke the host build using ``ninja -C out`` every +time you make a change, but that's tedious. Instead, let's use ``pw_watch``. + +Go ahead and start ``pw_watch``: + +.. code-block:: bash + + $ pw watch + +When ``pw_watch`` starts up, it will automatically build the directory we +generated in ``out``. Additionally, ``pw_watch`` watches source code files for +changes, and triggers a Ninja build whenever it notices a file has been saved. +You might be surprised how much time it can save you! + +With ``pw watch`` running, try modifying +``pw_status/public/pw_status/status.h`` and watch the build re-trigger when you +save the file. + +See below for a demo of this in action: + +.. image:: https://storage.googleapis.com/pigweed-media/pw_watch_build_demo.gif + :width: 800 + :alt: build example using pw watch + +Running Unit Tests +================== +Fun fact, you've been running the unit tests already! Ninja builds targeting +the host automatically build and run the unit tests. Unit tests err on the side +of being quiet in the success case, and only output test results when there's a +failure. + +To see a test failure, modify ``pw_status/status_test.cc`` to fail by changing +one of the strings in the "KnownString" test. + +.. image:: https://storage.googleapis.com/pigweed-media/pw_watch_test_demo.gif + :width: 800 + :alt: example test failure using pw watch + +Running tests as part of the build isn't particularly expensive because GN +caches passing tests. Each time you build, only the tests that are affected +(whether directly or transitively) by the code changes since the last build +will be re-built and re-run. + +Try running the ``pw_status`` test manually: + +.. code-block:: bash + + $ ./out/pw_strict_host_{clang,gcc}_debug/obj/pw_status/test/status_test + +Depending on your host OS, the compiler will default to either ``clang`` or +``gcc``. + +Building for a Device +===================== +A Pigweed "target" is a build configuration that includes a toolchain, default +library configurations, and more to result in binaries that run natively on the +target. With the default build invocation, you're already building for a device +target (the STMicroelectronics STM32F429I-DISC1) in parallel with the host +build! + +If you want to build JUST for the device, you can kick of watch with: + +.. code-block:: bash + + $ pw watch stm32f429i + +This is equivalent to the following Ninja invocation: + +.. code-block:: bash + + $ ninja -C out stm32f429i + +Running Tests on a Device +========================= +While tests run automatically on the host, it takes a few more steps to get +tests to run automatically on a device, too. Even though we've verified tests +pass on the host, it's crucial to verify the same with on-device testing. We've +encountered some unexpected bugs that can only be found by running the unit +tests directly on the device. + +1. Connect Device(s) +-------------------- +Connect any number of STM32F429I-DISC1 boards to your computer using the mini +USB port on the board (**not** the micro USB). Pigweed will automatically +detect the boards and distribute the tests across the devices. More boards = +faster tests! Keep in mind that you may have to make some environment specific +updates to ensure you have permissions to use the USB device. For example, on +Linux you may need to update your udev rules and ensure you're in the plugdev +and dialout groups. + +.. image:: https://storage.googleapis.com/pigweed-media/stm32f429i-disc1_connected.jpg + :width: 800 + :alt: development boards connected via USB + +2. Launch Test Server +--------------------- +To allow Ninja to run tests on an arbitrary number of devices, Ninja will send +test requests to a server running in the background. Launch the server in +another window using the command below (remember, you'll need to activate the +Pigweed environment first). + +.. code-block:: bash + + $ stm32f429i_disc1_test_server + +**Note:** If you attach or detach any more boards to your workstation you'll +need to relaunch this server. + +3. Configure GN +--------------- +Tell GN to use the testing server by enabling a build arg specific to the +stm32f429i-disc1 target. + +.. code-block:: bash + + $ gn args out + # Append this line to the file that opens in your editor to tell GN to run + # on-device unit tests. + pw_use_test_server = true + +Done! +----- +Whenever you make code changes and trigger a build, all the affected unit tests +will be run across the attached boards! + +See the demo below for an example of what this all looks like put together: + +.. image:: https://storage.googleapis.com/pigweed-media/pw_watch_on_device_demo.gif + :width: 800 + :alt: pw watch running on-device tests + +Building the Documentation +========================== +In addition to the markdown documentation, Pigweed has a collection of +information-rich RST files that are used to generate HTML documentation. All +the docs are hosted at https://pigweed.dev/, and are built as a part of the +default build invocation. This makes it easier to make changes and see how they +turn out. Once built, you can find the rendered HTML documentation at +``out/docs/gen/docs/html``. + +You can explicitly build just the documentation with the command below. + +.. code-block:: bash + + $ ninja -C out docs + +This concludes the introduction to developing for upstream Pigweed. + +Building Tests Individually +=========================== +Sometimes it's faster to incrementally build a single test target rather than +waiting for the whole world to build and all tests to run. GN has a built-in +tool, ``gn outputs``, that will translate a GN build step into a Ninja build +step. In order to build and run the right test, it's important to explicitly +specify which target to build the test under (e.g. host, SM32F529I-DISC1). +This can be done by appending the GN path to the target toolchain in parenthesis +after the desired GN build step label as seen in the example below. + +.. code-block:: none + + $ gn outputs out "//pw_status:status_test.run(//targets/host/pigweed_internal:pw_strict_host_clang_debug)" + pw_strict_host_clang_debug/obj/pw_status/status_test.run.pw_pystamp + + $ ninja -C out pw_strict_host_clang_debug/obj/pw_status/status_test.run.pw_pystamp + ninja: Entering directory `out' + [4/4] ACTION //pw_status:status_test.run(//targets/host/pigweed_internal:pw_strict_host_clang_debug) + +The ``.run`` following the test target name is a sub-target created as part of +the ``pw_test`` GN template. If you remove ``.run``, the test will build but +not attempt to run. + +In macOS and Linux, ``xargs`` can be used to turn this into a single command: + +.. code-block:: bash + + $ gn outputs out "//pw_status:status_test.run(//targets/host/pigweed_internal:pw_strict_host_clang_debug)" | xargs ninja -C out + +Next steps +========== + +Editor setup +----------------------- +Check out the :ref:`module-pw_ide` for setting up editor configurations or run +the following for a quick setup: + +.. code-block:: bash + + pw ide sync + +Check out other modules +----------------------- +If you'd like to see more of what Pigweed has to offer, dive into the +:ref:`docs-module-guides`. + +Check out the sample project +---------------------------- +We have a `sample project +<https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md>`_ +that demonstrates how to use Pigweed in your own project. Note that there are +many ways to leverage Pigweed and the sample project is one approach. + +Check out the Hackaday Supercon talk about Pigweed +-------------------------------------------------- +We gave a talk at Hackaday's 2021 supercon, `Give Pigweed a Whirl +<https://hackaday.com/2021/01/13/remoticon-video-pigweed-brings-embedded-unit-testing-library-integration-to-commandline/>`_ + +We've made improvements since we gave the talk; for example, we now have RTOS +primitives. + +Set up Pigweed for your own project +------------------------------------ +We don't yet have thorough documentation for leveraging Pigweed in a separate +project (our intended use case!). The `sample project +<https://pigweed.googlesource.com/pigweed/sample_project/+/main/README.md>`_ +shows how to use Pigweed as a library in your broader project, but you may need +further guidance. + +Dropping into our `chat room <https://discord.gg/M9NSeTA>`_ is the most +immediate way to get help. Alternatively, you can send a note to the `mailing +list <https://groups.google.com/forum/#!forum/pigweed>`_. |