1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
|
<html devsite>
<head>
<title>A/B (Seamless) System Updates</title>
<meta name="project_path" value="/_project.yaml" />
<meta name="book_path" value="/_book.yaml" />
</head>
<body>
<!--
Copyright 2018 The Android Open Source Project
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<p>A/B system updates, also known as seamless updates, ensure a workable
booting system remains on the disk during an <a href="/devices/tech/ota/index.html">
over-the-air (OTA) update</a>. This approach reduces the likelihood of
an inactive device after an update, which means fewer device
replacements and device reflashes at repair and warranty centers. Other
commercial-grade operating systems such as
<a href="https://www.chromium.org/chromium-os">ChromeOS</a> also use A/B
updates successfully.
</p>
<p>A/B system updates provide the following benefits:</p>
<ul>
<li>
OTA updates can occur while the system is running, without
interrupting the user. Users can continue to use their devices during
an OTA—the only downtime during an update is when the device
reboots into the updated disk partition.
</li>
<li>
After an update, rebooting takes no longer than a regular reboot.
</li>
<li>
If an OTA fails to apply (for example, because of a bad flash), the
user will not be affected. The user will continue to run the old OS,
and the client is free to re-attempt the update.
</li>
<li>
If an OTA update is applied but fails to boot, the device will reboot
back into the old partition and remains usable. The client is free to
re-attempt the update.
</li>
<li>
Any errors (such as I/O errors) affect only the <strong>unused</strong>
partition set and can be retried. Such errors also become less likely
because the I/O load is deliberately low to avoid degrading the user
experience.
</li>
<li>
Updates can be streamed to A/B devices, removing the need to download
the package before installing it. Streaming means it's not necessary
for the user to have enough free space to store the update package on
<code>/data</code> or <code>/cache</code>.
</li>
<li>
The cache partition is no longer used to store OTA update packages, so
there is no need to ensure that the cache partition is large enough for
future updates.
</li>
<li>
<a href="/security/verifiedboot/dm-verity.html">dm-verity</a>
guarantees a device will boot an uncorrupted image. If a device
doesn't boot due to a bad OTA or dm-verity issue, the device can
reboot into an old image. (Android <a href="/security/verifiedboot/">
Verified Boot</a> does not require A/B updates.)
</li>
</ul>
<h2 id="overview">About A/B system updates</h2>
<p>
A/B updates require changes to both the client and the system. The OTA
package server, however, should not require changes: update packages
are still served over HTTPS. For devices using Google's OTA
infrastructure, the system changes are all in AOSP, and the client code
is provided by Google Play services. OEMs not using Google's OTA
infrastructure will be able to reuse the AOSP system code but will
need to supply their own client.
</p>
<p>
For OEMs supplying their own client, the client needs to:
</p>
<ul>
<li>
Decide when to take an update. Because A/B updates happen in the
background, they are no longer user-initiated. To avoid disrupting
users, it is recommended that updates are scheduled when the device
is in idle maintenance mode, such as overnight, and on Wi-Fi.
However, your client can use any heuristics you want.
</li>
<li>
Check in with your OTA package servers and determine whether an
update is available. This should be mostly the same as your existing
client code, except that you will want to signal that the device
supports A/B. (Google's client also includes a
<strong>Check now</strong> button for users to check for the latest
update.)
</li>
<li>
Call <code>update_engine</code> with the HTTPS URL for your update
package, assuming one is available. <code>update_engine</code> will
update the raw blocks on the currently unused partition as it streams
the update package.
</li>
<li>
Report installation successes or failures to your servers, based on
the <code>update_engine</code> result code. If the update is applied
successfully, <code>update_engine</code> will tell the bootloader to
boot into the new OS on the next reboot. The bootloader will fallback
to the old OS if the new OS fails to boot, so no work is required
from the client. If the update fails, the client needs to decide when
(and whether) to try again, based on the detailed error code. For
example, a good client could recognize that a partial ("diff") OTA
package fails and try a full OTA package instead.
</li>
</ul>
<p>Optionally, the client can:</p>
<ul>
<li>
Show a notification asking the user to reboot. If you want to
implement a policy where the user is encouraged to routinely update,
then this notification can be added to your client. If the client
does not prompt users, then users will get the update next time they
reboot anyway. (Google's client has a per-update configurable delay.)
</li>
<li>
Show a notification telling users whether they booted into a new
OS version or whether they were expected to do so but fell back to
the old OS version. (Google's client typically does neither.)
</li>
</ul>
<p>On the system side, A/B system updates affect the following:</p>
<ul>
<li>
Partition selection (slots), the <code>update_engine</code> daemon,
and bootloader interactions (described below)
</li>
<li>
Build process and OTA update package generation (described in
<a href="/devices/tech/ota/ab/ab_implement.html">Implementing A/B
Updates</a>)
</li>
</ul>
<aside class="note">
<strong>Note:</strong> A/B system updates implemented through OTA are
recommended for new devices only.
</aside>
<h3 id="slots">Partition selection (slots)</h3>
<p>
A/B system updates use two sets of partitions referred to as
<em>slots</em> (normally slot A and slot B). The system runs from
the <em>current</em> slot while the partitions in the <em>unused</em>
slot are not accessed by the running system during normal operation.
This approach makes updates fault resistant by keeping the unused
slot as a fallback: If an error occurs during or immediately after
an update, the system can rollback to the old slot and continue to
have a working system. To achieve this goal, no partition used by
the <em>current</em> slot should be updated as part of the OTA
update (including partitions for which there is only one copy).
</p>
<p>
Each slot has a <em>bootable</em> attribute that states whether the
slot contains a correct system from which the device can boot. The
current slot is bootable when the system is running, but the other
slot may have an old (still correct) version of the system, a newer
version, or invalid data. Regardless of what the <em>current</em>
slot is, there is one slot that is the <em>active</em> slot (the one
the bootloader will boot form on the next boot) or the
<em>preferred</em> slot.
</p>
<p>
Each slot also has a <em>successful</em> attribute set by the user
space, which is relevant only if the slot is also bootable. A
successful slot should be able to boot, run, and update itself. A
bootable slot that was not marked as successful (after several
attempts were made to boot from it) should be marked as unbootable
by the bootloader, including changing the active slot to another
bootable slot (normally to the slot running immediately before the
attempt to boot into the new, active one). The specific details of
the interface are defined in
<code><a href="https://android.googlesource.com/platform/hardware/libhardware/+/master/include/hardware/boot_control.h" class="external-link">
boot_control.h</a></code>.
</p>
<h3 id="update-engine">Update engine daemon</h3>
<p>
A/B system updates use a background daemon called
<code>update_engine</code> to prepare the system to boot into a new,
updated version. This daemon can perform the following actions:
</p>
<ul>
<li>
Read from the current slot A/B partitions and write any data to
the unused slot A/B partitions as instructed by the OTA package.
</li>
<li>
Call the <code>boot_control</code> interface in a pre-defined
workflow.
</li>
<li>
Run a <em>post-install</em> program from the <em>new</em>
partition after writing all the unused slot partitions, as
instructed by the OTA package. (For details, see
<a href="#post-installation">Post-installation</a>).
</li>
</ul>
<p>
As the <code>update_engine</code> daemon is not involved in the boot
process itself, it is limited in what it can do during an update by
the <a href="/security/selinux/">SELinux</a> policies and features
in the <em>current</em> slot (such policies and features can't be
updated until the system boots into a new version). To maintain a
robust system, the update process <strong>should not</strong> modify
the partition table, the contents of partitions in the current slot,
or the contents of non-A/B partitions that can't be wiped with a
factory reset.
</p>
<h4 id="update_engine_source">Update engine source</h4>
<p>
The <code>update_engine</code> source is located in
<code><a href="https://android.googlesource.com/platform/system/update_engine/" class="external">system/update_engine</a></code>.
The A/B OTA dexopt files are split between <code>installd</code> and
a package manager:
</p>
<ul>
<li>
<code><a href="https://android.googlesource.com/platform/frameworks/native/+/master/cmds/installd/" class="external-link">frameworks/native/cmds/installd/</a></code>ota*
includes the postinstall script, the binary for chroot, the
installd clone that calls dex2oat, the post-OTA move-artifacts
script, and the rc file for the move script.
</li>
<li>
<code><a href="https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/OtaDexoptService.java" class="external-link">frameworks/base/services/core/java/com/android/server/pm/OtaDexoptService.java</a></code>
(plus <code><a href="https://android.googlesource.com/platform/frameworks/base/+/master/services/core/java/com/android/server/pm/OtaDexoptShellCommand.java" class="external-link">OtaDexoptShellCommand</a></code>)
is the package manager that prepares dex2oat commands for
applications.
</li>
</ul>
<p>
For a working example, refer to <code><a href="https://android.googlesource.com/device/google/marlin/+/nougat-dr1-release/device-common.mk" class="external-link">/device/google/marlin/device-common.mk</a></code>.
</p>
<h4 id="update_engine_logs">Update engine logs</h4>
<p>
For Android 8.x releases and earlier, the <code>update_engine</code>
logs can be found in <code>logcat</code> and in the bug report. To
make the <code>update_engine</code> logs available in the file system,
patch the following changes into your build:
</p>
<ul>
<li><a
href="https://android-review.googlesource.com/c/platform/system/update_engine/+/486618">
Change 486618</a></li>
<li><a
href="https://android-review.googlesource.com/c/platform/system/core/+/529080">
Change 529080</a></li>
<li><a
href="https://android-review.googlesource.com/c/platform/system/update_engine/+/529081">
Change 529081</a></li>
<li><a
href="https://android-review.googlesource.com/c/platform/system/sepolicy/+/534660">
Change 534660</a></li>
<li><a
href="https://android-review.googlesource.com/c/platform/system/update_engine/+/594637">
Change 594637</a></li>
</ul>
<p>These changes save a copy of the most recent
<code>update_engine</code> log to
<code>/data/misc/update_engine_log/update_engine.<var>YEAR</var>-<var>TIME</var></code>.
In addition to the current log, the five most recent logs are saved
under <code>/data/misc/update_engine_log/</code>. Users
with the <strong>log</strong> group ID will be able to access the
file system logs.</p>
<h3 id="bootloader-interactions">Bootloader interactions</h3>
<p>
The <code>boot_control</code> HAL is used by
<code>update_engine</code> (and possibly other daemons) to instruct
the bootloader what to boot from. Common example scenarios and their
associated states include the following:
</p>
<ul>
<li>
<strong>Normal case</strong>: The system is running from its
current slot, either slot A or B. No updates have been applied so
far. The system's current slot is bootable, successful, and the
active slot.
</li>
<li>
<strong>Update in progress</strong>: The system is running from
slot B, so slot B is the bootable, successful, and active slot.
Slot A was marked as unbootable since the contents of slot A are
being updated but not yet completed. A reboot in this state should
continue booting from slot B.
</li>
<li>
<strong>Update applied, reboot pending</strong>: The system is
running from slot B, slot B is bootable and successful, but slot A
was marked as active (and therefore is marked as bootable). Slot A
is not yet marked as successful and some number of attempts to
boot from slot A should be made by the bootloader.
</li>
<li>
<strong>System rebooted into new update</strong>: The system is
running from slot A for the first time, slot B is still bootable
and successful while slot A is only bootable, and still active but
not successful. A user space daemon, <code>update_verifier</code>,
should mark slot A as successful after some checks are made.
</li>
</ul>
<h3 id="streaming-updates">Streaming update support</h3>
<p>
User devices don't always have enough space on <code>/data</code> to
download the update package. As neither OEMs nor users want to waste
space on a <code>/cache</code> partition, some users go without
updates because the device has nowhere to store the update package.
To address this issue, Android 8.0 added support for streaming A/B
updates that write blocks directly to the B partition as they are
downloaded, without having to store the blocks on <code>/data</code>.
Streaming A/B updates need almost no temporary storage and require
just enough storage for roughly 100 KiB of metadata.
</p>
<p>To enable streaming updates in Android 7.1, cherrypick the following
patches:</p>
<ul>
<li>
<a href="https://android-review.googlesource.com/333624" class="external">
Allow to cancel a proxy resolution request</a>
</li>
<li>
<a href="https://android-review.googlesource.com/333625" class="external">
Fix terminating a transfer while resolving proxies</a>
</li>
<li>
<a href="https://android-review.googlesource.com/333626" class="external">
Add unit test for TerminateTransfer between ranges</a>
</li>
<li>
<a href="https://android-review.googlesource.com/333627" class="external">
Cleanup the RetryTimeoutCallback()</a>
</li>
</ul>
<p>
These patches are required to support streaming A/B updates in
Android 7.1 and later whether using
<a href="https://www.android.com/gms/">Google Mobile Services
(GMS)</a> or any other update client.
</p>
<h2 id="life-of-an-a-b-update">Life of an A/B update</h2>
<p>
The update process starts when an OTA package (referred to in code as a
<em>payload</em>) is available for downloading. Policies in the device
may defer the payload download and application based on battery level,
user activity, charging status, or other policies. In addition,
because the update runs in the background, users might not know an
update is in progress. All of this means the update process might be
interrupted at any point due to policies, unexpected reboots, or user
actions.
</p>
<p>
Optionally, metadata in the OTA package itself indicates the update
can be streamed; the same package can also be used for non-streaming
installation. The server may use the metadata to tell the client it's
streaming so the client will hand off the OTA to
<code>update_engine</code> correctly. Device manufacturers with their
own server and client can enable streaming updates by ensuring the
server identifies the update is streaming (or assumes all updates are
streaming) and the client makes the correct call to
<code>update_engine</code> for streaming. Manufacturers can use the
fact that the package is of the streaming variant to send a flag to
the client to trigger hand off to the framework side as streaming.
</p>
<p>After a payload is available, the update process is as follows:</p>
<table>
<tr>
<th>Step</th>
<th>Activities</th>
</tr>
<tr>
<td>1</td>
<td>The current slot (or "source slot") is marked as successful (if
not already marked) with <code>markBootSuccessful()</code>.</td>
</tr>
<tr>
<td>2</td>
<td>
The unused slot (or "target slot") is marked as unbootable by
calling the function <code>setSlotAsUnbootable()</code>. The
current slot is always marked as successful at the beginning of
the update to prevent the bootloader from falling back to the
unused slot, which will soon have invalid data. If the system has
reached the point where it can start applying an update, the
current slot is marked as successful even if other major
components are broken (such as the UI in a crash loop) as it is
possible to push new software to fix these problems.
<br /><br />
The update payload is an opaque blob with the instructions to
update to the new version. The update payload consists of the
following:
<ul>
<li>
<em>Metadata</em>. A relatively small portion of the update
payload, the metadata contains a list of operations to produce
and verify the new version on the target slot. For example, an
operation could decompress a certain blob and write it to
specific blocks in a target partition, or read from a source
partition, apply a binary patch, and write to certain blocks
in a target partition.
</li>
<li>
<em>Extra data</em>. As the bulk of the update payload, the
extra data associated with the operations consists of the
compressed blob or binary patch in these examples.
</li>
</ul>
</td>
</tr>
<tr>
<td>3</td>
<td>The payload metadata is downloaded.</td>
</tr>
<tr>
<td>4</td>
<td>
For each operation defined in the metadata, in order, the
associated data (if any) is downloaded to memory, the operation is
applied, and the associated memory is discarded.
</td>
</tr>
<tr>
<td>5</td>
<td>
The whole partitions are re-read and verified against the expected
hash.
</td>
</tr>
<tr>
<td>6</td>
<td>
The post-install step (if any) is run. In the case of an error
during the execution of any step, the update fails and is
re-attempted with possibly a different payload. If all the steps
so far have succeeded, the update succeeds and the last step is
executed.
</td>
</tr>
<tr>
<td>7</td>
<td>
The <em>unused slot</em> is marked as active by calling
<code>setActiveBootSlot()</code>. Marking the unused slot as
active doesn't mean it will finish booting. The bootloader (or
system itself) can switch the active slot back if it doesn't read
a successful state.
</td>
</tr>
<tr>
<td>8</td>
<td>
Post-installation (described below) involves running a program
from the "new update" version while still running in the old
version. If defined in the OTA package, this step is
<strong>mandatory</strong> and the program must return with exit
code <code>0</code>; otherwise, the update fails.
</td>
</tr>
<td>9</td>
<td>
After the system successfully boots far enough into the new slot
and finishes the post-reboot checks, the now current slot
(formerly the "target slot") is marked as successful by calling
<code>markBootSuccessful()</code>.
</td>
<tr>
</table>
<aside class="note">
<strong>Note:</strong> Steps 3 and 4 take most of the update time as
they involve writing and downloading large amounts of data, and are
likely to be interrupted for reasons of policy or reboot.
</aside>
<h3 id="post-installation">Post-installation</h3>
<p>
For every partition where a post-install step is defined,
<code>update_engine</code> mounts the new partition into a specific
location and executes the program specified in the OTA relative to
the mounted partition. For example, if the post-install program is
defined as <code>usr/bin/postinstall</code> in the system partition,
this partition from the unused slot will be mounted in a fixed
location (such as <code>/postinstall_mount</code>) and the
<code>/postinstall_mount/usr/bin/postinstall</code> command is
executed.
</p>
<p>
For post-installation to succeed, the old kernel must be able to:
</p>
<ul>
<li>
<strong>Mount the new filesystem format</strong>. The filesystem
type cannot change unless there's support for it in the old
kernel, including details such as the compression algorithm used
if using a compressed filesystem (i.e. SquashFS).
</li>
<li>
<strong>Understand the new partition's post-install program format</strong>.
If using an Executable and Linkable Format (ELF) binary, it should
be compatible with the old kernel (e.g. a 64-bit new program
running on an old 32-bit kernel if the architecture switched from
32- to 64-bit builds). Unless the loader (<code>ld</code>) is
instructed to use other paths or build a static binary, libraries
will be loaded from the old system image and not the new one.
</li>
</ul>
<p>
For example, you could use a shell script as a post-install program
interpreted by the old system's shell binary with a <code>#!</code>
marker at the top), then set up library paths from the new
environment for executing a more complex binary post-install
program. Alternatively, you could run the post-install step from a
dedicated smaller partition to enable the filesystem format in the
main system partition to be updated without incurring backward
compatibility issues or stepping-stone updates; this would allow
users to update directly to the latest version from a factory image.
</p>
<p>
The new post-install program is limited by the SELinux policies
defined in the old system. As such, the post-install step is
suitable for performing tasks required by design on a given device
or other best-effort tasks (i.e. updating the A/B-capable firmware
or bootloader, preparing copies of databases for the new version,
etc.). The post-install step is <strong>not suitable</strong> for
one-off bug fixes before reboot that require unforeseen permissions.
</p>
<p>
The selected post-install program runs in the
<code>postinstall</code> SELinux context. All the files in the new
mounted partition will be tagged with <code>postinstall_file</code>,
regardless of what their attributes are after rebooting into that
new system. Changes to the SELinux attributes in the new system
won't impact the post-install step. If the post-install program
needs extra permissions, those must be added to the post-install
context.
</p>
<h3 id="after_reboot">After reboot</h3>
<p>
After rebooting, <code>update_verifier</code> triggers the integrity
check using dm-verity. This check starts before zygote to avoid Java
services making any irreversible changes that would prevent a safe
rollback. During this process, bootloader and kernel may also
trigger a reboot if verified boot or dm-verity detect any
corruption. After the check completes, <code>update_verifier</code>
marks the boot successful.
</p>
<p>
<code>update_verifier</code> will read only the blocks listed in
<code>/data/ota_package/care_map.txt</code>, which is included in an
A/B OTA package when using the AOSP code. The Java system update
client, such as GmsCore, extracts <code>care_map.txt</code>, sets up
the access permission before rebooting the device, and deletes the
extracted file after the system successfully boots into the new
version.
</p>
</body>
</html>
|
