Build Lineage 16 - K Robot Lab

Published: June 27, 2023

Table of contents

Setup Docker
About project structure
libnvphs
Upgrade to Android.bp
Stuck on boot

Here is the story of how I continued upgrading Android on my Nvidia Shield tablet because one day I saw a popup in the app, where I read ebooks from my public library, that soon they will drop support of Android 7.

Setup Docker

Docker setup is the same as for Lineage 15 build

I just wiped the whole /lineage/ folder. Maybe there is a better way. Something like

cd /lineage
make clean
repo init -b lineage-16.0
repo sync --force-sync --force-remove-dirty

Repo command reference. But most likely it will fail. So I just repeat from the beginning with another branch

rm -r /lineage
mkdir /lineage
cd /lineage
repo init -u https://github.com/LineageOS/android.git -b lineage-16.0 --git-lfs

Then we follow the build instructions. And very soon the breakfast shieldtablet command spits these errors.

build/make/core/product_config.mk:234: error: Can not locate config makefile for product "lineage_shieldtablet".

...

frameworks/native/build/phone-xhdpi-2048-dalvik-heap.mk:20: error: _nic.PRODUCTS.[[device/nvidia/shieldtablet/lineage_shieldtablet.mk]]: "vendor/nvidia/shieldtablet/shieldtablet-vendor.mk" does not exist.

...

** Don't have a product spec for: 'lineage_shieldtablet'
** Do you have the right repo manifest?

About project structure

Also there is no extract-files.sh script in device/nvidia/shieldtablet folder. Because of that some folders do not exist and breakfast fails. For example

makefiles Android.mk, BoardConfigVendor.mk, shieldtablet-vendor.mk in vendor/nvidia/shieldtablet
directory vendor/nvidia/shieldtablet/proprietary
etc, vendor in proprietary

They say that you need to become familiar with the repo tool. They send you to the official Getting started page. But I guarantee you, they won't tell you all shortcuts that normal people would use. So let's begin.

repo is like Git Submodules, but for some reason no one likes submodules. When you run repo sync for the first time, it dowloads many repositories and orginizes them in directories and subdirectories. This organization is conformed with the build system, which means that it composes a huge build tree for you.

This process can be more transparent and customizable. There is at least one tool that serves very similar purpose, and this is the tool that I very like - Buildroot. It has very tiny repository. It's basically a set of Makefiles and scripts which create full Linux-based operating system. All components like applications, tools, frameworks, libraries, drivers and so on, it downloads from public repositories, then compiles, then puts it together into the image of your choise like tar archive, cpio ramfs, or ISO. But because you can select a subset of tools from the whole variety of open source projects, thus it downloads only that subset. On the other side, Android seems very monolithic, it downloads everything and all this tools always in the image.

In order to review the build system I need to find the originating script build/make/core/product_config.mk and review actions from the breakfast command.

In the root directory there is a hidden folder, .repo. In the main manifest file (a thing similar to .gitmodules) search for build/make

grep build/make .repo/manifests/default.xml

And you find that you need android_build repository. Then we locate the source of our error.

And you know what? Makefiles are scary. But from my supeficial lookup I can tell that it creates a list of products by finding $(SRC_TARGET_DIR)/product/AndroidProducts.mk files. This error is safe to ignore because this is how it reilizes that it need to download android_device_nvidia_shieldtablet

Here's device specific manifest .repo/local_manifests/roomservice.xml that defines many repositories for Nvidia components

<?xml version="1.0" encoding="UTF-8"?>
<manifest>
  <project name="LineageOS/android_device_nvidia_shieldtablet" path="device/nvidia/shieldtablet" remote="github" />
  <project name="LineageOS/android_device_nvidia_icera" path="device/nvidia/icera" remote="github" />
  <project name="LineageOS/android_device_nvidia_t124-common" path="device/nvidia/t124-common" remote="github" />
  <project name="LineageOS/android_device_nvidia_touch" path="device/nvidia/touch" remote="github" />
  <project name="LineageOS/android_device_nvidia_shield-common" path="device/nvidia/shield-common" remote="github" />
  <project name="LineageOS/android_kernel_nvidia_shield" path="kernel/nvidia/shield" remote="github" />
  <project name="LineageOS/android_hardware_nvidia_interfaces" path="hardware/nvidia/interfaces" remote="github" />
  <project name="LineageOS/android_hardware_nvidia_light" path="hardware/nvidia/light" remote="github" />
  <project name="LineageOS/android_hardware_nvidia_memtrack" path="hardware/nvidia/memtrack" remote="github" />
  <project name="LineageOS/android_hardware_nvidia_power" path="hardware/nvidia/power" remote="github" />
  <project name="LineageOS/android_hardware_nvidia_thermal" path="hardware/nvidia/thermal" remote="github" />
  <project name="LineageOS/android_device_nvidia_tegra-common" path="device/nvidia/tegra-common" remote="github" />
</manifest>

I think this step with extracting proprietary files can be done better, but for now I just copy the system archive from previous build. Extract archive

tar xvf /system.tar.gz -C /system_dump/ --numeric-owner
cd device/nvidia/shieldtablet/
./extract-files.sh /system_dump/

libnvphs

And brunch time! Oh no, not again

hardware/nvidia/power/Android.mk: error: "vendor.nvidia.hardware.power@1.0-service (EXECUTABLES android-arm) missing libnvphs (SHARED_LIBRARIES android-arm)"

Ok, this library, libnvphs I can find in proprietary_vendor_nvidia repository. Which means I need to fix proprietary files manually again.

git clone -b lineage-16.0 --single-branch https://github.com/TheMuppets/proprietary_vendor_nvidia.git /proprietary_vendor_nvidia
cp -v /proprietary_vendor_nvidia/t124/nvphs/lib/libnvphs.so /lineage/vendor/nvidia/shieldtablet/proprietary/vendor/lib/

I check that makefile and trying to find where other libraries are located

find . -type f -name "libhardware.so"

It appears that they all are part of platform/prebuilts/vndk/v27 link. Of course proprietary Nvidia library will be missing.

According to this list I conclude that libnvphs was introduced in new versions of Shield tablet. So I comment out that line in the makefile that requires the library. It could fail during linkage with many undefined reference errors or, as before with files in proprietary blobs, it's just extra (or leftover) from another models.

It did fail (output split into lines by me):

[ 98% 72880/73717] target Executable: ven...vendor.nvidia.hardware.power@1.0-service)
FAILED: /lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/LINKED/vendor.nvidia.hardware.power@1.0-service

/bin/bash -c "/usr/bin/ccache prebuilts/clang/host/linux-x86/clang-4691093/bin/clang++ 
-pie -nostdlib -Bdynamic -Wl,-dynamic-linker,/system/bin/linker -Wl,--gc-sections -Wl,-z,nocopyreloc 
-Wl,-rpath-link=/lineage/out/target/product/shieldtablet/obj/lib 

/lineage/out/target/product/shieldtablet/obj/lib/crtbegin_dynamic.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/service.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/Power.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/nvpowerhal.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/timeoutpoker.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/powerhal_parser.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/powerhal_utils.o 
/lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/tegra_sata_hal.o 

-Wl,--whole-archive  -Wl,--no-whole-archive   
/lineage/out/target/product/shieldtablet/obj/STATIC_LIBRARIES/libunwind_llvm_intermediates/libunwind_llvm.a  
/lineage/out/target/product/shieldtablet/obj/STATIC_LIBRARIES/libclang_rt.ubsan_minimal-arm-android_intermediates/libclang_rt.ubsan_minimal-arm-android.a  
/lineage/out/target/product/shieldtablet/obj/STATIC_LIBRARIES/libcompiler_rt-extras_intermediates/libcompiler_rt-extras.a   
/lineage/out/target/product/shieldtablet/obj/STATIC_LIBRARIES/libatomic_intermediates/libatomic.a 
/lineage/out/target/product/shieldtablet/obj/STATIC_LIBRARIES/libgcc_intermediates/libgcc.a 

-Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -Wl,--build-id=md5 -Wl,--warn-shared-textrel 
-Wl,--fatal-warnings -Wl,--no-undefined-version -Wl,--icf=safe -Wl,--hash-style=gnu -Wl,-m,armelf -Wl,--no-fix-cortex-a8   

-target arm-linux-androideabi 
-Bprebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/arm-linux-androideabi/bin 
-Wl,--exclude-libs,libunwind_llvm.a 
-Wl,--exclude-libs,libclang_rt.ubsan_minimal-arm-android.a 
-Wl,--no-undefined 

/lineage/out/target/product/shieldtablet/obj/lib/libhardware.so 
/lineage/out/target/product/shieldtablet/obj/lib/libhidlbase.so 
/lineage/out/target/product/shieldtablet/obj/lib/libhidltransport.so 
/lineage/out/target/product/shieldtablet/obj/lib/liblog.so 
/lineage/out/target/product/shieldtablet/obj/lib/libcutils.so 
/lineage/out/target/product/shieldtablet/obj/lib/libutils.so 
/lineage/out/target/product/shieldtablet/obj/lib/libexpat.so 
/lineage/out/target/product/shieldtablet/obj/lib/vendor.nvidia.hardware.power@1.0.so 
/lineage/out/target/product/shieldtablet/obj/lib/libc++.so 
/lineage/out/target/product/shieldtablet/obj/lib/libc.so 
/lineage/out/target/product/shieldtablet/obj/lib/libm.so 
/lineage/out/target/product/shieldtablet/obj/lib/libdl.so 

-o /lineage/out/target/product/shieldtablet/obj/EXECUTABLES/vendor.nvidia.hardware.power@1.0-service_intermediates/LINKED/vendor.nvidia.hardware.power@1.0-service 
/lineage/out/target/product/shieldtablet/obj/lib/crtend_android.o"


hardware/nvidia/power/nvpowerhal.cpp:796: error: undefined reference to 'NvPHSCancelThroughputHints'
hardware/nvidia/power/nvpowerhal.cpp:793: error: undefined reference to 'NvPHSSendThroughputHints'
clang-6.0: error: linker command failed with exit code 1 (use -v to see invocation)

Function NvPHSCancelThroughputHints is called but needs a library (gitlab mirror). So where to put that library?

One easy solution is this:

#ifeq ($(TARGET_TEGRA_PHS),nvphs)
#    LOCAL_CFLAGS += -DUSE_NVPHS
#    LOCAL_SHARED_LIBRARIES += libnvphs
#endif

Yes, just comment out this part where the flag is set and the library added to the list.

Another solution is to follow this guide. Actually I found an example first in this repo

And a third option is

download libnvphs.so from this repo

wget -O /system_dump/system/vendor/lib/libnvphs.so https://github.com/BigTopKrazies/vendor_nvidia/raw/8.1/shield/common/lib/libnvphs.so

add to device/nvidia/shieldtablet/proprietary-files.txt this line

vendor/lib/libnvphs.so

I don't know what path to specify

OUT is obsolete. Use OUT_DIR instead. See https://android.googlesource.com/platform/build/+/master/Changes.md#OUT

So, I'd rather puth in the $(LOCAL_PATH)

cp /system_dump/system/vendor/lib/libnvphs.so /lineage/hardware/nvidia/power/

New error (when BUILD_SHARED_LIBRARY)

libnvphs: Unused source files: hardware/nvidia/power/libnvphs.so

Means ...

brunch is just breakfast + mka bacon - you can do it manually.

Upgrade to Android.bp

What I usually do, I search for similar use cases and try to copy how someone already did it

But strangely I don't see Android.mk files, I see is Android.bp files - this is a new format. There is a tool that can convert mk files to bp

# build a tool
make blueprint_tools
# convert to new format
cd hardware/nvidia/power/
androidmk Android.mk > Android.bp

The downside of bp format: it doesn’t have fancy string manipulation stuff like

LOCAL_CFLAGS += -DTARGET_TEGRA_VERSION=$(TARGET_TEGRA_VERSION:t=)

and not supporting conditionals.

The build script will look like this (but I didn't test it)

cc_binary {
    name: "powerhal.tegra",

    cflags: [
        "-DGPU_IS_LEGACY",
        "-DPOWER_MODE_SET_INTERACTIVE",
        "-DUSE_NVPHS",
        "-DLINEAGE_PROFILES",
    ]

    srcs: [
        "service.cpp",
        "Power.cpp",
        "nvpowerhal.cpp",
        "timeoutpoker.cpp",
        "powerhal_parser.cpp",
        "powerhal_utils.cpp",
        "tegra_sata_hal.cpp",
        // "power_floor_t210.cpp"
    ],

    shared_libs: [
        "libhardware",
        "libhidlbase",
        "libhidltransport",
        "liblog",
        "libcutils",
        "libutils",
        "libdl",
        "libexpat",
        "vendor.nvidia.hardware.power@1.0",
        "libnvphs",
        "vendor.lineage.power@1.0",
    ],

    init_rc: ["vendor.nvidia.hardware.power@1.0-service.rc"],
    intf_fragments: ["vendor.nvidia.hardware.power@1.0-service.xml"],
    relative_install_path: "hw",
    vendor: true,
    owner: "nvidia",
}

Stuck on boot

My build finished, but it stuck on boot logo. I found a forum post precisely describing my problem. No answer.

Where to find logs?

I tried adb logcat - nothing
Maybe I should build ROM again, but this time disable some security features: persist.sys.usb.config=mtp,ad ro.secure=0 ro.adb.secure=0 (link). But then the question about messing with boot.img is another headache. Unpack boot.img, and then pack it back
I tried to find previous boot log in /proc/last_kmsg from TWRP.
But in my version it probably will use pstore (link). Or should I say definitely!, because Lineage 15 uses kernel 3.10.96+, and pstore is enabled in kernel >3.4 (link)
But maybe some stuff should be mounted manually (link)