Skip to content

Odroid

This page is about hacks specific to Odroid (N2/C4) boards. Mali stuffs should go to the Mali page.

Status summary

  • Odroid N2: Amlogic S922X, 4x Cortex-A73 1.8 GHz, 2x ARM Cortex-A53 1.9 GHz, ARM Mali-G52 MP4

  • Odroid C4: Amlogic S905X3, quad core Cortex-A55 SoC, Mali G31 MP2

  • Feb 2021
  • Linux/Panfrost: can run q3. Use Armbian + 3rd party ppa.
  • Linux/Arm: X is software rendered. GL no X support (?). No Vulkan. Cannot run q3 etc. Can run OpenCL/ACL.

  • AOSP/Arm: said to support GL & Vulkan. Haven't tried.

  • Odorid XU4: Samsung Exynos5422

  • Feb 2021: Official OS image Ubuntu 20.04 18.04 Armbian Focal all tried. Cannot boot from eMMC (no HDMI output). Can boot from uSD.
  • Linux/Arm: no OpenGL support. "failed to load driver: exynos". Only GLES. Can run glbenchmark-es. Ref
  • Linux/Panfrost: untested. said to support GL.

A good page summarizing Amlogic Meson SoCs.

Use a serial port as console

NB (Felix, Jan 2021): this is not required for hacking the mali GPU driver, which can be done via ethernet connection (SSH)

Don't have to connect the power wire (RED), which makes power cycling board difficult (i.e. you have to disconnect both power cable and the UART's power wire)

Complete uart setup following the UART Interface. Open /media/boot/boot.ini and change the kernel command line to use ttyS1 rather than S0, which is default setting.

# Default Console Device Setting
setenv condev "console=ttyS0,115200n8"   # on both

# Use console and print all the log msg
setenv condev "console=ttyS1,115200n8 ignore_loglevel"   # on both

FYI - kernel parameters

OS imgs

We have several choices (ubuntu, android, and debian), but I used ubuntu.

First, download pre-built image (using 18.04.4. 20.04 does not work as intended. maybe some bugs?)

Second, 1) build your own kernel from host machine, 2) copy them to emmc, 3) reboot target device attaching it.

NOTE - ubuntu 20.04 was updated 01/26, worth trying - we may be able to use WebGL-Wayland Drivers. - (Update Jan 2021): the pre-installed OS (Ubuntu 20.04 seems to work well) - ubuntu 20.04 img (ubuntu-20.04-4.9-minimal-odroid-n2-20210202.img.xz) does not work. Package server seems down.

Outdated apt source?

Jan 2021: In the official Ubuntu image 20.04, /etc/apt/sources.list.d/odroid.list still points to 18.04 repo (bionic, http://deb.odroid.in/c4/dists/), which no longer exists. Thus an error.

Change that line to the 20.04 repo

deb http://deb.odroid.in/c4/ focal  main

Install userspace runtime

Mali user-space runtime

Note: must update apt repo file to be 20.04 LTS (focal). See above.

Check Mali OCL impl (Ubuntu 20.04. OpenCL 2.0)

ls /usr/lib/aarch64-linux-gnu/libMali.so
odroid@odroid:~/clinfo$ strings /usr/lib/aarch64-linux-gnu/libMali.so |grep OPENCL
#define __OPENCL_VERSION__ CL_VERSION_2_0

By default, you can find /usr/lib/aarch64-linux-gnu/libMali.so when you port OS image. If not, do the following things.

sudo apt install mali-fbdev

or, if you want to update it with the recent one,

apt-get download mali-fbdev
$ ar -xv mali-fbdev_*
$ tar -xvf data.tar.xz 
$ rm usr/lib/aarch64-linux-gnu/libOpenCL.so*
$ cp -r usr/* /usr/

Vulkan issue

As of Feb 2021, Vulkan seems only supported for AOSP. Not Ubuntu. Hence, projects like vkQuake may be difficult to run on Ubuntu. There existed (old) effort running vkQuake on AOSP. It's unclear though how much effort to port it to Odroid/newer AOSP. (https://github.com/SaschaWillems/vkQuake/tree/master/Android)

https://forum.odroid.com/viewtopic.php?f=176&t=36675&start=200

OpenCL installation on ODROID-N2

hardware info: 

sudo apt install clinfo
sudo clinfo --all

NB: the clinfo from Ubuntu seems to expect OPENCL 1.0. Bad. Ref

odroid@odroid:~/clinfo$ dpkg-query -l | grep -i opencl
ii  clinfo                                2.2.18.04.06-1                     arm64        Query OpenCL system information

Grab clinfo upstream and build. Good.

Sample clinfo output: Odroid N2, Mali G52 Odroid C4, Mali G31 Hikey960, Mali G71

get the dev files

sudo apt install ocl-icd-opencl-dev

actual icd (Installable Client Driver) for CPU (why do this?)

sudo apt install pocl-opencl-icd

and finally make the driver available echo "/usr/lib/aarch64-linux-gnu/libmali.so" > /etc/OpenCL/vendors/mali.icd

ref: https://blog.tribly.de/posts/controller/

Build custom kernel

Good build instruction per device is in the ODROID wiki. e.g. kernel installation is for N2 and C4

Configuration

  • Use defconfig
$ make odroidg12_defconfig

  • DEVMEM related stuffs are already in the odroid defconfig

  • Set GPU driver as out-of-tree kernel module

#
# ARM GPU Configuration
#
CONFIG_MALI_MIDGARD=m                         
# CONFIG_MALI_GATOR_SUPPORT is not set
# CONFIG_MALI_MIDGARD_DVFS is not set
# CONFIG_MALI_MIDGARD_ENABLE_TRACE is not set
# CONFIG_MALI_DEVFREQ is not set
# CONFIG_MALI_DMA_FENCE is not set
CONFIG_MALI_PLATFORM_NAME="devicetree"
# CONFIG_MALI_EXPERT is not set
# CONFIG_MALI_KUTF is not set
  • Build kernel
$ make -j$(expr $(expr $(nproc) \* 6) \/ 5)

Download a recent Mali Bifrost GPU driver

  • Does not work. I used the one in the kernel tree

Out-of-tree Mali kdriver installation

If you followed all the steps from ODROID wiki, the next step is to build Mali kdrvier.

  • The driver is automatically loaded as form of module when you boot up. Remove the original module from /lib/modules/$(uname -r)/kernel/drivers/gpu/arm/midgard/mali_kbase.ko

  • run depmod -a so that the OS won't auto load the mod in the future

  • Visit the original kernel tree and directly copy the original (old Mali kdriver) to the target device or you can directly clone our repo

orig branch: original source tree, master branch: recording added

$ git clone https://github.com/bakhi/odroid-mali-kdriver -b orig
  • Build scripts from kernel source. Note that it is possible that some header files are not founded. Simple solution is to copy them directly from the original source tree
$ cd /usr/src/linux-$(uname -r)
$ make scripts
  • Create symbolic link to the kernel source tree
$ sudo ln -s /usr/src/linux-$(uname -r) /lib/modules/$(uname -r)/build
  • (optional) Change the filepath in

(Heejin: Now some files generated by kernel module and others by debugFS, I will integrate them in the future.)

  1. Open "tgx/tgx_as.c" and change the FILE_PATH to your local one

  2. Build kdriver by

cd odroid-mali-kdriver/midgard
make -j6
sudo insmod mali_kbase.ko

Sample dmsg output

odroid@odroid:~/odroid-mali-kdriver/midgard$ sudo dmesg|tail
[  794.559095] mali_plat=ffffff8001fd3670
[  794.561886] find tzd id: 0
[  794.564439] gpu cooling register okay with err=0
[  794.569568] find tzd id: 0
[  794.572023] gpu core cooling register okay with err=0
[  794.577213] shader_present=1, tiler_present=1, l2_present=1
[  794.582993] Mali_pwr_on:gpu_irq : 200
[  794.586980] [CONFIG] pm policy: always_on
[  794.591837] mali ffe40000.bifrost: GPU identified as 0x3 arch 7.0.9 r0p0 status 0
[  794.600815] mali ffe40000.bifrost: Probed as mali0

Check the sysfs existence: 

$ ls /sys/bus/platform/drivers/mali/

MISC

# disable X11
$ sudo systemctl enable multi-user.target --force
$ sudo systemctl set-default multi-user.target

# change back to enable X11
$ sudo systemctl set-default graphical.target

Record

Reference app: vecadd https://github.com/smistad/OpenCL-Getting-Started

Run the GPU application

the kernel will dump mem contents (tgx_mem_contents) , pgt (tgx_pgt), synced address spaces (tgx_sync_as) under FILE_PATH as defined in tgx_as.c

# clean old contents. assuming  FILE_PATH=/tmp/
rm -f /tmp/tgx_mem_contents /tmp/tgx_sync_as /tmp/tgx_pgt

Post processing

Create sub-directory in midgard 

cd ~/odroid-mali-kdriver/midgard
mkdir -p trace

./send_trace.sh [tag] which dumps and writs the regIO/rbuf stuffs to the file. (Note: this script tries to copy traces over SSH. Can turn off)

./send_trace.sh my

Run python script clean_reg.py ./trace/[tag] , which transforms the regIO format. That's all.

python clean_reg.py trace/my

Finally you will have three files regarding memory (tgx_mem_contents, tgx_pgt, tgx_sync_as) and two files about trace (XXX_reg_history, XXX_rbuf_history)

Mali Kdriver differences bet r7p0 and r23p0

Some data structure are different (e.g. kbase_mmu_table does not exist in r7p0)

Some regIO are not in the r7p0 (e.g. L2_CONFIG)

Troubleshooting (Bifrost r7p0)

1. kernel build and module load/unload

  • If configuring mali GPU driver as module installation, the kernel loads module when boot up. If you try to unload module by $ sudo rmmod mali_kbase , the system get stuck and the reason is unclear.
  • Just remove mali_kbase.ko from your /lib/module/drivers/gpu/arm and do sudo depmod -a to remove mali module from the list of modules initially loaded.

2. Mali driver compilation

  • If you compile the driver with CONFIG_MALI_MIDGARD_ENALBE_TRACE and CONFIG_MALI_DEBUG, the seg fault happens that causes unmapped virtual memory access.

  • The reason is pm_callback_power_on() puts some traces before the trace buffer is allocated which is bug. To fix it, allocate buffer prior to the GPU power on. Note that this is fixed in the r23p0.

GLES

OS image: ubuntu-20.04-4.9-mate-odroid-n2-20210126.img.xz

kernel tree for custom build: https://github.com/hardkernel/linux

root@odroid:/home/odroid# glmark2-es2-fbdev 
Error: eglInitialize() failed with error: 0x3001
Error: eglInitialize() failed with error: 0x3001
Error: main: Could not initialize canvas

https://forum.odroid.com/viewtopic.php?t=34656

Kernel 5.9

http://docs.linuxfactory.or.kr/install/odroidn2/image.html

Kernel source

https://github.com/tobetter/linux/tree/odroid-5.10.y

https://github.com/chewitt/linux/tree/amlogic-5.11.y

GPU Acceleration with ARM Mali Panfrost

https://forum.odroid.com/viewtopic.php?p=312611#p312611

http://docs.linuxfactory.or.kr/guides/compile_kernel.html

Prebuilt Ubuntu img with kernel 5.10

$ sudo apt install linux-headers-$(uname -r)

Add contrib, if not in

$ cat /etc/apt/sources.list.d/ppa-linuxfactory-or-kr.list 
deb http://ppa.linuxfactory.or.kr disco main non-free contrib
deb-src http://ppa.linuxfactory.or.kr disco main non-free contrib

$ sudo apt update