OP-TEE

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca _start() Run a lottery to decide the primary hart. Use amoadd.w to decide which core is the primary hart. For primary hart: reset_primary() For secondary hart: reset_secondary() reset_primary() Zero .bss section. set_tp() Set $tp to thread_core_local[hartid]. Save current hart ID to thread_core_local[hartid].hart_id. thread_init_thread_core_local() Set thread_core_local.curr_thread to THREAD_ID_INVALID for all cores (CFG_TEE_CORE_NB_CORE). Set thread_core_local.flag to THREAD_CLF_TMP to indicate that it’s using the temporary stack for all cores (CFG_TEE_CORE_NB_CORE). Set first core’s thread_core_local[0].tmp_stack_va_end to stack_tmp[0]. plat_primary_init_early() ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 // core/arch/riscv/include/mm/generic_ram_layout.h /* * Generic RAM layout configuration directives * * Mandatory directives: * CFG_TDDRAM_START * CFG_TDDRAM_SIZE * CFG_SHMEM_START * CFG_SHMEM_SIZE * * Optional directives: * CFG_TEE_LOAD_ADDR If defined sets TEE_LOAD_ADDR. If not, TEE_LOAD_ADDR * is set by the platform or defaults to TEE_RAM_START. * CFG_TEE_RAM_VA_SIZE Some platforms may have specific needs * * Optional directives when pager is enabled: * CFG_TDSRAM_START If no set, emulated at CFG_TDDRAM_START * CFG_TDSRAM_SIZE Default to CFG_CORE_TDSRAM_EMUL_SIZE * * Optional directive when CFG_SECURE_DATA_PATH is enabled: * CFG_TEE_SDP_MEM_SIZE If CFG_TEE_SDP_MEM_BASE is not defined, SDP test * memory byte size can be set by CFG_TEE_SDP_MEM_SIZE. * * This header file produces the following generic macros upon the mandatory * and optional configuration directives listed above: * * TEE_RAM_START TEE core RAM physical base address * TEE_RAM_VA_SIZE TEE core virtual memory address range size * TEE_RAM_PH_SIZE TEE core physical RAM byte size * TA_RAM_START TA contexts/pagestore RAM physical base address * TA_RAM_SIZE TA contexts/pagestore RAM byte size * TEE_SHMEM_START Non-secure static shared memory physical base address * TEE_SHMEM_SIZE Non-secure static shared memory byte size * * TDDRAM_BASE Main/external secure RAM base address * TDDRAM_SIZE Main/external secure RAM byte size * TDSRAM_BASE On-chip secure RAM base address, required by pager. * TDSRAM_SIZE On-chip secure RAM byte size, required by pager. * * TEE_LOAD_ADDR Only defined here if CFG_TEE_LOAD_ADDR is defined. * Otherwise we expect the platform_config.h to define it * unless which LEE_LOAD_ADDR defaults to TEE_RAM_START. * * TEE_RAM_VA_SIZE Set to CFG_TEE_RAM_VA_SIZE or defaults to * CORE_MMU_PGDIR_SIZE. * * TEE_SDP_TEST_MEM_BASE Define if a SDP memory pool is required and none set. * Always defined in the inner top (high addresses) * of CFG_TDDRAM_START/_SIZE. * TEE_SDP_TEST_MEM_SIZE Set to CFG_TEE_SDP_MEM_SIZE or a default size. * * ---------------------------------------------------------------------------- * TEE RAM layout without CFG_WITH_PAGER *_ * +----------------------------------+ <-- CFG_TDDRAM_START * | TEE core secure RAM (TEE_RAM) | * +----------------------------------+ * | Trusted Application RAM (TA_RAM) | * +----------------------------------+ * | SDP test memory (optional) | * +----------------------------------+ <-- CFG_TDDRAM_START + CFG_TDDRAM_SIZE * * +----------------------------------+ <-- CFG_SHMEM_START * | Non-secure static SHM | * +----------------------------------+ <-- CFG_SHMEM_START + CFG_SHMEM_SIZE * * ---------------------------------------------------------------------------- * TEE RAM layout with CFG_WITH_PAGER=y and undefined CFG_TDSRAM_START/_SIZE * * +----------------------------------+ <-- CFG_TDDRAM_START * | TEE core secure RAM (TEE_RAM) | | | CFG_CORE_TDSRAM_EMUL_SIZE * +----------------------------------+ --|-' * | reserved (for kasan) | | TEE_RAM_VA_SIZE * +----------------------------------+ --' * | TA RAM / Pagestore (TA_RAM) | * +----------------------------------+ <---- align with CORE_MMU_PGDIR_SIZE * +----------------------------------+ <-- * | SDP test memory (optional) | | CFG_TEE_SDP_MEM_SIZE * +----------------------------------+ <-+ CFG_TDDRAM_START + CFG_TDDRAM_SIZE * * +----------------------------------+ <-- CFG_SHMEM_START * | Non-secure static SHM | | * +----------------------------------+ v CFG_SHMEM_SIZE * * ---------------------------------------------------------------------------- * TEE RAM layout with CFG_WITH_PAGER=y and define CFG_TDSRAM_START/_SIZE * * +----------------------------------+ <-- CFG_TDSRAM_START * | TEE core secure RAM (TEE_RAM) | | CFG_TDSRAM_SIZE * +----------------------------------+ --' * * +----------------------------------+ <- CFG_TDDRAM_START * | TA RAM / Pagestore (TA_RAM) | * |----------------------------------+ <---- align with CORE_MMU_PGDIR_SIZE * |----------------------------------+ <-- * | SDP test memory (optional) | | CFG_TEE_SDP_MEM_SIZE * +----------------------------------+ <-+ CFG_TDDRAM_START + CFG_TDDRAM_SIZE * * +----------------------------------+ <-- CFG_SHMEM_START * | Non-secure static SHM | | * +----------------------------------+ v CFG_SHMEM_SIZE */ 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 // core/arch/riscv/mm/core_mmu_arch.c static struct mmu_pgt root_pgt[CFG_TEE_CORE_NB_CORE] __aligned(RISCV_PGSIZE) __section(".nozi.mmu.root_pgt"); static struct mmu_pgt pool_pgts[RISCV_MMU_MAX_PGTS] __aligned(RISCV_PGSIZE) __section(".nozi.mmu.pool_pgts"); static struct mmu_pgt user_pgts[CFG_NUM_THREADS] __aligned(RISCV_PGSIZE) __section(".nozi.mmu.usr_pgts"); struct mmu_partition { struct mmu_pgt *root_pgt; struct mmu_pgt *pool_pgts; struct mmu_pgt *user_pgts; unsigned int pgts_used; unsigned int asid; }; static struct mmu_partition default_partition __nex_data = { .root_pgt = root_pgt, // Root page tables, per-core. .pool_pgts = pool_pgts, // Page tables pool. .user_pgts = user_pgts, // User page tables, per-thread. .pgts_used = 0, // Increased when a pool page table is allocated. // See: core_mmu_pgt_alloc(). .asid = 0 }; 1 2 3 4 5 6 7 8 9 10 // include/mm/tee_mmu_types.h struct tee_mmap_region { unsigned int type; /* enum teecore_memtypes */ unsigned int region_size; paddr_t pa; vaddr_t va; size_t size; uint32_t attr; /* TEE_MATTR_* above */ }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 // include/mm/core_mmu.h /* * Memory area type: * MEM_AREA_END: Reserved, marks the end of a table of mapping areas. * MEM_AREA_TEE_RAM: core RAM (read/write/executable, secure, reserved to TEE) * MEM_AREA_TEE_RAM_RX: core private read-only/executable memory (secure) * MEM_AREA_TEE_RAM_RO: core private read-only/non-executable memory (secure) * MEM_AREA_TEE_RAM_RW: core private read/write/non-executable memory (secure) * MEM_AREA_INIT_RAM_RO: init private read-only/non-executable memory (secure) * MEM_AREA_INIT_RAM_RX: init private read-only/executable memory (secure) * MEM_AREA_NEX_RAM_RO: nexus private read-only/non-executable memory (secure) * MEM_AREA_NEX_RAM_RW: nexus private r/w/non-executable memory (secure) * MEM_AREA_TEE_COHERENT: teecore coherent RAM (secure, reserved to TEE) * MEM_AREA_TEE_ASAN: core address sanitizer RAM (secure, reserved to TEE) * MEM_AREA_IDENTITY_MAP_RX: core identity mapped r/o executable memory (secure) * MEM_AREA_TA_RAM: Secure RAM where teecore loads/exec TA instances. * MEM_AREA_NSEC_SHM: NonSecure shared RAM between NSec and TEE. * MEM_AREA_NEX_NSEC_SHM: nexus non-secure shared RAM between NSec and TEE. * MEM_AREA_RAM_NSEC: NonSecure RAM storing data * MEM_AREA_RAM_SEC: Secure RAM storing some secrets * MEM_AREA_ROM_SEC: Secure read only memory storing some secrets * MEM_AREA_IO_NSEC: NonSecure HW mapped registers * MEM_AREA_IO_SEC: Secure HW mapped registers * MEM_AREA_EXT_DT: Memory loads external device tree * MEM_AREA_MANIFEST_DT: Memory loads manifest device tree * MEM_AREA_TRANSFER_LIST: Memory area mapped for Transfer List * MEM_AREA_RES_VASPACE: Reserved virtual memory space * MEM_AREA_SHM_VASPACE: Virtual memory space for dynamic shared memory buffers * MEM_AREA_TS_VASPACE: TS va space, only used with phys_to_virt() * MEM_AREA_DDR_OVERALL: Overall DDR address range, candidate to dynamic shm. * MEM_AREA_SEC_RAM_OVERALL: Whole secure RAM * MEM_AREA_MAXTYPE: lower invalid 'type' value */ enum teecore_memtypes { MEM_AREA_END = 0, MEM_AREA_TEE_RAM, MEM_AREA_TEE_RAM_RX, MEM_AREA_TEE_RAM_RO, MEM_AREA_TEE_RAM_RW, MEM_AREA_INIT_RAM_RO, MEM_AREA_INIT_RAM_RX, MEM_AREA_NEX_RAM_RO, MEM_AREA_NEX_RAM_RW, MEM_AREA_TEE_COHERENT, MEM_AREA_TEE_ASAN, MEM_AREA_IDENTITY_MAP_RX, MEM_AREA_TA_RAM, MEM_AREA_NSEC_SHM, MEM_AREA_NEX_NSEC_SHM, MEM_AREA_RAM_NSEC, MEM_AREA_RAM_SEC, MEM_AREA_ROM_SEC, MEM_AREA_IO_NSEC, MEM_AREA_IO_SEC, MEM_AREA_EXT_DT, MEM_AREA_MANIFEST_DT, MEM_AREA_TRANSFER_LIST, MEM_AREA_RES_VASPACE, MEM_AREA_SHM_VASPACE, MEM_AREA_TS_VASPACE, MEM_AREA_PAGER_VASPACE, MEM_AREA_SDP_MEM, MEM_AREA_DDR_OVERALL, MEM_AREA_SEC_RAM_OVERALL, MEM_AREA_MAXTYPE }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 // mm/core_mmu.c /* Define the platform's memory layout. */ struct memaccess_area { paddr_t paddr; size_t size; }; #define MEMACCESS_AREA(a, s) { .paddr = a, .size = s } // secure_only[] defines the bases and the sizes of secure memories // used by OP-TEE. static struct memaccess_area secure_only[] __nex_data = { #ifdef CFG_CORE_PHYS_RELOCATABLE MEMACCESS_AREA(0, 0), #else #ifdef TRUSTED_SRAM_BASE MEMACCESS_AREA(TRUSTED_SRAM_BASE, TRUSTED_SRAM_SIZE), #endif // e.g. // TRUSTED_DRAM_BASE = TDDRAM_BASE = CFG_TDDRAM_START = 0xf1000000 // TRUSTED_DRAM_SIZE = TDDRAM_SIZE = CFG_TDDRAM_SIZE = 0x01000000 (16 MB) MEMACCESS_AREA(TRUSTED_DRAM_BASE, TRUSTED_DRAM_SIZE), #endif }; // nsec_shared[] defines the bases and sizes of the non-secure static // shared memories. static struct memaccess_area nsec_shared[] __nex_data = { #ifdef CFG_CORE_RESERVED_SHM // e.g. // TEE_SHMEM_START = TEE_SHMEM_START = CFG_SHMEM_START // TEE_SHMEM_SIZE = TEE_SHMEM_SIZE = CFG_SHMEM_SIZE MEMACCESS_AREA(TEE_SHMEM_START, TEE_SHMEM_SIZE), #endif }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 // mm/core_mmu.c static struct tee_mmap_region static_memory_map[CFG_MMAP_REGIONS #if defined(CFG_CORE_ASLR) || defined(CFG_CORE_PHYS_RELOCATABLE) + 1 #endif + 1] __nex_bss; ..... // _start() -> core_init_mmu_map() /* * core_init_mmu_map() - init tee core default memory mapping * * This routine sets the static default TEE core mapping. If @seed is > 0 * and configured with CFG_CORE_ASLR it will map tee core at a location * based on the seed and return the offset from the link address. * * If an error happened: core_init_mmu_map is expected to panic. * * Note: this function is weak just to make it possible to exclude it from * the unpaged area. */ void __weak core_init_mmu_map(unsigned long seed, struct core_mmu_config *cfg) { #ifndef CFG_NS_VIRTUALIZATION vaddr_t start = ROUNDDOWN((vaddr_t)__nozi_start, SMALL_PAGE_SIZE); #else vaddr_t start = ROUNDDOWN((vaddr_t)__vcore_nex_rw_start, SMALL_PAGE_SIZE); #endif vaddr_t len = ROUNDUP((vaddr_t)__nozi_end, SMALL_PAGE_SIZE) - start; // tmp_mmap is allocated from the heap (__heap1_start or __heap2_start). struct tee_mmap_region *tmp_mmap = get_tmp_mmap(); unsigned long offs = 0; if (IS_ENABLED(CFG_CORE_PHYS_RELOCATABLE) && (core_mmu_tee_load_pa & SMALL_PAGE_MASK)) panic("OP-TEE load address is not page aligned"); check_sec_nsec_mem_config(); /* * Add a entry covering the translation tables which will be * involved in some virt_to_phys() and phys_to_virt() conversions. */ static_memory_map[0] = (struct tee_mmap_region){ .type = MEM_AREA_TEE_RAM, .region_size = SMALL_PAGE_SIZE, .pa = start, .va = start, .size = len, .attr = core_mmu_type_to_attr(MEM_AREA_IDENTITY_MAP_RX), }; COMPILE_TIME_ASSERT(CFG_MMAP_REGIONS >= 13); // Initalize memory maps. offs = init_mem_map(tmp_mmap, ARRAY_SIZE(static_memory_map), seed); check_mem_map(tmp_mmap); // Set page table entries for memory maps. [core_init_mmu(tmp_mmap);](/posts/optee-memory-mgnt/) dump_xlat_table(0x0, CORE_MMU_BASE_TABLE_LEVEL); // Set cfg->satp[]. core_init_mmu_regs(cfg); cfg->map_offset = offs; // Copy the temp memory maps. memcpy(static_memory_map, tmp_mmap, sizeof(static_memory_map)); } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 // mm/core_mmu.c // _start() -> core_init_mmu_map() -> init_mem_map() -> collect_mem_ranges() // e.g. // D/TC:0 add_phys_mem:677 VCORE_UNPG_RX_PA type TEE_RAM_RX 0xf1000000 size 0x00092000 // D/TC:0 add_phys_mem:677 VCORE_UNPG_RW_PA type TEE_RAM_RW 0xf1092000 size 0x0016e000 // D/TC:0 add_phys_mem:677 ta_base type TA_RAM 0xf1200000 size 0x00e00000 // D/TC:0 add_va_space:717 type RES_VASPACE size 0x00a00000 // D/TC:0 add_va_space:717 type SHM_VASPACE size 0x02000000 // D/TC:0 dump_mmap_table:849 type TEE_RAM_RX va 0xf1000000..0xf1091fff pa 0xf1000000..0xf1091fff size 0x00092000 (smallpg) // D/TC:0 dump_mmap_table:849 type TEE_RAM_RW va 0xf1092000..0xf11fffff pa 0xf1092000..0xf11fffff size 0x0016e000 (smallpg) // D/TC:0 dump_mmap_table:849 type RES_VASPACE va 0xf1200000..0xf1bfffff pa 0x00000000..0x009fffff size 0x00a00000 (pgdir) // D/TC:0 dump_mmap_table:849 type SHM_VASPACE va 0xf1c00000..0xf3bfffff pa 0x00000000..0x01ffffff size 0x02000000 (pgdir) // D/TC:0 dump_mmap_table:849 type TA_RAM va 0xf3c00000..0xf49fffff pa 0xf1200000..0xf1ffffff size 0x00e00000 (pgdir) static size_t collect_mem_ranges(struct tee_mmap_region *memory_map, size_t num_elems) { const struct core_mmu_phys_mem *mem = NULL; vaddr_t ram_start = secure_only[0].paddr; size_t last = 0; #define ADD_PHYS_MEM(_type, _addr, _size) \ add_phys_mem(memory_map, num_elems, #_addr, (_type), \ (_addr), (_size), &last) if (IS_ENABLED(CFG_CORE_RWDATA_NOEXEC)) { ADD_PHYS_MEM(MEM_AREA_TEE_RAM_RO, ram_start, VCORE_UNPG_RX_PA - ram_start); ADD_PHYS_MEM(MEM_AREA_TEE_RAM_RX, VCORE_UNPG_RX_PA, VCORE_UNPG_RX_SZ); ADD_PHYS_MEM(MEM_AREA_TEE_RAM_RO, VCORE_UNPG_RO_PA, VCORE_UNPG_RO_SZ); if (IS_ENABLED(CFG_NS_VIRTUALIZATION)) { ADD_PHYS_MEM(MEM_AREA_NEX_RAM_RO, VCORE_UNPG_RW_PA, VCORE_UNPG_RW_SZ); ADD_PHYS_MEM(MEM_AREA_NEX_RAM_RW, VCORE_NEX_RW_PA, VCORE_NEX_RW_SZ); } else { ADD_PHYS_MEM(MEM_AREA_TEE_RAM_RW, VCORE_UNPG_RW_PA, VCORE_UNPG_RW_SZ); } if (IS_ENABLED(CFG_WITH_PAGER)) { ADD_PHYS_MEM(MEM_AREA_INIT_RAM_RX, VCORE_INIT_RX_PA, VCORE_INIT_RX_SZ); ADD_PHYS_MEM(MEM_AREA_INIT_RAM_RO, VCORE_INIT_RO_PA, VCORE_INIT_RO_SZ); } } else { ADD_PHYS_MEM(MEM_AREA_TEE_RAM, TEE_RAM_START, TEE_RAM_PH_SIZE); } if (IS_ENABLED(CFG_NS_VIRTUALIZATION)) { ADD_PHYS_MEM(MEM_AREA_SEC_RAM_OVERALL, TRUSTED_DRAM_BASE, TRUSTED_DRAM_SIZE); } else { /* * Every guest will have own TA RAM if virtualization * support is enabled. */ paddr_t ta_base = 0; size_t ta_size = 0; core_mmu_get_ta_range(&ta_base, &ta_size); ADD_PHYS_MEM(MEM_AREA_TA_RAM, ta_base, ta_size); } if (IS_ENABLED(CFG_CORE_SANITIZE_KADDRESS) && IS_ENABLED(CFG_WITH_PAGER)) { /* * Asan ram is part of MEM_AREA_TEE_RAM_RW when pager is * disabled. */ ADD_PHYS_MEM(MEM_AREA_TEE_ASAN, ASAN_MAP_PA, ASAN_MAP_SZ); } #undef ADD_PHYS_MEM /* Collect device memory info from SP manifest */ if (IS_ENABLED(CFG_CORE_SEL2_SPMC)) collect_device_mem_ranges(memory_map, num_elems, &last); for (mem = phys_mem_map_begin; mem < phys_mem_map_end; mem++) { /* Only unmapped virtual range may have a null phys addr */ assert(mem->addr || !core_mmu_type_to_attr(mem->type)); add_phys_mem(memory_map, num_elems, mem->name, mem->type, mem->addr, mem->size, &last); } if (IS_ENABLED(CFG_SECURE_DATA_PATH)) verify_special_mem_areas(memory_map, phys_sdp_mem_begin, phys_sdp_mem_end, "SDP"); add_va_space(memory_map, num_elems, MEM_AREA_RES_VASPACE, CFG_RESERVED_VASPACE_SIZE, &last); add_va_space(memory_map, num_elems, MEM_AREA_SHM_VASPACE, SHM_VASPACE_SIZE, &last); memory_map[last].type = MEM_AREA_END; return last; } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 // mm/core_mmu.c static void add_phys_mem(struct tee_mmap_region *memory_map, size_t num_elems, const char *mem_name __maybe_unused, enum teecore_memtypes mem_type, paddr_t mem_addr, paddr_size_t mem_size, size_t *last) { size_t n = 0; paddr_t pa; paddr_size_t size; if (!mem_size) /* Discard null size entries */ return; /* * If some ranges of memory of the same type do overlap * each others they are coalesced into one entry. To help this * added entries are sorted by increasing physical. * * Note that it's valid to have the same physical memory as several * different memory types, for instance the same device memory * mapped as both secure and non-secure. This will probably not * happen often in practice. */ DMSG("%s type %s 0x%08" PRIxPA " size 0x%08" PRIxPASZ, mem_name, teecore_memtype_name(mem_type), mem_addr, mem_size); // Iternate the existing memory maps. // If there's any existing memory region overlap with the one // we intend to add, if both of their memory types are same, // merge them into a single memory region. // After the iteration, 'n' will be the position to be added into memory maps. while (true) { if (n >= (num_elems - 1)) { EMSG("Out of entries (%zu) in memory_map", num_elems); panic(); } if (n == *last) break; pa = memory_map[n].pa; size = memory_map[n].size; // Merge the overlapped memory regions. if (mem_type == memory_map[n].type && ((pa <= (mem_addr + (mem_size - 1))) && (mem_addr <= (pa + (size - 1))))) { DMSG("Physical mem map overlaps 0x%" PRIxPA, mem_addr); memory_map[n].pa = MIN(pa, mem_addr); memory_map[n].size = MAX(size, mem_size) + (pa - memory_map[n].pa); return; } if (mem_type < memory_map[n].type || (mem_type == memory_map[n].type && mem_addr < pa)) break; /* found the spot where to insert this memory */ n++; } // Insert the memory region into the memory maps. memmove(memory_map + n + 1, memory_map + n, sizeof(struct tee_mmap_region) * (*last - n)); (*last)++; memset(memory_map + n, 0, sizeof(memory_map[0])); memory_map[n].type = mem_type; memory_map[n].pa = mem_addr; memory_map[n].size = mem_size; } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 // mm/core_mmu.c static void add_va_space(struct tee_mmap_region *memory_map, size_t num_elems, enum teecore_memtypes type, size_t size, size_t *last) { size_t n = 0; DMSG("type %s size 0x%08zx", teecore_memtype_name(type), size); // Find the position to insert the memory region into the memory maps. while (true) { if (n >= (num_elems - 1)) { EMSG("Out of entries (%zu) in memory_map", num_elems); panic(); } if (n == *last) break; if (type < memory_map[n].type) break; n++; } // Insert the memory region into the memory maps without physical address. memmove(memory_map + n + 1, memory_map + n, sizeof(struct tee_mmap_region) * (*last - n)); (*last)++; memset(memory_map + n, 0, sizeof(memory_map[0])); memory_map[n].type = type; memory_map[n].size = size; } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 // core/arch/riscv/mm/core_mmu_arch.c // _start() -> reset_primary() -> core_init_mmu_map() -> core_init_mmu() // The memory maps are created in init_mem_map(). // core_init_mmu() will set the page table entries for memory maps // on the root page table (root_pgt) by calling: // core_init_mmu_prtn_tee() -> core_mmu_map_region(). // Page table entries are also copied to the root page tables of the // rest of the cores in core_init_mmu_prtn_tee(). void core_init_mmu(struct tee_mmap_region *mm) { uint64_t max_va = 0; size_t n = 0; static_assert((RISCV_MMU_MAX_PGTS * RISCV_MMU_PGT_SIZE) == sizeof(pool_pgts)); /* Initialize default pagetables */ core_init_mmu_prtn_tee(&default_partition, mm); for (n = 0; !core_mmap_is_end_of_table(mm + n); n++) { vaddr_t va_end = mm[n].va + mm[n].size - 1; if (va_end > max_va) max_va = va_end; } set_user_va_idx(&default_partition); core_init_mmu_prtn_ta(&default_partition); assert(max_va < BIT64(RISCV_MMU_VA_WIDTH)); } 1 2 3 4 5 // core/arch/riscv/kernel/entry.S LOCAL_DATA boot_mmu_config , : /* struct core_mmu_config */ .skip CORE_MMU_CONFIG_SIZE END_DATA boot_mmu_config 1 2 3 4 5 6 7 8 9 10 // core/arch/riscv/mm/core_mmu_arch.h struct core_mmu_config { // satp[] is set to root page table (root_pgt) for each core by: // _start() -> reset_primary() -> core_init_mmu_map() -> // core_init_mmu_regs(). // And is set to satp CSR in: set_satp() for each core. unsigned long satp[CFG_TEE_CORE_NB_CORE]; uint32_t map_offset; }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 // core/include/mm/tee_mm.h struct _tee_mm_entry_t { struct _tee_mm_pool_t *pool; struct _tee_mm_entry_t *next; uint32_t offset; /* offset in pages/sections */ uint32_t size; /* size in pages/sections */ }; typedef struct _tee_mm_entry_t tee_mm_entry_t; struct _tee_mm_pool_t { tee_mm_entry_t *entry; paddr_t lo; /* low boundary of the pool */ paddr_size_t size; /* pool size */ uint32_t flags; /* Config flags for the pool */ uint8_t shift; /* size shift */ unsigned int lock; #ifdef CFG_WITH_STATS size_t max_allocated; #endif }; typedef struct _tee_mm_pool_t tee_mm_pool_t; 1 2 3 4 5 6 7 8 9 10 // core/mm/core_mmu.c /* Physical Secure DDR pool */ tee_mm_pool_t tee_mm_sec_ddr; /* Virtual memory pool for core mappings */ tee_mm_pool_t core_virt_mem_pool; /* Virtual memory pool for shared memory mappings */ tee_mm_pool_t core_virt_shm_pool;

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca Currently, RISC-V OP-TEE doesn’t define native interrupts. All interrupts are foreign interrupts: 1 2 3 4 5 6 // core/arch/riscv/include/kernel/thread_arch.h #define THREAD_EXCP_FOREIGN_INTR (CSR_XIE_SIE | CSR_XIE_TIE | CSR_XIE_EIE) #define THREAD_EXCP_NATIVE_INTR (0) #define THREAD_EXCP_ALL (THREAD_EXCP_FOREIGN_INTR |\ THREAD_EXCP_NATIVE_INTR) i.e. OP-TEE DOES NOT handle any interrupts. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 // arch/riscv/kernel/thread_rv.S FUNC thread_trap_vect , : // mscratch/sscratch = 0: Trap is from kernel. // mscratch/sscratch = 1: Trap is from user. csrrw tp, CSR_XSCRATCH, tp bnez tp, 0f /* Read tp back */ csrrw tp, CSR_XSCRATCH, tp j [trap_from_kernel](/posts/optee-interrupts/) 0: /* Now tp is [thread_core_local](https://app.notion.com/p/KVM_SET_USER_MEMORY_REGION-817fabdad8494e53b35403ee09ff4b8f?pvs=21) */ j [trap_from_user](/posts/optee-interrupts/) thread_trap_vect_end: END_FUNC thread_trap_vect 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 // arch/riscv/kernel/thread_vs.S LOCAL_FUNC trap_from_kernel, : /* Save sp, a0, a1 into temporary spaces of thread_core_local */ store_xregs tp, THREAD_CORE_LOCAL_X0, REG_SP store_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 csrr a0, CSR_XCAUSE /* MSB of cause differentiates between interrupts and exceptions */ // P.S. bge is signed comparison. bge a0, zero, exception_from_kernel interrupt_from_kernel: /* Get thread context as sp */ get_thread_ctx sp, a0 /* Load and save kernel sp */ load_xregs tp, THREAD_CORE_LOCAL_X0, REG_A0 store_xregs sp, THREAD_CTX_REG_SP, REG_A0 /* Restore user a0, a1 which can be saved later */ load_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 /* Save all other GPRs */ store_xregs sp, THREAD_CTX_REG_RA, REG_RA store_xregs sp, THREAD_CTX_REG_GP, REG_GP store_xregs sp, THREAD_CTX_REG_T0, REG_T0, REG_T2 store_xregs sp, THREAD_CTX_REG_S0, REG_S0, REG_S1 store_xregs sp, THREAD_CTX_REG_A0, REG_A0, REG_A7 store_xregs sp, THREAD_CTX_REG_S2, REG_S2, REG_S11 store_xregs sp, THREAD_CTX_REG_T3, REG_T3, REG_T6 /* Save XIE */ csrr t0, CSR_XIE store_xregs sp, THREAD_CTX_REG_IE, REG_T0 /* Mask all interrupts */ csrw CSR_XIE, x0 /* Save XSTATUS */ csrr t0, CSR_XSTATUS store_xregs sp, THREAD_CTX_REG_STATUS, REG_T0 /* Save XEPC */ csrr t0, CSR_XEPC store_xregs sp, THREAD_CTX_REG_EPC, REG_T0 /* * a0 = cause * a1 = sp * Call thread_interrupt_handler(cause, regs) */ csrr a0, CSR_XCAUSE mv a1, sp /* Load tmp_stack_va_end as current sp. */ load_xregs tp, THREAD_CORE_LOCAL_TMP_STACK_VA_END, REG_SP call [thread_interrupt_handler](/posts/optee-interrupts/) /* Get thread context as sp */ get_thread_ctx sp, t0 /* Restore XEPC */ load_xregs sp, THREAD_CTX_REG_EPC, REG_T0 csrw CSR_XEPC, t0 /* Restore XIE */ load_xregs sp, THREAD_CTX_REG_IE, REG_T0 csrw CSR_XIE, t0 /* Restore XSTATUS */ load_xregs sp, THREAD_CTX_REG_STATUS, REG_T0 csrw CSR_XSTATUS, t0 /* Set scratch as thread_core_local */ csrw CSR_XSCRATCH, tp /* Restore all GPRs */ load_xregs sp, THREAD_CTX_REG_RA, REG_RA load_xregs sp, THREAD_CTX_REG_GP, REG_GP load_xregs sp, THREAD_CTX_REG_T0, REG_T0, REG_T2 load_xregs sp, THREAD_CTX_REG_S0, REG_S0, REG_S1 load_xregs sp, THREAD_CTX_REG_A0, REG_A0, REG_A7 load_xregs sp, THREAD_CTX_REG_S2, REG_S2, REG_S11 load_xregs sp, THREAD_CTX_REG_T3, REG_T3, REG_T6 load_xregs sp, THREAD_CTX_REG_SP, REG_SP XRET exception_from_kernel: /* * Update core local flags. * flags = (flags << THREAD_CLF_SAVED_SHIFT) | THREAD_CLF_ABORT; */ lw a0, THREAD_CORE_LOCAL_FLAGS(tp) slli a0, a0, THREAD_CLF_SAVED_SHIFT ori a0, a0, THREAD_CLF_ABORT li a1, (THREAD_CLF_ABORT << THREAD_CLF_SAVED_SHIFT) and a1, a0, a1 bnez a1, sel_tmp_sp /* Select abort stack */ load_xregs tp, THREAD_CORE_LOCAL_ABT_STACK_VA_END, REG_A1 j set_sp sel_tmp_sp: /* We have an abort while using the abort stack, select tmp stack */ load_xregs tp, THREAD_CORE_LOCAL_TMP_STACK_VA_END, REG_A1 ori a0, a0, THREAD_CLF_TMP /* flags |= THREAD_CLF_TMP; */ set_sp: mv sp, a1 sw a0, THREAD_CORE_LOCAL_FLAGS(tp) /* * Save state on stack */ addi sp, sp, -THREAD_ABT_REGS_SIZE /* Save kernel sp */ load_xregs tp, THREAD_CORE_LOCAL_X0, REG_A0 store_xregs sp, THREAD_ABT_REG_SP, REG_A0 /* Restore kernel a0, a1 which can be saved later */ load_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 /* Save all other GPRs */ store_xregs sp, THREAD_ABT_REG_RA, REG_RA store_xregs sp, THREAD_ABT_REG_GP, REG_GP store_xregs sp, THREAD_ABT_REG_TP, REG_TP store_xregs sp, THREAD_ABT_REG_T0, REG_T0, REG_T2 store_xregs sp, THREAD_ABT_REG_S0, REG_S0, REG_S1 store_xregs sp, THREAD_ABT_REG_A0, REG_A0, REG_A7 store_xregs sp, THREAD_ABT_REG_S2, REG_S2, REG_S11 store_xregs sp, THREAD_ABT_REG_T3, REG_T3, REG_T6 /* Save XIE */ csrr t0, CSR_XIE store_xregs sp, THREAD_ABT_REG_IE, REG_T0 /* Mask all interrupts */ csrw CSR_XIE, x0 /* Save XSTATUS */ csrr t0, CSR_XSTATUS store_xregs sp, THREAD_ABT_REG_STATUS, REG_T0 /* Save XEPC */ csrr t0, CSR_XEPC store_xregs sp, THREAD_ABT_REG_EPC, REG_T0 /* Save XTVAL */ csrr t0, CSR_XTVAL store_xregs sp, THREAD_ABT_REG_TVAL, REG_T0 /* Save XCAUSE */ csrr a0, CSR_XCAUSE store_xregs sp, THREAD_ABT_REG_CAUSE, REG_A0 /* * a0 = cause * a1 = sp (struct thread_abort_regs *regs) * Call abort_handler(cause, regs) */ mv a1, sp call abort_handler /* * Restore state from stack */ /* Restore XEPC */ load_xregs sp, THREAD_ABT_REG_EPC, REG_T0 csrw CSR_XEPC, t0 /* Restore XIE */ load_xregs sp, THREAD_ABT_REG_IE, REG_T0 csrw CSR_XIE, t0 /* Restore XSTATUS */ load_xregs sp, THREAD_ABT_REG_STATUS, REG_T0 csrw CSR_XSTATUS, t0 /* Set scratch as thread_core_local */ csrw CSR_XSCRATCH, tp /* Update core local flags */ lw a0, THREAD_CORE_LOCAL_FLAGS(tp) srli a0, a0, THREAD_CLF_SAVED_SHIFT sw a0, THREAD_CORE_LOCAL_FLAGS(tp) /* Restore all GPRs */ load_xregs sp, THREAD_ABT_REG_RA, REG_RA load_xregs sp, THREAD_ABT_REG_GP, REG_GP load_xregs sp, THREAD_ABT_REG_TP, REG_TP load_xregs sp, THREAD_ABT_REG_T0, REG_T0, REG_T2 load_xregs sp, THREAD_ABT_REG_S0, REG_S0, REG_S1 load_xregs sp, THREAD_ABT_REG_A0, REG_A0, REG_A7 load_xregs sp, THREAD_ABT_REG_S2, REG_S2, REG_S11 load_xregs sp, THREAD_ABT_REG_T3, REG_T3, REG_T6 load_xregs sp, THREAD_ABT_REG_SP, REG_SP XRET END_FUNC trap_from_kernel 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 // arch/riscv/kernel/thread_rv.S LOCAL_FUNC trap_from_user, : /* Save user sp, a0, a1 into temporary spaces of thread_core_local */ store_xregs tp, THREAD_CORE_LOCAL_X0, REG_SP store_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 csrr a0, CSR_XCAUSE /* MSB of cause differentiates between interrupts and exceptions */ bge a0, zero, exception_from_user interrupt_from_user: /* Get thread context as sp */ get_thread_ctx sp, a0 /* Save user sp */ load_xregs tp, THREAD_CORE_LOCAL_X0, REG_A0 store_xregs sp, THREAD_CTX_REG_SP, REG_A0 /* Restore user a0, a1 which can be saved later */ load_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 /* Save user gp */ store_xregs sp, THREAD_CTX_REG_GP, REG_GP /* * Set the scratch register to 0 such in case of a recursive * exception thread_trap_vect() knows that it is emitted from kernel. */ csrrw gp, CSR_XSCRATCH, zero /* Save user tp we previously swapped into CSR_XSCRATCH */ store_xregs sp, THREAD_CTX_REG_TP, REG_GP /* Set kernel gp */ .option push .option norelax la gp, __global_pointer$ .option pop /* Save all other GPRs */ store_xregs sp, THREAD_CTX_REG_RA, REG_RA store_xregs sp, THREAD_CTX_REG_T0, REG_T0, REG_T2 store_xregs sp, THREAD_CTX_REG_S0, REG_S0, REG_S1 store_xregs sp, THREAD_CTX_REG_A0, REG_A0, REG_A7 store_xregs sp, THREAD_CTX_REG_S2, REG_S2, REG_S11 store_xregs sp, THREAD_CTX_REG_T3, REG_T3, REG_T6 /* Save XIE */ csrr t0, CSR_XIE store_xregs sp, THREAD_CTX_REG_IE, REG_T0 /* Mask all interrupts */ csrw CSR_XIE, x0 /* Save XSTATUS */ csrr t0, CSR_XSTATUS store_xregs sp, THREAD_CTX_REG_STATUS, REG_T0 /* Save XEPC */ csrr t0, CSR_XEPC store_xregs sp, THREAD_CTX_REG_EPC, REG_T0 /* * a0 = cause * a1 = sp * Call thread_interrupt_handler(cause, regs) */ csrr a0, CSR_XCAUSE mv a1, sp /* Load tmp_stack_va_end as current sp. */ load_xregs tp, THREAD_CORE_LOCAL_TMP_STACK_VA_END, REG_SP call [thread_interrupt_handler](/posts/optee-interrupts/) /* Get thread context as sp */ get_thread_ctx sp, t0 /* Restore XEPC */ load_xregs sp, THREAD_CTX_REG_EPC, REG_T0 csrw CSR_XEPC, t0 /* Restore XIE */ load_xregs sp, THREAD_CTX_REG_IE, REG_T0 csrw CSR_XIE, t0 /* Restore XSTATUS */ load_xregs sp, THREAD_CTX_REG_STATUS, REG_T0 csrw CSR_XSTATUS, t0 /* Set scratch as thread_core_local */ csrw CSR_XSCRATCH, tp /* Restore all GPRs */ load_xregs sp, THREAD_CTX_REG_RA, REG_RA load_xregs sp, THREAD_CTX_REG_GP, REG_GP load_xregs sp, THREAD_CTX_REG_TP, REG_TP load_xregs sp, THREAD_CTX_REG_T0, REG_T0, REG_T2 load_xregs sp, THREAD_CTX_REG_S0, REG_S0, REG_S1 load_xregs sp, THREAD_CTX_REG_A0, REG_A0, REG_A7 load_xregs sp, THREAD_CTX_REG_S2, REG_S2, REG_S11 load_xregs sp, THREAD_CTX_REG_T3, REG_T3, REG_T6 load_xregs sp, THREAD_CTX_REG_SP, REG_SP XRET exception_from_user: /* a0 is CSR_XCAUSE */ li a1, CAUSE_USER_ECALL bne a0, a1, abort_from_user ecall_from_user: /* Load and set kernel sp from thread context */ get_thread_ctx a0, a1 load_xregs a0, THREAD_CTX_KERN_SP, REG_SP /* Now sp is kernel sp, create stack for struct thread_scall_regs */ addi sp, sp, -THREAD_SCALL_REGS_SIZE /* Save user sp */ load_xregs tp, THREAD_CORE_LOCAL_X0, REG_A0 store_xregs sp, THREAD_SCALL_REG_SP, REG_A0 /* Restore user a0, a1 which can be saved later */ load_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 /* Save user gp */ store_xregs sp, THREAD_SCALL_REG_GP, REG_GP /* * Set the scratch register to 0 such in case of a recursive * exception thread_trap_vect() knows that it is emitted from kernel. */ csrrw gp, CSR_XSCRATCH, zero /* Save user tp we previously swapped into CSR_XSCRATCH */ store_xregs sp, THREAD_SCALL_REG_TP, REG_GP /* Set kernel gp */ .option push .option norelax la gp, __global_pointer$ .option pop /* Save other caller-saved registers */ store_xregs sp, THREAD_SCALL_REG_RA, REG_RA store_xregs sp, THREAD_SCALL_REG_T0, REG_T0, REG_T2 store_xregs sp, THREAD_SCALL_REG_A0, REG_A0, REG_A7 store_xregs sp, THREAD_SCALL_REG_T3, REG_T3, REG_T6 /* Save XIE */ csrr a0, CSR_XIE store_xregs sp, THREAD_SCALL_REG_IE, REG_A0 /* Mask all interrupts */ csrw CSR_XIE, zero /* Save XSTATUS */ csrr a0, CSR_XSTATUS store_xregs sp, THREAD_SCALL_REG_STATUS, REG_A0 /* Save XEPC */ csrr a0, CSR_XEPC store_xregs sp, THREAD_SCALL_REG_EPC, REG_A0 /* * a0 = struct thread_scall_regs *regs * Call thread_scall_handler(regs) */ mv a0, sp call [thread_scall_handler](/posts/optee-interrupts/) /* * Save kernel sp we'll had at the beginning of this function. * This is when this TA has called another TA because * __thread_enter_user_mode() also saves the stack pointer in this * field. */ get_thread_ctx a0, a1 addi t0, sp, THREAD_SCALL_REGS_SIZE store_xregs a0, THREAD_CTX_KERN_SP, REG_T0 /* * We are returning to U-Mode, on return, the program counter * is set to xsepc (pc=xepc), we add 4 (size of an instruction) * to continue to next instruction. */ load_xregs sp, THREAD_SCALL_REG_EPC, REG_T0 addi t0, t0, 4 csrw CSR_XEPC, t0 /* Restore XIE */ load_xregs sp, THREAD_SCALL_REG_IE, REG_T0 csrw CSR_XIE, t0 /* Restore XSTATUS */ load_xregs sp, THREAD_SCALL_REG_STATUS, REG_T0 csrw CSR_XSTATUS, t0 /* Set scratch as thread_core_local */ csrw CSR_XSCRATCH, tp /* Restore caller-saved registers */ load_xregs sp, THREAD_SCALL_REG_RA, REG_RA load_xregs sp, THREAD_SCALL_REG_GP, REG_GP load_xregs sp, THREAD_SCALL_REG_TP, REG_TP load_xregs sp, THREAD_SCALL_REG_T0, REG_T0, REG_T2 load_xregs sp, THREAD_SCALL_REG_A0, REG_A0, REG_A7 load_xregs sp, THREAD_SCALL_REG_T3, REG_T3, REG_T6 load_xregs sp, THREAD_SCALL_REG_SP, REG_SP XRET abort_from_user: /* * Update core local flags */ lw a0, THREAD_CORE_LOCAL_FLAGS(tp) slli a0, a0, THREAD_CLF_SAVED_SHIFT ori a0, a0, THREAD_CLF_ABORT sw a0, THREAD_CORE_LOCAL_FLAGS(tp) /* * Save state on stack */ /* Load abt_stack_va_end and set it as sp */ load_xregs tp, THREAD_CORE_LOCAL_ABT_STACK_VA_END, REG_SP /* Now sp is abort sp, create stack for struct thread_abort_regs */ addi sp, sp, -THREAD_ABT_REGS_SIZE /* Save user sp */ load_xregs tp, THREAD_CORE_LOCAL_X0, REG_A0 store_xregs sp, THREAD_ABT_REG_SP, REG_A0 /* Restore user a0, a1 which can be saved later */ load_xregs tp, THREAD_CORE_LOCAL_X1, REG_A0, REG_A1 /* Save user gp */ store_xregs sp, THREAD_ABT_REG_GP, REG_GP /* * Set the scratch register to 0 such in case of a recursive * exception thread_trap_vect() knows that it is emitted from kernel. */ csrrw gp, CSR_XSCRATCH, zero /* Save user tp we previously swapped into CSR_XSCRATCH */ store_xregs sp, THREAD_ABT_REG_TP, REG_GP /* Set kernel gp */ .option push .option norelax la gp, __global_pointer$ .option pop /* Save all other GPRs */ store_xregs sp, THREAD_ABT_REG_RA, REG_RA store_xregs sp, THREAD_ABT_REG_T0, REG_T0, REG_T2 store_xregs sp, THREAD_ABT_REG_S0, REG_S0, REG_S1 store_xregs sp, THREAD_ABT_REG_A0, REG_A0, REG_A7 store_xregs sp, THREAD_ABT_REG_S2, REG_S2, REG_S11 store_xregs sp, THREAD_ABT_REG_T3, REG_T3, REG_T6 /* Save XIE */ csrr t0, CSR_XIE store_xregs sp, THREAD_ABT_REG_IE, REG_T0 /* Mask all interrupts */ csrw CSR_XIE, x0 /* Save XSTATUS */ csrr t0, CSR_XSTATUS store_xregs sp, THREAD_ABT_REG_STATUS, REG_T0 /* Save XEPC */ csrr t0, CSR_XEPC store_xregs sp, THREAD_ABT_REG_EPC, REG_T0 /* Save XTVAL */ csrr t0, CSR_XTVAL store_xregs sp, THREAD_ABT_REG_TVAL, REG_T0 /* Save XCAUSE */ csrr a0, CSR_XCAUSE store_xregs sp, THREAD_ABT_REG_CAUSE, REG_A0 /* * a0 = cause * a1 = sp (struct thread_abort_regs *regs) * Call abort_handler(cause, regs) */ mv a1, sp call abort_handler /* * Restore state from stack */ /* Restore XEPC */ load_xregs sp, THREAD_ABT_REG_EPC, REG_T0 csrw CSR_XEPC, t0 /* Restore XIE */ load_xregs sp, THREAD_ABT_REG_IE, REG_T0 csrw CSR_XIE, t0 /* Restore XSTATUS */ load_xregs sp, THREAD_ABT_REG_STATUS, REG_T0 csrw CSR_XSTATUS, t0 /* Set scratch as thread_core_local */ csrw CSR_XSCRATCH, tp /* Update core local flags */ lw a0, THREAD_CORE_LOCAL_FLAGS(tp) srli a0, a0, THREAD_CLF_SAVED_SHIFT sw a0, THREAD_CORE_LOCAL_FLAGS(tp) /* Restore all GPRs */ load_xregs sp, THREAD_ABT_REG_RA, REG_RA load_xregs sp, THREAD_ABT_REG_GP, REG_GP load_xregs sp, THREAD_ABT_REG_TP, REG_TP load_xregs sp, THREAD_ABT_REG_T0, REG_T0, REG_T2 load_xregs sp, THREAD_ABT_REG_S0, REG_S0, REG_S1 load_xregs sp, THREAD_ABT_REG_A0, REG_A0, REG_A7 load_xregs sp, THREAD_ABT_REG_S2, REG_S2, REG_S11 load_xregs sp, THREAD_ABT_REG_T3, REG_T3, REG_T6 load_xregs sp, THREAD_ABT_REG_SP, REG_SP XRET END_FUNC trap_from_user 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 // core/arch/riscv/kernel/thread_arch.c void thread_interrupt_handler(unsigned long cause, struct thread_ctx_regs *regs) { switch (cause & LONG_MAX) { case IRQ_XTIMER: [thread_foreign_interrupt_handler](/posts/optee-interrupts/)(regs); break; case IRQ_XSOFT: [thread_foreign_interrupt_handler](/posts/optee-interrupts/)(regs); break; case IRQ_XEXT: [thread_foreign_interrupt_handler](/posts/optee-interrupts/)(regs); break; default: thread_unhandled_trap(cause, regs); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 // core/arch/riscv/kernel/thread_rv.S /* * void thread_foreign_interrupt_handler(struct thread_ctx_regs *regs) */ FUNC thread_foreign_interrupt_handler , : /* Mask all interrupt. */ csrw CSR_XIE, x0 mv s0, a0 /* tp = struct thread_core_local */ /* * Update core local flags */ lw s2, THREAD_CORE_LOCAL_FLAGS(tp) slli s2, s2, THREAD_CLF_SAVED_SHIFT ori s2, s2, THREAD_CLF_TMP ori s2, s2, THREAD_CLF_FIQ sw s2, THREAD_CORE_LOCAL_FLAGS(tp) /* * Mark current thread as suspended. * a0 = THREAD_FLAGS_EXIT_ON_FOREIGN_INTR * a1 = status * a2 = epc * thread_state_suspend(flags, status, pc) */ li a0, THREAD_FLAGS_EXIT_ON_FOREIGN_INTR LDR a1, THREAD_CTX_REG_STATUS(s0) LDR a2, THREAD_CTX_REG_EPC(s0) call [thread_state_suspend](/posts/optee-threads/) /* Now return value a0 contains suspended thread ID. */ /* Update core local flags */ lw s3, THREAD_CORE_LOCAL_FLAGS(tp) srli s3, s3, THREAD_CLF_SAVED_SHIFT ori s3, s3, THREAD_CLF_TMP sw s3, THREAD_CORE_LOCAL_FLAGS(tp) /* Passing thread index in a0, and prepare to return to REE. */ mv a4, a0 li a0, TEEABI_OPTEED_RETURN_CALL_DONE li a1, OPTEE_ABI_RETURN_RPC_FOREIGN_INTR mv a2, zero mv a3, zero mv a5, zero j [thread_return_to_udomain](/posts/optee-threads/) END_FUNC thread_foreign_interrupt_handler 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 // core/arch/riscv/kernel/thread_arch.c void thread_scall_handler(struct thread_scall_regs *regs) { struct ts_session *sess = NULL; uint32_t state = 0; /* Enable native interrupts */ state = thread_get_exceptions(); thread_unmask_exceptions(state & ~THREAD_EXCP_NATIVE_INTR); // Do nothing in RISC-V. thread_user_save_vfp(); sess = ts_get_current_session(); /* Restore foreign interrupts which are disabled on exception entry */ thread_restore_foreign_intr(); assert(sess && sess->handle_scall); if (!sess->handle_scall(regs)) { setup_unwind_user_mode(regs); [thread_exit_user_mode](/posts/optee-interrupts/)(regs->a0, regs->a1, regs->a2, regs->a3, regs->sp, regs->ra, regs->status); } } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 // core/arch/riscv/kernel/thread_rv.S /* * void thread_exit_user_mode(unsigned long a0, unsigned long a1, * unsigned long a2, unsigned long a3, * unsigned long sp, unsigned long pc, * unsigned long status); */ FUNC thread_exit_user_mode , : /* Set kernel stack pointer */ mv sp, a4 /* Set xSTATUS */ csrw CSR_XSTATUS, a6 /* * Zeroize xSCRATCH to indicate to thread_trap_vect() * that we are executing in kernel. */ csrw CSR_XSCRATCH, zero /* * Mask all interrupts first. Interrupts will be unmasked after * returning from __thread_enter_user_mode(). */ csrw CSR_XIE, zero /* Set epc as thread_unwind_user_mode() */ csrw CSR_XEPC, a5 XRET END_FUNC thread_exit_user_mode

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 // core/arch/riscv/include/kernel/thread_arch.h struct thread_core_local { unsigned long x[4]; uint32_t hart_id; vaddr_t tmp_stack_va_end; short int curr_thread; uint32_t flags; vaddr_t abt_stack_va_end; #ifdef CFG_TEE_CORE_DEBUG unsigned int locked_count; /* Number of spinlocks held */ #endif #ifdef CFG_CORE_DEBUG_CHECK_STACKS bool stackcheck_recursion; #endif #ifdef CFG_FAULT_MITIGATION struct ftmn_func_arg *ftmn_arg; #endif } THREAD_CORE_LOCAL_ALIGNED; 1 2 3 4 // core/kernel/thread.c // Per-core local threads. struct thread_core_local thread_core_local[CFG_TEE_CORE_NB_CORE] __nex_bss; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 // core/arch/riscv/kernel/entry.S /* * Implement based on the transport method used to communicate between * untrusted domain and trusted domain. It could be an SBI/ECALL-based to * a security monitor running in M-Mode and panic or messaging-based across * domains where we return to a messaging callback which parses and handles * messages. * * void thread_return_to_udomain(unsigned long arg0, unsigned long arg1, * unsigned long arg2, unsigned long arg3, * unsigned long arg4, unsigned long arg5); */ FUNC thread_return_to_udomain , : /* Caller should provide arguments in a0~a5 */ // E.g. when booting: // $a0: If boot core: TEEABI_OPTEED_RETURN_ENTRY_DONE. // Otherwise: TEEABI_OPTEED_RETURN_ON_DONE. // $a1: If boot core: thread_vector_table. // Otherwise: 0x0 (OPTEE_ABI_RETURN_OK) on success // or anything else to indicate error condition. // $a2 ~ $a5: 0x0 #if defined(CFG_RISCV_WITH_M_MODE_SM) jal [thread_return_to_udomain_by_mpxy](/posts/optee-sbi-mpxy/) #else /* Other protocol */ #endif /* ABI to REE should not return */ panic_at_abi_return END_FUNC thread_return_to_udomain 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 // core/arch/riscv/kernel/thread_optee_abi_rv.S /* * Vector table supplied to M-mode secure monitor (e.g., openSBI) at * initialization. * * Note that M-mode secure monitor depends on the layout of this vector table, * any change in layout has to be synced with M-mode secure monitor. */ FUNC thread_vector_table , : , .identity_map, , nobti .option push .option norvc j [vector_std_abi_entry](/posts/optee-threads/) j [vector_fast_abi_entry](/posts/optee-threads/) j . j . j . j . j vector_fiq_entry j . j . .option pop END_FUNC thread_vector_table DECLARE_KEEP_PAGER thread_vector_table 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 // core/arch/riscv/kernel/thread_optee_abi_rv.S LOCAL_FUNC vector_std_abi_entry, : , .identity_map // sbi_mpxy_get_shmem() returns VA of the shared memory. jal sbi_mpxy_get_shmem // Shared memory contents: struct thread_abi_args *arg jal [thread_handle_std_abi](/posts/optee-threads/) /* * Normally thread_handle_std_abi() should return via * thread_exit(), thread_rpc(), but if thread_handle_std_abi() * hasn't switched stack (error detected) it will do a normal "C" * return. */ /* Restore thread_handle_std_abi() return value */ mv a1, a0 li a2, 0 li a3, 0 li a4, 0 // Function ID: This will be placed as the first item in the shared memory. li a0, TEEABI_OPTEED_RETURN_CALL_DONE mv a5, zero /* Return to untrusted domain */ j [thread_return_to_udomain](/posts/optee-threads/) END_FUNC vector_std_abi_entry 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 // core/arch/riscv/kernel/thread_optee_abi_rv.S LOCAL_FUNC vector_fast_abi_entry , : , .identity_map // sbi_mpxy_get_shmem() returns VA of the shared memory. jal sbi_mpxy_get_shmem // Save the shared memory contents to $a0 ~ $a7 mv t0, a0 ld a0, 0(t0) ld a1, 8(t0) ld a2, 16(t0) ld a3, 24(t0) ld a4, 32(t0) ld a5, 40(t0) ld a6, 48(t0) ld a7, 56(t0) // Allocate space on stack to save $a0 ~ $a7. addi sp, sp, -THREAD_ABI_ARGS_SIZE // Store $a0 ~ $a7 to stack so that we can pass // the pointer to thread_handle_fast_abi() store_xregs sp, THREAD_ABI_ARGS_A0, REG_A0, REG_A7 mv a0, sp jal thread_handle_fast_abi // Restore $a0 ~ $a7 from the stack. load_xregs sp, THREAD_ABI_ARGS_A0, REG_A1, REG_A7 // Release the allocated space. addi sp, sp, THREAD_ABI_ARGS_SIZE // Function ID: This will be placed as the first item in the shared memory. li a0, TEEABI_OPTEED_RETURN_CALL_DONE /* Return to untrusted domain */ j [thread_return_to_udomain](/posts/optee-threads/) END_FUNC vector_fast_abi_entry thread_handle_std_abi() If args->a0 == OPTEE_ABI_CALL_RETURN_FROM_RPC: thread_resume_from_rpc() Otherwise: thread_alloc_and_run() thread_alloc_and_run() Call __thread_alloc_and_run() with pc parameter set to thread_std_abi_entry(). So when thread is resumed, thread_std_abi_entry() will be executed. __thread_alloc_and_run() Find the free thread whose state is THREAD_STATE_FREE. If found, set thread’s state to THREAD_STATE_ACTIVE. Set the current thread ID (l->curr_thread) to the founded thread ID. Call init_regs() to initialize the registers to be restored of the thread. thread->regs.epc is set to pc. Call thread_resume() to resume the thread. thread_resume_from_rpc() Check if the state of the thread to be resumed (indicated by thread_id) is THREAD_STATE_SUSPENDED. If yes, set thread’s state to THREAD_STATE_ACTIVE. Set the current thread ID (l->curr_thread) to thread_id. Call thread_resume() to resume the thread. Otherwise, return and do nothing. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 // core/arch/riscv/kernel/thread_rv.S /* void thread_resume(struct thread_ctx_regs *regs) */ FUNC thread_resume , : /* Disable global interrupts first */ csrc CSR_XSTATUS, CSR_XSTATUS_IE /* Restore epc */ load_xregs a0, THREAD_CTX_REG_EPC, REG_T0 csrw CSR_XEPC, t0 /* Restore ie */ load_xregs a0, THREAD_CTX_REG_IE, REG_T0 csrw CSR_XIE, t0 /* Restore status */ load_xregs a0, THREAD_CTX_REG_STATUS, REG_T0 csrw CSR_XSTATUS, t0 /* Check if previous privilege mode by status.SPP */ b_if_prev_priv_is_u t0, 1f /* Set scratch as zero to indicate that we are in kernel mode */ csrw CSR_XSCRATCH, zero j 2f 1: /* Resume to U-mode, set scratch as tp to be used in the trap handler */ csrw CSR_XSCRATCH, tp 2: /* Restore all general-purpose registers */ load_xregs a0, THREAD_CTX_REG_RA, REG_RA, REG_TP load_xregs a0, THREAD_CTX_REG_T0, REG_T0, REG_T2 load_xregs a0, THREAD_CTX_REG_S0, REG_S0, REG_S1 load_xregs a0, THREAD_CTX_REG_S2, REG_S2, REG_S11 load_xregs a0, THREAD_CTX_REG_T3, REG_T3, REG_T6 load_xregs a0, THREAD_CTX_REG_A0, REG_A0, REG_A7 XRET END_FUNC thread_resume 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 // core/arch/riscv/kernel/thread_optee_abi_rv.S FUNC thread_std_abi_entry , : jal [__thread_std_abi_entry](/posts/optee-threads/) /* Save return value */ mv s0, a0 /* Disable all interrupts */ csrw CSR_XIE, x0 /* Switch to temporary stack */ jal thread_get_tmp_sp mv sp, a0 /* * We are returning from thread_alloc_and_run() * set thread state as free */ // Set: // current thread's state to THREAD_STATE_FREE. // l->curr_thread to THREAD_ID_INVALID. jal thread_state_free /* Restore __thread_std_abi_entry() return value */ mv a1, s0 li a2, 0 li a3, 0 li a4, 0 li a0, TEEABI_OPTEED_RETURN_CALL_DONE mv a5, zero /* Return to untrusted domain */ jal thread_return_to_udomain END_FUNC thread_std_abi_entry __thread_std_abi_entry() Call std_abi_entry() thread_handle_fast_abi() tee_entry_fast() tee_entry_fast() __tee_entry_fast() __tee_entry_fast() If args->a0: OPTEE_ABI_CALLS_COUNT: tee_entry_get_api_call_count() OPTEE_ABI_CALLS_UID: tee_entry_get_api_uuid() … thread_enter_user_mode() Disable all interrupts. Call xstatus_for_xret() to get xstatus with xstatus.PIE set to 1, xstatus.PP set to U-mode. The returned xstatus will be saved along with $a0, $a1, $a2, $a3, user_sp, entry_func, and $xie to current thread’s reg context (struct thread_ctx_regs). Call __thread_enter_user_mode() to switch to U-mode. entry_func is set to thread_ctx_regs.ra and then set to $xepc so that when xret is called to return to U-mode, entry_func will be executed. Re-enable original interrupts. thread_state_suspend() Current thread’s context (struct thread_ctx) will be updated with: flags |= THREAD_FLAGS_COPY_ARGS_ON_RETURN regs.status = xstatus to return regs.epc = [.thread_rpc_return](/posts/optee-rpc/#hl-0-84) (defined within thread_rpc_xstatus()) So when the thread is returned from RPC by thread_resume_from_rpc() , .thread_rpc_return will be called. state = THREAD_STATE_SUSPENDED Current thread ID (l->curr_thread) is set to THREAD_ID_INVALID to indicate no active current thread. thread_mask_exceptions() Mask the interrupts. thread_unmask_exceptions() Unmask the interrupts.

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca mpxy_opteed_channel_init() Check if MPXY extension is supported by OpenSBI. Extract MPXY channel ID from DT: compatible = “riscv,sbi-mpxy-opteed"; riscv,sbi-mpxy-channel-id ← Defines MPXY channel ID. Save MPXY channel ID to mpxy_opteed_ctx.channel_id. opensbi-domain-instance ← Defines the OpenSBI domain used by OP-TEE (not used by OP-TEE). 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 chosen { opensbi-domains { trusted-domain { compatible = "opensbi,domain,instance"; regions = <0x04 0x3f>; possible-harts = <0x03 0x01>; next-addr = <0x00 0xf1000000>; next-mode = <0x01>; phandle = <0x02>; }; }; sbi-mpxy-opteed { opensbi-domain-instance = <0x02>; riscv,sbi-mpxy-channel-id = <0x02>; compatible = "riscv,sbi-mpxy-opteed"; }; sbi_mpxy_setup_shmem() ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca entry_open_session() Open the session based on the UUID. tee_ta_open_session() tee_ta_open_session() tee_ta_init_session() Call ts_ctx->ops->enter_open_session() callback, For pseudo TAs, the callback is: pseudo_ta_enter_open_session() For user TAs, the callback is user_ta_enter_open_session() tee_ta_init_session() Look for already loaded TA tee_ta_init_session_with_context() If the TA for this UUID is not loaded yet: Look for secure partition stmm_init_session() Look for pseudo TA tee_ta_init_pseudo_ta_session() Look for user TA tee_ta_init_user_ta_session() If tee_ta_init_user_ta_session() returns TEE_SUCCESS, call tee_ta_complete_user_ta_session() entry_invoke_command() Call tee_ta_get_session() to get the opened session from arg->session. Call tee_ta_invoke_command() on the opened session. Call ts_ctx->ops->enter_invoke_cmd(): For pseudo TAs, the callback is: pseudo_ta_enter_invoke_cmd() For user TAs, the call back is: user_ta_enter_invoke_cmd()

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca tee_ta_init_pseudo_ta_session() Look up the pseudo TA based on UUID. Create pseudo TA context for the UUID, assign ctx->ts_ctx.ops to pseudo_ta_ops. pseudo_ta_ops will be the global callbacks for the pseudo TAs: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 // core/kernel/pseudo_ta.c TEE_Result tee_ta_init_pseudo_ta_session(const TEE_UUID *uuid, struct tee_ta_session *s) { ..... ctx->ref_count = 1; ctx->flags = ta->flags; stc->pseudo_ta = ta; ctx->ts_ctx.uuid = ta->uuid; ctx->ts_ctx.ops = &pseudo_ta_ops; ..... } 1 2 3 4 5 6 7 8 // core/kernel/pseudo_ta.c static const struct ts_ops pseudo_ta_ops = { .enter_open_session = pseudo_ta_enter_open_session, .enter_invoke_cmd = pseudo_ta_enter_invoke_cmd, .enter_close_session = pseudo_ta_enter_close_session, .destroy = pseudo_ta_destroy, }; pseudo_ta_enter_open_session() Call stc->pseudo_ta->open_session_entry_point() callback, if defined. E.g. If the opened session is for pseudo TA: rtc.pta, open_session_entry_point() callback is: open_session(): ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca tee_ta_init_user_ta_session() Initialize user TA for the UUID, assign ctx->ts_ctx.ops to user_ta_ops by calling set_ta_ctx_ops(). user_ta_ops will be the global callbacks for the user TAs; assign tee_ta_session->ts_sess.handle_scall to scall_handle_user_ta(). scall_handle_user_ta() will be the default callback to handle the syscall from user TA: 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 // core/kernel/user_ta.c TEE_Result tee_ta_init_user_ta_session(const TEE_UUID *uuid, struct tee_ta_session *s) { TEE_Result res = TEE_SUCCESS; struct user_ta_ctx *utc = NULL; ..... // Initialize lists. TAILQ_INIT(&utc->open_sessions); TAILQ_INIT(&utc->cryp_states); TAILQ_INIT(&utc->objects); TAILQ_INIT(&utc->storage_enums); condvar_init(&utc->ta_ctx.busy_cv); utc->ta_ctx.ref_count = 1; /* * Set context TA operation structure. It is required by generic * implementation to identify userland TA versus pseudo TA contexts. */ // utc->ta_ctx->ts_ctx.ops = [user_ta_ops](/posts/optee-user-tas/). set_ta_ctx_ops(&utc->ta_ctx); utc->ta_ctx.ts_ctx.uuid = *uuid; // vm_info_init() will set: utc->uctx->ts_ctx = &utc->ta_ctx.ts_ctx. res = vm_info_init(&utc->uctx, &utc->ta_ctx.ts_ctx); if (res) { condvar_destroy(&utc->ta_ctx.busy_cv); free_utc(utc); return res; } ..... utc->ta_ctx.is_initializing = true; ..... s->ts_sess.ctx = &utc->ta_ctx.ts_ctx; s->ts_sess.handle_scall = s->ts_sess.ctx->ops->handle_scall; /* * Another thread trying to load this same TA may need to wait * until this context is fully initialized. This is needed to * handle single instance TAs. */ TAILQ_INSERT_TAIL(&tee_ctxes, &utc->ta_ctx, link); return TEE_SUCCESS; } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 // core/kernel/user_ta.c /* * Note: this variable is weak just to ease breaking its dependency chain * when added to the unpaged area. */ const struct ts_ops user_ta_ops __weak __relrodata_unpaged("user_ta_ops") = { .enter_open_session = user_ta_enter_open_session, .enter_invoke_cmd = user_ta_enter_invoke_cmd, .enter_close_session = user_ta_enter_close_session, #if defined(CFG_TA_STATS) .dump_mem_stats = user_ta_enter_dump_memstats, #endif .dump_state = user_ta_dump_state, #ifdef CFG_FTRACE_SUPPORT .dump_ftrace = user_ta_dump_ftrace, #endif .release_state = user_ta_release_state, .destroy = user_ta_ctx_destroy, .get_instance_id = user_ta_get_instance_id, .handle_scall = scall_handle_user_ta, #ifdef CFG_TA_GPROF_SUPPORT .gprof_set_status = user_ta_gprof_set_status, #endif }; tee_ta_complete_user_ta_session() Call ldelf_load_ldelf() to load ldelf program to the memory for this User TA. ldelf is responsible for loading the user TA ELF image residing in REE to the memory. If ldelf_load_ldelf() returns TEE_SUCCESS, call ldelf_init_with_ldelf() ldelf_load_ldelf() loads user TA ELF and fill in user TA ELF’s information to struct user_mode_ctx. user_ta_enter_open_session() Call user_ta_enter() with function ID: UTEE_ENTRY_FUNC_OPEN_SESSION. user_ta_enter_invoke_cmd() Call user_ta_enter() with function ID: UTEE_ENTRY_FUNC_INVOKE_COMMAND. user_ta_enter() Call thread_enter_user_mode() to switch to U-mode. utc->utcx.entry_func (user TA’s entry function address, filled by ldelf) will be called after switching to U-mode. E.g. For optee_example_hello_world, i.e. 8aaaf200-2450-11e4-abe2-0002a5d5c51b.elf, the entry_func is 0x400405f8 => __ta_entry(). __ta_entry() is the first user TA API called from TEE core (defined in ta/user_ta_header.c). It’s assigned in TA’s Makefile: ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca Reference: Trusted Applications — OP-TEE documentation documentation ree_fs_ta_open() Call rpc_load() to request TA from tee-supplicant. Validate the loaded TA. rpc_load() Call thread_rpc_cmd() with OPTEE_RPC_CMD_LOAD_TA RPC command without struct thread_param_memref parameter to request the size of TA. OPTEE_RPC_CMD_LOAD_TA RPC command is saved to struct optee_msg_arg.cmd. struct optee_msg_arg is stored in the shared memory shared with the untrusted domain. Call thread_rpc_alloc_payload() to allocate data for TA. Call thread_rpc_alloc() to allocate shared memory for TA. Call thread_rpc() with rpc_args (rv[THREAD_RPC_NUM_ARGS]). rpc_args’s first element is set to OPTEE_ABI_RETURN_RPC_CMD function. The RPC command is set to OPTEE_RPC_CMD_SHM_ALLOC to allocate the shared memory for TA. Call thread_rpc_cmd() with OPTEE_RPC_CMD_LOAD_TA RPC command again with struct thread_param_memref parameter to load TA.

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca ldelf_hex.c is generated by scripts/gen_ldelf_hex.py from the input file: ldelf.elf and is included by OP-TEE, i.e. ldelf_data[], ldelf_code_size, ldelf_data_size, and ldelf_entry. ldelf sources: <optee-src>/ldelf/ ldelf is running in U-mode. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 // core/kernel/ldelf_loader.c /* * This function may leave a few mappings behind on error, but that's taken * care of by tee_ta_init_user_ta_session() since the entire context is * removed then. */ TEE_Result ldelf_load_ldelf(struct user_mode_ctx *uctx) { TEE_Result res = TEE_SUCCESS; vaddr_t stack_addr = 0; vaddr_t code_addr = 0; vaddr_t rw_addr = 0; vaddr_t bb_addr = 0; uint32_t prot = 0; uctx->is_32bit = is_32bit; // Allocate memory for bounce buffer. res = alloc_and_map_fobj(uctx, BOUNCE_BUFFER_SIZE, TEE_MATTR_PRW, 0, &bb_addr); if (res) return res; uctx->bbuf = (void *)bb_addr; uctx->bbuf_size = BOUNCE_BUFFER_SIZE; // Allocate stack memory for ldelf. res = alloc_and_map_fobj(uctx, LDELF_STACK_SIZE, TEE_MATTR_URW | TEE_MATTR_PRW, VM_FLAG_LDELF, &stack_addr); if (res) return res; uctx->ldelf_stack_ptr = stack_addr + LDELF_STACK_SIZE; // Allocate code section memory for ldelf. res = alloc_and_map_fobj(uctx, ldelf_code_size, TEE_MATTR_PRW, VM_FLAG_LDELF, &code_addr); if (res) return res; // Assign uctx->entry_func to ldelf's entry point, i.e. [_ldelf_start()](/posts/optee-ldelf/). uctx->entry_func = code_addr + ldelf_entry; // Allocate data section memory for ldelf. rw_addr = ROUNDUP(code_addr + ldelf_code_size, SMALL_PAGE_SIZE); res = alloc_and_map_fobj(uctx, ldelf_data_size, TEE_MATTR_URW | TEE_MATTR_PRW, VM_FLAG_LDELF, &rw_addr); if (res) return res; vm_set_ctx(uctx->ts_ctx); // Copy ldelf codes. // ldelf_data[] includes both ldelf's codes and data. // ldelf_data[] is generated by script/gen_ldelf_hex.py. memcpy((void *)code_addr, ldelf_data, ldelf_code_size); // Copy ldelf data. res = copy_to_user((void *)rw_addr, ldelf_data + ldelf_code_size, ldelf_data_size); if (res) return res; prot = TEE_MATTR_URX; if (IS_ENABLED(CFG_CORE_BTI)) prot |= TEE_MATTR_GUARDED; res = vm_set_prot(uctx, code_addr, ROUNDUP(ldelf_code_size, SMALL_PAGE_SIZE), prot); if (res) return res; DMSG("ldelf load address %#"PRIxVA, code_addr); return TEE_SUCCESS; } 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 // core/kernel/ldelf_loader.c TEE_Result ldelf_init_with_ldelf(struct ts_session *sess, struct user_mode_ctx *uctx) { TEE_Result res = TEE_SUCCESS; struct ldelf_arg *arg = NULL; uint32_t panic_code = 0; uint32_t panicked = 0; uaddr_t usr_stack = 0; struct ldelf_arg *arg_bbuf = NULL; // Set user stack. usr_stack = uctx->ldelf_stack_ptr; usr_stack -= ROUNDUP(sizeof(*arg), STACK_ALIGNMENT); arg = (struct ldelf_arg *)usr_stack; // Temporay overwrites handle_scall() with [scall_handle_ldelf()](/posts/optee-ldelf/). sess->handle_scall = scall_handle_ldelf; res = clear_user(arg, sizeof(*arg)); if (res) return res; // Copy UUID to &arg->uuid. res = PUT_USER_SCALAR(uctx->ts_ctx->uuid, &arg->uuid); if (res) return res; // uctx->entry_func is set to ldelf's entry point, i.e. [_ldelf_start()](/posts/optee-ldelf/), // in [ldelf_load_ldelf()](/posts/optee-ldelf/). // Switch to U-mode to execute ldelf. // ldelf will fill in arg parameter of the TA ELF. // E.g. arg->is_32bit, arg->entry_func, arg->load_addr, arg->stack_ptr... etc. res = [thread_enter_user_mode](/posts/optee-threads/)((vaddr_t)arg, 0, 0, 0, usr_stack, uctx->entry_func, is_32bit, &panicked, &panic_code); // Restore handle_scall(). sess->handle_scall = sess->ctx->ops->handle_scall; thread_user_clear_vfp(uctx); ldelf_sess_cleanup(sess); if (panicked) { abort_print_current_ts(); EMSG("ldelf panicked"); return TEE_ERROR_GENERIC; } if (res) { EMSG("ldelf failed with res: %#"PRIx32, res); return res; } res = BB_MEMDUP_USER(arg, sizeof(*arg), &arg_bbuf); if (res) return res; if (is_user_ta_ctx(uctx->ts_ctx)) { /* * This is already checked by the elf loader, but since it runs * in user mode we're not trusting it entirely. */ if (arg_bbuf->flags & ~TA_FLAGS_MASK) return TEE_ERROR_BAD_FORMAT; to_user_ta_ctx(uctx->ts_ctx)->ta_ctx.flags = arg_bbuf->flags; } // Copy TA ELF's information to struct user_mode_ctx. uctx->is_32bit = arg_bbuf->is_32bit; uctx->entry_func = arg_bbuf->entry_func; uctx->load_addr = arg_bbuf->load_addr; uctx->stack_ptr = arg_bbuf->stack_ptr; uctx->dump_entry_func = arg_bbuf->dump_entry; #ifdef CFG_FTRACE_SUPPORT uctx->ftrace_entry_func = arg_bbuf->ftrace_entry; sess->fbuf = arg_bbuf->fbuf; #endif uctx->dl_entry_func = arg_bbuf->dl_entry; bb_free(arg_bbuf, sizeof(*arg)); return TEE_SUCCESS; } scall_handle_ldelf() ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca thread_rpc_cmd() Call thread_rpc() with rpc_args (rv[THREAD_RPC_NUM_ARGS]). rpc_args’s first element is set to OPTEE_ABI_RETURN_RPC_CMD function. thread_rpc() Call __thread_rpc() __thread_rpc() Call xstatus_for_xret() to get xstatus with xstatus.PIE set to 0, xstatus.PP set to S-mode. The returned xstatus is passed to thread_rpc_xstatus(). Call thread_rpc_xstatus(). 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 // core/arch/riscv/kernel/thread_optee_abi_rv.S /* * void thread_rpc_xstatus(uint32_t rv[THREAD_RPC_NUM_ARGS], * unsigned long status); */ FUNC thread_rpc_xstatus , : // Allocate stack space for 8 registers. /* Use stack for temporary storage */ addi sp, sp, -REGOFF(8) /* Read xSTATUS */ csrr a2, CSR_XSTATUS /* Mask all maskable exceptions before switching to temporary stack */ csrw CSR_XIE, x0 /* Save return address xSTATUS and pointer to rv */ // $a0: rv[THREAD_RPC_NUM_ARGS] // $a1: xstatus to restore // $a2: current xstatus STR a0, REGOFF(0)(sp) STR a1, REGOFF(1)(sp) STR s0, REGOFF(2)(sp) STR ra, REGOFF(3)(sp) STR a2, REGOFF(4)(sp) #ifdef CFG_UNWIND addi s0, sp, REGOFF(8) #endif /* Save thread state */ jal thread_get_ctx_regs // $a0 = current thread's struct thread_ctx. // Restore $ra. LDR ra, REGOFF(3)(sp) /* Save ra, sp, gp, tp, and s0~s11 */ store_xregs a0, THREAD_CTX_REG_RA, REG_RA, REG_TP store_xregs a0, THREAD_CTX_REG_S0, REG_S0, REG_S1 store_xregs a0, THREAD_CTX_REG_S2, REG_S2, REG_S11 /* Get to tmp stack */ jal thread_get_tmp_sp // $a0 = tmp stack. /* Get pointer to rv */ LDR s1, REGOFF(0)(sp) // $s1 = rv[THREAD_RPC_NUM_ARGS] /* xSTATUS to restore */ LDR a1, REGOFF(1)(sp) // $a1 = xstatus to restore /* Switch to tmp stack */ mv sp, a0 /* Early load rv[] into s2-s4 */ lw s2, 0(s1) lw s3, 4(s1) lw s4, 8(s1) li a0, THREAD_FLAGS_COPY_ARGS_ON_RETURN la a2, .thread_rpc_return // We are about to return to untrusted domain for RPC, // suspend the current thread. jal [thread_state_suspend](/posts/optee-threads/) // $a0 = thread index before suspend. mv a4, a0 /* thread index */ mv a1, s2 /* rv[0] */ mv a2, s3 /* rv[1] */ mv a3, s4 /* rv[2] */ li a0, TEEABI_OPTEED_RETURN_CALL_DONE mv a5, zero /* Return to untrusted domain */ // $a0: TEEABI_OPTEED_RETURN_CALL_DONE // $a1: rv[0], e.g. OPTEE_ABI_RETURN_RPC_CMD // $a2: rv[1] // $a3: rv[2] // $a4: thread index before suspend. jal [thread_return_to_udomain](/posts/optee-threads/) .thread_rpc_return: /* * Jumps here from thread_resume() above when RPC has returned. * At this point has the stack pointer been restored to the value * stored in THREAD_CTX above. */ /* Get pointer to rv[] */ LDR a4, REGOFF(0)(sp) /* Store a0-a3 into rv[] */ sw a0, 0(a4) sw a1, 4(a4) sw a2, 8(a4) sw a3, 12(a4) /* Pop saved XSTATUS from stack */ LDR s0, REGOFF(4)(sp) csrw CSR_XSTATUS, s0 /* Pop s0 from stack */ LDR s0, REGOFF(2)(sp) addi sp, sp, REGOFF(8) ret END_FUNC thread_rpc_xstatus DECLARE_KEEP_PAGER thread_rpc_xstatus

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca std_abi_entry() If args->a0: OPTEE_ABI_CALL_WITH_ARG or OPTEE_ABI_CALL_WITH_RPC_ARG: std_entry_with_parg() call_entry_std() tee_entry_std() __tee_entry_std() OPTEE_ABI_CALL_WITH_REGD_ARG: std_entry_with_regd_arg() __tee_entry_std() Call thread_set_foreign_intr() to enable all foreign interrupts. If arg->cmd: OPTEE_MSG_CMD_OPEN_SESSION: entry_open_session() OPTEE_MSG_CMD_CLOSE_SESSION: entry_close_session() OPTEE_MSG_CMD_INVOKE_COMMAND: entry_invoke_command() OPTEE_MSG_CMD_CANCEL: entry_cancel() OPTEE_MSG_CMD_REGISTER_SHM: register_shm() OPTEE_MSG_CMD_UNREGISTER_SHM: unregister_shm() …

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca TEEC_InitializeContext() teec_open_dev() Open /dev/teeX device. TEEC_OpenSession() Call ioctl() with TEE_IOC_OPEN_SESSION command. This will eventually trap to Linux Kernel’s tee_ioctl(). TEEC_InvokeCommand() Call ioctl() with TEE_IOC_INVOKE command. The command ID for TA is passed through arg->func. TEEC_CloseSession() Call ioctl() with TEE_IOC_CLOSE_SESSION command. TEEC_FinalizeContext() Close /dev/teeX device.

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/optee_os/-/tree/dev-optee-mpxy Commit ID: 75df9ba41a404aec897399ead0ff0aebcbff48ca Base addresses: U-Boot SPL: 0x80000000 OpenSBI: 0x80100000 OP-TEE: 0xf1000000 U-Boot proper: 0x81200000 / Relocated to: 0x17f76f000, relocate offset = 0xfe558000 E.g. 0x81217b66 → 0x17f76fb66; 0x81232884 → 0x17f78a884 Linux Kernel: 0x80200000 / 0xffffffff80000000 1 2 3 (gdb) add-symbol-file u-boot-spl 0x80000000 (gdb) add-symbol-file fw_dynamic.elf 0x80100000 (gdb) add-symbol-file tee.elf 0xf1000000 1 2 # For the relocated U-Boot (gdb) add-symbol-file u-boot -o 0xfe558000 When traps, mscratch/sscratch: If 0: Trap from kernel. If !0: Trap from user. See thread_init_per_cpu(), thread_trap_vect(). thread_init_per_cpu() sets mscratch/sscratch to 0 to indicate that the trap is from kernel. Fast Calls and Yielding Calls Fast Calls execute atomic operations. The call appears to be atomic from the perspective of the calling PE, and returns when the requested operation has completed. Yielding Calls start operations that can be pre-empted by a Non-secure interrupt. The call can return before the requested operation has completed.

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/opensbi/-/tree/dev-optee-mpxy Commit ID: 7d4c90953afe3bd86f9e2501bd4c2501e8db1898 mpxy_opteed_init() Match compatible string: "riscv,sbi-mpxy-opteed". Allocate channel. opteed_domain_setup() Setup domain for OP-TEE dispatcher by looking up the domain specified by opensbi-domain-instance phandle in DTS. Assign the domain name to opteed_domain_name. Get channel ID from DTS property: riscv,sbi-mpxy-channel-id. Initialize channel: 1 2 3 4 5 6 // lib/utils/mpxy/fdt_mpxy_opteed.c channel->channel_id = channel_id; channel->send_message = mpxy_opteed_send_message; channel->attrs.msg_proto_id = SBI_MPXY_MSGPROTO_TEE_ID; channel->attrs.msg_data_maxlen = PAGE_SIZE; Register channel: sbi_mpxy_register_channel() ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/opensbi/-/tree/dev-optee-mpxy Commit ID: 7d4c90953afe3bd86f9e2501bd4c2501e8db1898 init_coldboot() sbi_mpxy_init() sbi_mpxy_init() Allocate struct mpxy_state for each domain. sbi_platform_mpxy_init() Call platform-defined mpxy_init(). e.g. For generic platform: fdt_mpxy_init(). fdt_mpxy_init() Iterate MPXY drivers in fdt_mpxy_drivers[]. MPXY drivers are generated at compile time: 1 2 3 4 5 6 7 8 9 10 11 // lib/utils/mpxy/objects.mk libsbiutils-objs-$(CONFIG_FDT_MPXY) += mpxy/fdt_mpxy.o libsbiutils-objs-$(CONFIG_FDT_MPXY) += mpxy/fdt_mpxy_drivers.o carray-fdt_mpxy_drivers-$(CONFIG_FDT_MPXY_RPMI_MBOX) += fdt_mpxy_rpmi_mbox libsbiutils-objs-$(CONFIG_FDT_MPXY_RPMI_MBOX) += mpxy/fdt_mpxy_rpmi_mbox.o carray-fdt_mpxy_drivers-$(CONFIG_FDT_MPXY_MM) += fdt_mpxy_mm libsbiutils-objs-$(CONFIG_FDT_MPXY_MM) += mpxy/fdt_mpxy_mm.o carray-fdt_mpxy_drivers-$(CONFIG_FDT_MPXY_OPTEED) += fdt_mpxy_opteed libsbiutils-objs-$(CONFIG_FDT_MPXY_OPTEED) += mpxy/fdt_mpxy_opteed.o 1 2 3 4 5 // lib/utils/mpxy/fdt_mpxy_drivers.carray HEADER: sbi_utils/mpxy/fdt_mpxy.h TYPE: struct fdt_mpxy NAME: fdt_mpxy_drivers 1 2 3 4 5 6 7 // include/sbi_utils/mpxy/fdt_mpxy.h struct fdt_mpxy { const struct fdt_match *match_table; int (*init)(void *fdt, int nodeoff, const struct fdt_match *match); void (*exit)(void); }; Call drv->init(). ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/linux/-/tree/dev-optee-mpxy Commit ID: df5dc01764820f113312f7a39f221b49985bbd7a Kernel provides a TEE bus infrastructure where a Trusted Application is represented as a device identified via Universally Unique Identifier (UUID) and client drivers register a table of supported device UUIDs. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 // include/linux/tee_core.h /** * struct tee_device - TEE Device representation * @name: name of device * @desc: description of device * @id: unique id of device * @flags: represented by TEE_DEVICE_FLAG_REGISTERED above * @dev: embedded basic device structure * @cdev: embedded cdev * @num_users: number of active users of this device * @c_no_user: completion used when unregistering the device * @mutex: mutex protecting @num_users and @idr * @idr: register of user space shared memory objects allocated or * registered on this device * @pool: shared memory pool */ struct tee_device { char name[TEE_MAX_DEV_NAME_LEN]; const struct tee_desc *desc; int id; unsigned int flags; struct device dev; struct cdev cdev; size_t num_users; struct completion c_no_users; struct mutex mutex; /* protects num_users and idr */ struct idr idr; struct tee_shm_pool *pool; }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 // include/linux/tee_core.h /** * struct tee_desc - Describes the TEE driver to the subsystem * @name: name of driver * @ops: driver operations vtable * @owner: module providing the driver * @flags: Extra properties of driver, defined by TEE_DESC_* below */ #define TEE_DESC_PRIVILEGED 0x1 struct tee_desc { const char *name; const struct tee_driver_ops *ops; struct module *owner; u32 flags; }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 // include/linux/tee_core.h /** * struct tee_driver_ops - driver operations vtable * @get_version: returns version of driver * @open: called when the device file is opened * @release: release this open file * @open_session: open a new session * @close_session: close a session * @system_session: declare session as a system session * @invoke_func: invoke a trusted function * @cancel_req: request cancel of an ongoing invoke or open * @supp_recv: called for supplicant to get a command * @supp_send: called for supplicant to send a response * @shm_register: register shared memory buffer in TEE * @shm_unregister: unregister shared memory buffer in TEE */ struct tee_driver_ops { void (*get_version)(struct tee_device *teedev, struct tee_ioctl_version_data *vers); int (*open)(struct tee_context *ctx); void (*release)(struct tee_context *ctx); int (*open_session)(struct tee_context *ctx, struct tee_ioctl_open_session_arg *arg, struct tee_param *param); int (*close_session)(struct tee_context *ctx, u32 session); int (*system_session)(struct tee_context *ctx, u32 session); int (*invoke_func)(struct tee_context *ctx, struct tee_ioctl_invoke_arg *arg, struct tee_param *param); int (*cancel_req)(struct tee_context *ctx, u32 cancel_id, u32 session); int (*supp_recv)(struct tee_context *ctx, u32 *func, u32 *num_params, struct tee_param *param); int (*supp_send)(struct tee_context *ctx, u32 ret, u32 num_params, struct tee_param *param); int (*shm_register)(struct tee_context *ctx, struct tee_shm *shm, struct page **pages, size_t num_pages, unsigned long start); int (*shm_unregister)(struct tee_context *ctx, struct tee_shm *shm); }; 1 2 3 4 5 6 7 8 9 10 11 // include/linux/tee_core.h /** * struct tee_shm_pool - shared memory pool * @ops: operations * @private_data: private data for the shared memory manager */ struct tee_shm_pool { const struct tee_shm_pool_ops *ops; void *private_data; }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 // include/linux/tee_core.h /** * struct tee_shm_pool_ops - shared memory pool operations * @alloc: called when allocating shared memory * @free: called when freeing shared memory * @destroy_pool: called when destroying the pool */ struct tee_shm_pool_ops { int (*alloc)(struct tee_shm_pool *pool, struct tee_shm *shm, size_t size, size_t align); void (*free)(struct tee_shm_pool *pool, struct tee_shm *shm); void (*destroy_pool)(struct tee_shm_pool *pool); }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 // include/linux/tee_drv.h /** * struct tee_context - driver specific context on file pointer data * @teedev: pointer to this drivers struct tee_device * @data: driver specific context data, managed by the driver * @refcount: reference counter for this structure * @releasing: flag that indicates if context is being released right now. * It is needed to break circular dependency on context during * shared memory release. * @supp_nowait: flag that indicates that requests in this context should not * wait for tee-supplicant daemon to be started if not present * and just return with an error code. It is needed for requests * that arises from TEE based kernel drivers that should be * non-blocking in nature. * @cap_memref_null: flag indicating if the TEE Client support shared * memory buffer with a NULL pointer. */ struct tee_context { struct tee_device *teedev; void *data; struct kref refcount; bool releasing; bool supp_nowait; bool cap_memref_null; }; do_initcalls() … tee_init() tee_init() ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/linux/-/tree/dev-optee-mpxy Commit ID: df5dc01764820f113312f7a39f221b49985bbd7a OP-TEE provides a pseudo Trusted Application (PTA): drivers/tee/optee/device.c in order to support device enumeration. In other words, OP-TEE driver invokes this application to retrieve a list of Trusted Applications which can be registered as devices on the TEE bus. 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 // drivers/tee/optee/optee_private.h /** * struct optee - main service struct * @supp_teedev: supplicant device * @teedev: client device * @ops: internal callbacks for different ways to reach secure * world * @ctx: driver internal TEE context * @smc: specific to SMC ABI * @ffa: specific to FF-A ABI * @call_queue: queue of threads waiting to call @invoke_fn * @notif: notification synchronization struct * @supp: supplicant synchronization struct for RPC to supplicant * @pool: shared memory pool * @rpc_param_count: If > 0 number of RPC parameters to make room for * @scan_bus_done flag if device registation was already done. * @scan_bus_work workq to scan optee bus and register optee drivers */ struct optee { struct tee_device *supp_teedev; struct tee_device *teedev; const struct optee_ops *ops; struct tee_context *ctx; union { struct optee_smc smc; struct optee_ffa ffa; }; struct optee_shm_arg_cache shm_arg_cache; struct optee_call_queue call_queue; struct optee_notif notif; struct optee_supp supp; struct tee_shm_pool *pool; unsigned int rpc_param_count; bool scan_bus_done; struct work_struct scan_bus_work; }; 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 // drivers/tee/optee/optee_private.h /** * struct optee_ops - OP-TEE driver internal operations * @do_call_with_arg: enters OP-TEE in secure world * @to_msg_param: converts from struct tee_param to OPTEE_MSG parameters * @from_msg_param: converts from OPTEE_MSG parameters to struct tee_param * * These OPs are only supposed to be used internally in the OP-TEE driver * as a way of abstracting the different methogs of entering OP-TEE in * secure world. */ struct optee_ops { int (*do_call_with_arg)(struct tee_context *ctx, struct tee_shm *shm_arg, u_int offs, bool system_thread); int (*to_msg_param)(struct optee *optee, struct optee_msg_param *msg_params, size_t num_params, const struct tee_param *params); int (*from_msg_param)(struct optee *optee, struct tee_param *params, size_t num_params, const struct optee_msg_param *msg_params); }; do_initcalls() … optee_core_init() optee_core_init() ...

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/linux/-/tree/dev-optee-mpxy Commit ID: df5dc01764820f113312f7a39f221b49985bbd7a TEE supplicant /etc/init.d/S30-tee-supplicant /dev/teepriv0 main() process_one_request() process_one_request() read_request() Issue TEE_IOC_SUPPL_RECV ioctl to receive the TEE supplicant request from OP-TEE. This will be blocked until TEE supplicant request is received. Spawn a new thread to process for the new request: thread_main() process_one_request() The original thread will continue to handle the TEE supplicant request from OP-TEE: If RPC command: OPTEE_MSG_RPC_CMD_LOAD_TA: Call load_ta() to load the TA according to the UUID. load_ta() will call TEECI_LoadSecureModule() to load the TA. TEECI_LoadSecureModule() will call fopen(), ftell() to open TA and get the size of TA. If the buffer size (ta_size) is not enough to hold the TA, return the required size to let the caller increase the buffer size and try again. Otherwise, call fread() to read TA and save it to the buffer. … Call write_response() to send the TEE supplicant response to OP-TEE. write_response() Issue TEE_IOC_SUPPL_SEND ioctl to send the TEE supplicant response to OP-TEE. This will unblock Call wait_for_completion_interruptible(**&req->c**) to wait for TEE supplicant to process.

⚠️ The code is based on: https://gitlab.com/riseproject/riscv-optee/linux/-/tree/dev-optee-mpxy Commit ID: df5dc01764820f113312f7a39f221b49985bbd7a 1 2 3 4 5 6 7 8 9 10 // arch/riscv/kernel/mpxy-sbi.c struct sbi_mpxy { void *shmem; phys_addr_t shmem_phys_addr; bool active; }; // Define per-cpu varible: sbi_mpxy. DEFINE_PER_CPU(struct sbi_mpxy, sbi_mpxy); do_initcalls() … sbi_mpxy_init() sbi_mpxy_init() Check SBI version and if MPXY extension is supported by OpenSBI. ...