ppc64le_hello - ``Hello World'' for a Little-Endian OpenPower world
===================================================================
This is an example of running a 64-bit little-endian payload using
skiboot (https://github.com/open-power/skiboot) or another OPAL
firmware.
It shows the bare minimum needed to start writing
a little-endian kernel for the OpenPower ecosystem. Whereas
skiboot/test/hello_world is big-endian, this shows how
to make OPAL calls from LE as well as other minutiae like
setting up a basic C environment.
What you get:
- 64-bit real-mode HV LE operation
- logging via sim inteface (mambo_write)
- logging via OPAL firmware (opal_write)
- calling C code, stack/BSS/linkage setup/TOC
- calling BE code from LE
- FDT parsing, dumping FDT
- Taking and returning from exceptions, handling
unrecoverable/nested exceptions.
- Timebase (i.e. the "timestamp counter"), decrementer
and hypervisor decrementer manipulation with some basic timer
support (done for periodic callbacks into OPAL).
- Running at HV alias addresses (loaded at 0x00000000200XXXXX,
linked at 0x80000000200XXXXX). The idea being that the code
will access physical RAM and its own data structures solely
using the HV addresses.
- SLB setup: demonstrates 1T segments with 4K base page
and 16M base page size. One segment (slot = 0) is used
to back the HV alias addresses with 16M pages. Another
segment maps EA to VA 1:1 using 4K pages.
- Very basic HTAB setup. Mapping and unmapping for pages
in the 4K and 16M segments, supporting MPSS (16M pages
in the 4K segment). No secondary PTEG. No eviction support.
Not SMP safe. Any access within the HV alias addresses
get mapped in. Any faults to other unmapped locations are
crashes, as addresses below 0x8000000000000000 should only
be explicit maps.
- Taking exception vectors with MMU on at the alternate
vector location (AIL) 0xc000000000004000.
- Running unpriviledged code.
- Running VM supervisor real mode code with VRMA
Hopefully the code is clear enough and not too messy. I'm not
trying hard enough ;-(. I think the only code that has had any
thought put into it is the exception vectors, and other places
look especially crappy because I wanted to avoid writing an
actual kernel.
Building
========
You will need a LE 64-bit toolchain to build (i.e. ppc64le-linux).
A good source of toolchains is kernel.org. For building skiboot
(for testing) you will also need the BE 64-bit toolchain
(i.e. powerpc64-linux).
Skiboot is pulled-in as a submodule.
You will need the POWER8 Functional Simulator set up or Benjamin
Herrenschmidt's PowerNV QEMU tree.
Good directions for POWER8 Functional Simulator:
https://www.flamingspork.com/blog/2014/12/03/running-skiboot-opal-on-the-power8-simulator/
Good directions for QEMU/PowerNV:
https://www.flamingspork.com/blog/2015/08/28/running-opal-in-qemu-the-powernv-platform/
$ make
For QEMU or real hardware:
$ make CONFIG_NOSIM=1
Running
=======
$ make test
This will run the simulator and pop-up an xterm for OPAL console 0. If you
don't like xterm, feel free to change it in ppc64le_hello.tcl.
For QEMU/PowerNV, something like following should do. Remember that skiboot.lid must be in $(CWD).
$ cd skiboot; ~/src/qemu/ppc64-softmmu/qemu-system-ppc64 -m 4G -M powernv -kernel ../ppc64le_hello
You're looking for output like -
...
20295980: (20295980): [20289411000,5] INIT: Starting kernel at 0x20010000, fdt at 0x303039f0 (size 0x13b4)
20296509: (20296507): Hello Mambo!
20297324: (20297322): Hello OPAL!
20297989: (20297987): _start = 0x8000000020010000
20317249: (20316639): _bss = 0x800000002001A948
20336509: (20335291): _stack = 0x800000002001B000
20355769: (20353943): _end = 0x800000002001C000
20375038: (20372604): KPCR = 0x800000002001A9E8
20394308: (20391266): TOC = 0x8000000020022900
20413578: (20409928): OPAL = 0x30000000
20427317: (20423363): FDT = 0x303039F0
20538569: (20534311): SLB size = 0x20
20582102: (20577768): TB freq = 512000000
20600359: (20595683): [20595886000,3] OPAL: Trying a CPU re-init with flags: 0x2
20609301: (20604625): [20601599000,3] SLW: Not found on chip 0
20613538: (20608862): [20609554000,3] ELOG: Error getting buffer to log error
20614306: (20609630): OPAL claims no HILE supported, pretend to know better...
20651344: (20646668): Unrecoverable exception stack top @ 0x800000002001CEC8
20691374: (20686085): HTAB (2048 ptegs, mask 0x7FF, size 0x40000) @ 0x8000000020040000
20866415: (20860060): SLB entries:
20875356: (20868963): 0: E 0x8000000008000000 V 0x4000000001000500
20906511: (20898864): 1: E 0x8000000 V 0x4000000000000400
20931456: (20922897): 2: E 0xC000000008000000 V 0x4000000002000400
21151537: (21141251): Pick your poison:
21163581: (21153295): Choices: (MMU = disabled):
21178712: (21338426): (q) poweroff
21180946: (21170660): (d) 5s delay
21191199: (21180913): (D) toggle 5s timer
21205925: (21195639): (e) test exception
21220012: (21209726): (n) test nested exception
21238572: (21228286): (f) dump FDT
21248825: (21238539): (M) enable MMU
21260356: (21250070): (m) disable MMU
21272526: (21262240): (t) test MMU
21282779: (21272493): (T) test MMU 16mb pages
21300061: (21289775): (u) test non-priviledged code
21300061: (21299783): (U) test VM real-mode code
21321177: (21310891): (H) enable HV dec
21334625: (21324339): (h) disable HV dec
21338435: (21324339): (I) run initrd
You can interact with this:
---------------------------
(q) will shut down the sim using the attn instruction.
(d) poll the timebase registers until 5s of real time pass.
(D) toggles a periodic timer callback every 5s.
(e) tests triggering a system call with parameter 0xfeed. The system
call handler will return back parameter << 16 | 0xface, i.e. 0xfeedface.
(n) tests triggering an exception from an exception by triggering
a system call with parameter 0xdead. The exc.c logic keys off the parameter
to trigger another exception, expectedly crashing the system, thus testing
that the exc.c code can distinguish between recoverable and non-recoverable
exceptions and handle them appropriately.
(f) dumps the FDT passed up by skiboot. I don't try very hard. This code needs
improvement and its just glue around parts of libfdt.
(M) enables running with MSR.IR and MSR.DR. HTAB, memory between &_start
and &_end and the AIL (alternate) vectors are mapped. When MMU is on,
exception vectors are taken at EA 0xc000000000004000 with MMU on.
(m) disables MSR.IR and MSR.DR, disabling MMU. Exception vectors are taken with
MMU off at RA 0x0.
(t) simple test for MMU code, mapping the same EA to different RAs and verifying
that the 4K mapping worked.
(T) simple 16MB page size mapping test.
(u) shows running unpriviledged code. There's no scheduler, process/thread
support so the hand-off between code is messy. We enter user code via
syscall, saving off the exception frame for a subsequent return from user
code, again via syscall.
(U) similar to (u), but shows running code in non-HV supervisor mode in
real mode, by using the VRMA/VPM features of HV mode.
(H) enables HV decrementer. You will start seeing hypervisor decrementer
exceptions.
(h) disable HV decrementer. You will no longer see any HV decrementer
exceptions.
(I) execute LE binary (not ELF) code stored in initrd. The first uint64_t
may either be code itself, a pointer to code or a pointer to a
ABIv2 function descriptor.
You should be able to run ppc64le_hello on a real Power8 machine via skiboot,
but you will need to build with CONFIG_NOSIM=1. It has been successfully done
before, but that was before I wrote most of the MMU code ;-). See Daniel
Axtens' blog post:
http://sthbrx.github.io/blog/2015/06/03/ppc64le-hello-on-real-hardware/
Issues
======
Q: Delay test seems to run longer or shorter than 5s wall clock.
A: From benh: "The way mambo (aka systemsim) works, there is no expectation
that the simulated timebase has any relationship whatsoever
to wall clock outside of sim. All you can do really is play
with the ratio emulated CPU cycles / TB cycles."
...change processor/cpu_frequency and processor/timebase_frequency
inside ppc64le_hello.tcl. On my machine (i5-2410M) the TB should run
about 1000 times faster than clock.
Q: Something doesn't work on real hardware!
A: Please file an issue and let me know! Daniel Axtens identified a number
of issues which I believe to have resolved, but I only get to play in
a simulator.
Q: Something doesn't work on QEMU!
A: 16MB (MPSS) pages don't seem to work as of 01-Oct-2015.
TODO
====
- Maybe show more callthru sim facilities? Like file I/O...
- Spinlocks
- SMP
Resources
=========
Useful stuff:
- PowerISA_V2.07_PUBLIC.pdf - Book 3S
- ABI53BitOpenPOWER_21July2014_pub.pdf - ELFv2 ABI used for PPC64LE
- P8_um_external_v1.1_2015JAN29_pub.pdf - POWER8 User Manual
Contact Info
============
Andrei Warkentin (andrey.warkentin@gmail.com).
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.3