DTB Woes

[khenriks]

Thanks for all your work on this! I've been trying to get your u-boot image to work for me for booting the mainline kernel from Debian (linux-image-4.14.0-0.bpo.3-arm64), and I've been having a lot of trouble, I think because of problems with DTB.

The DTB that you bundle with the image seems to actually work for me. That is, if I boot with booti $kernel_addr_r - $fdtcontroladdr everything seems to work well. However, I don't want to have to boot from the u-boot command line, and it seems like the best way to boot from an editable text file is with extlinux.conf. If fdt_addr were set to the value of fdtcontroladdr, then it would automatically load that one, but it's not and I don't know how to change the default environment built into the image. I also don't know how to get that DTB as a file; based on the commit message in ee12bea it looks like you applied a lot of custom patches to produce the image you provide.

I've also tried using other DTBs I've found, to no avail. CallMeFoxie provides a DTB and kernel as part of her image (https://github.com/CallMeFoxie/pine64-mainline-project), and they work together, but her DTB doesn't work with the Debian kernel. I've also tried using the DTB that Debian ships with their kernel package, as well as building the DTB myself from u-boot sources, and none of those work. Here, "doesn't work" means that after boot, either the Ethernet or the MMC (or both) don't work at all and don't show up.

I've tried following the instructions in your README, but that doesn't produce the same thing as the image you provide. If you can help me, I'd really appreciate it, as I need some help. An explanation of what might be going wrong with the DTBs would be great. A more comprehensive description of how to generate the image you provide would be great. The actual DTB as a file would be great. An explanation of how to change the default environment would be great. Really, any help you can provide would help tremendously.

[apritzel]

First of all, please have a look at the new image files I pushed, do they solve your problems?
I added the ATF and U-Boot branch names in case you want to rebuild something.

And you should try to boot using UEFI, to avoid many issues. For booting just the kernel, use bootefi $kernel_addr_r $fdtcontroladdr. Or you put your kernel into /efi/boot/bootaa64.efi and let U-Boot pick it up automatically. Even better, you could do like real computers and install grub-efi, that should bridge the gap between the bootloader and the kernel.

The DTB that you bundle with the image seems to actually work for me. That is, if I boot with booti $kernel_addr_r - $fdtcontroladdr everything seems to work well. However, I don't want to have to boot from the u-boot command line, and it seems like the best way to boot from an editable text file is with extlinux.conf.

If you desperately need to boot using booti, why not use boot.scr? That should be picked up automatically by the default U-Boot scripts.

If fdt_addr were set to the value of fdtcontroladdr,

Why fdt_addr? Where does that come from? It should at least be fdt_addr_r.
If you really need to set this in a particular image, you could try:
setenv fdt_addr_r $fdtcontroladdr; saveenv

then it would automatically load that one, but it's not and I don't know how to change the default environment built into the image. I also don't know how to get that DTB as a file; based on the commit message in ee12bea it looks like you applied a lot of custom patches to produce the image you provide.

If you run into trouble with the .dtb and want to experiment, your best bet is the .dtb from the kernel version you want to run. Second best choice is the .dtb in the latest mainline kernel.

[khenriks]

Thanks for your reply, and for your help with this.

I tried the new image, but it didn't fix anything that I was experiencing. I haven't tried the EFI boot though, that seems like a good possibility, and I can give grub-efi a try. I'd have to see if the kernel that Debian packages can be used for EFI boot.

If you desperately need to boot using booti, why not use boot.scr? That should be picked up automatically by the default U-Boot scripts.

I was kind of hoping to avoid needing a boot.scr, as that's not as easy to maintain as a plain text extlinux.conf file. I can do it if I have to, but hopefully it won't be necessary.

Why fdt_addr? Where does that come from?

That variable is documented at https://github.com/u-boot/u-boot/blob/master/doc/README.distro#L209. In short, it indicates an address to automatically load a device tree from when starting a kernel with extlinux.conf. Setting it with saveenv is not a good option for me because it doesn't scale to multiple boards: there are a couple variables (serial# and ethaddr) that get set automatically, but if they are loaded from an environment file they don't get recomputed. So setting fdt_addr on multiple boards using saveenv is a no-go.

I also tried using the .dtb from the kernel I'm running (I'm pretty sure that's the one Debian is shipping in /usr/lib/linux-image-4.14.0-0.bpo.3-arm64/allwinner/sun50i-a64-pine64-plus.dtb) but my MMC card is not recognized by the kernel when I use that.

Thank you very much for posting the full sources with the latest image; that's enough for me to build the image myself, so I should be able to retrieve the bundled .dtb that way, or probably even set fdt_addr in the default environment.

[apritzel]

Thanks for trying this!

I tried the new image, but it didn't fix anything that I was experiencing.

Can you please post exactly what you did and what went wrong? I don't care about random .dtbs not working, but the mainline kernel one, the one in my new images and the .dtb shipped with Debian (if they do) should work. Please post the kernel log somewhere, that should be enough to address your MMC issues (I guess it's due to a missing fixed.ko module). I prefer if you'd use the $fdtcontroladdr dtb from my new images first. Those are actually the same as the 4.15 kernel dtbs.
And please keep in mind that mainline 4.14 does not support Ethernet. The driver is technically in since 4.13, but it's disabled there and only gets enabled with 4.15-rc1. I see the Debian backported it to their 4.14 branch, but not sure that covers everything.

$fdt_addr

What I meant with $fdt_addr being wrong is that it should be fdt_addr_r, because it's used after relocation (compare kernel_addr_r). fdt_addr is used on platforms where the FDT is loaded before U-Boot. I understand that it clashes with the usage of $fdtcontroladdr somewhat.
In general those kind of issues are the reason why the normal U-Boot boot process just sucks if you want to use it in a generic way, and you should use UEFI instead.
And mind you: The UEFI support in the kernel is much better than you think, because all the serious ARM64 platforms (servers, networking appliances, ...) use it. So actually all bigger distributions first support UEFI enabled boot processes, then, maybe, some dodgy U-Boot platforms like the Pine64.
So with those images I posted you can use the generic Debian/Ubuntu/SuSE/Fedora arm64 UEFI .iso installer image (on some USB pen drive, for instance). This doesn't fully work till the end for various reasons, but it would be good to test this and report the issues encountered, so we can fix them.