On mips64le, syscall.Getdents calls the old getdents syscall

[lechner]

From Debian packaging. On mips64le, golang developers provided the older getdents syscall instead of the newer getdents64. Gocryptfs (and also Go-Fuse) depends on the newer syscall and produces nonsensical output on mips64le.

The Dirent structure that ships with golang was not meant be be used with getdents but with glibc's readdir (at least on Linux, please see NOTES in manpage). By coincidence or by design, it works with the newer syscall and is often used that way, but it does not work as shipped in golang on mips64le. An important difference is the implementation of the Type member, which was declared explicitly for the newer call. In the older linux_dirent structure it is located at Reclen - 1. One could perhaps make a case that syscall.Getdirent should be renamed on mips64le, or that both calls should be provided.

While we could probably fix gocryptfs via build constraints, I think the issue should be addressed upstream. What do you think, please? Thank you!

[rfjakob]

Bear with me for a second, I have to dig a little to understand what is going on...

On mips64le, as you noted, getdents is used instead of getdents64: https://github.com/golang/go/blob/bcc86d5f42fddc2eca1bb066668388aa6807c38c/src/syscall/syscall_linux_mips64x.go#L13

	_SYS_getdents  = SYS_GETDENTS

And getdents returns linux_dirent, which defined in https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/fs/readdir.c?h=v4.15#n147 :

struct linux_dirent {
	unsigned long	d_ino;
	unsigned long	d_off;
	unsigned short	d_reclen;
	char		d_name[1];
};

In Go mips64le, syscall.Dirent is defined here https://github.com/golang/go/blob/bcc86d5f42fddc2eca1bb066668388aa6807c38c/src/syscall/ztypes_linux_mips64le.go#L133 :

type Dirent struct {
	Ino       uint64
	Off       int64
	Reclen    uint16
	Name      [256]int8
	Type      uint8
	Pad_cgo_0 [5]byte
}

And this is wrong, because it has a Type field, which the the linux_dirent struct has not. Looks like a bug in the Go stdlib.

[rfjakob]

Reported upstream as golang/go#23624 .

The fix for gocryptfs will be to switch to unix.Getdents from package golang.org/x/sys/unix, which uses getdents64: https://github.com/golang/sys/blob/3dbebcf8efb6a5011a60c2b4591c1022a759af8a/unix/zsyscall_linux_mips64le.go#L655 :

r0, _, e1 := Syscall(SYS_GETDENTS64, uintptr(fd), uintptr(_p0), uintptr(len(buf)))

[bradfitz]

@rfjakob, indeed. Please use golang.org/x/sys/unix instead of Go's built-in syscall package.

/cc @tklauser

[lechner]

Thanks for addressing this so quickly! Would you like to see a pull request for the switch to x/sys/unix?

[rfjakob]

Thanks for the offer, but I think I'll have time later today to do it myself

[rfjakob]

@lechner Pushed to master. I'll assume this fixes the problem on mips - please reopen if it does not!

[rfjakob]

For go-fuse, this usage of syscall.Dirent is problematic: https://github.com/hanwen/go-fuse/blob/6975cb38d30430450455caad8a62d5eba93e0e4e/fuse/test/xfs_test.go#L34
Do the go-fuse tests fail on mips?

But it looks like this should be switched to unix.Dirent as well. I'll send a PR.

[lechner]

First of all, gocryptfs now builds on mips64le. As you can see in the log, all tests pass. Thank you!

The issue you found in Go-Fuse may be limited to the tests, which do not presently run in Debian. Perhaps I should connect with Hanwen about that.

In an unrelated issue gccgo 1.8.3 on s390x refused to initialize a syscall.Timespec from a unix.Timespec. I thought maybe we can sidestep that unexpected error by switching to x/sys/unix altogether.

Thank you for all your hard work, and also for providing such high quality software!

[lechner]

Here are some reasons why golang.org/x/sys/unix is an attractive alternative. A more recent discussion can be found here. Thank you!