Non-root user cannot login (Linux kernel version 2.6.36.10) !!! SOLVED !!!

I noticed the problem first when I tried to test cgi-bin using mini-httpd on my Sheeva Plug. The server would not produce any output. I then cross compiled mini-httpd
and added logging. It would fork-execve the cgi script but get a SIGCHLD and no HTML output would be produced. Turns out that /etc/mini-httpd.conf wants to run
as user "nobody". When I changed this to "root" cgi-bin started working. I tried user "gopal" and this did not work. I checked permissions and things seemed
fine for non-root users. Yet the problem. I was intrigued.

I then tried logging in as user "gopal" using ssh. Sshd authenticates me fine, prints the 'motd' and then the shell gets killed. User "root" had no problems. I
thought it was sshd related and cross-compiled telnetd. Ran telnetd both as standalone and under inetd. Same result - when it did execve of the shell
it would get killed.

I decided to run telnetd under gdb. I cross-compiled gdb, installed gdbserver on the ARM box. I got rid of the fork() in telnetd. I saw that telnetd would go all
the way (open pty, authenticate user, setuid/setgid to user) and when it did execve() of "/bin/bash" it would just die. I ran telnetd under strace and noticed
that the last syscall was execve() and then it would print "killed". No shell.

I then decided to login as "gopal" from console. No luck. I googled this issue and things such as PAM related misconfiguration, /etc/nologins file, /etc/securetty
file etc. popped up. I decided to cross-compile the /bin/login program and make sure there was nothing wrong with credentials. Again my login program went all the
way up to execve of the shell and then would get a signal and terminate.

Suspecting bash, I even tried tcsh. That had the same problem too. It occured to me that root was able to do execve but non-root was not. So I decided to change the
login program to remain root until it did the execve but then do the setuid/setgid calls in the bash shell. To do this, I cross-compiled my own bash and added the
above calls to hardcode userid "gopal" in bash. Lo and behold my bash was spawned and it successfully changed it's UID/GID to "gopal". But bash was crippled. It could
not run any commands. Basically, once the process was non-root it was hosed.

I then decided to put printks in the do_execve() function in the kernel. I wrote a hello_world program, downloaded it into the box, and ran it as
"sudo -u gopal /tmp/hello". When I saw the printk's getting hit and the return value of do_execve() was -EPERM (-1), I knew that the end was in sight. It was just a matter
of finding out why the EPERM was happenning. Followed inside search_binary_handler() which interprets binary formats such as AOUT and ELF. elf_map() calls do_mmap() that
calls security_file_mmap(). Because I had not enabled CONFIG_SECURITY it called cap_file_mmap(). cap_file_mmap() checks if the address being mapped is below the
'dac_mmap_min_addr'. It does not allow non-root to mmap at this address. The kernel config CONFIG_DEFAULT_MMAP_MIN_ADDR was set to 4096. My printk showed that the
address being mapped was 32786. So why was I having trouble ? Turns out that the value of dac_mmap_min_addr can be set by sysctl "/proc/sys/vm/mmap_min_addr". Doing a
cat of this file showed it was set to 65536 ! Bingo ! I then did

# sysctl -w vm.mmap_min_addr=32768

Then did ssh to the box and Voila ! I was logged in as user "gopal" !!!

So who was changing the default setting from 4096 to 65536 ? A quick grep showed /etc/sysctl.d/10-process-security.conf was the culprit ! Changed it from 65536 to
32768. Rebooted the box and login for "gopal" was still working ! I've attached the contents of the sysctl file below so you can look at the comments:

# cat /etc/sysctl.d/10-process-security.conf

# protect bottom 64k of memory from mmap to prevent NULL-dereference
# attacks against potential future kernel security vulnerabilities.
# (Added in kernel 2.6.23.)
vm.mmap_min_addr = 32768

I think the Posix Discretionary Access Controls (DAC) were added in 2.6.23. I'm using 2.6.35.10. Did the helpful folks at Ubuntu who created the distribution check
this on different architectures and make sure it was compatible with the kernel config variable ? I'm running an Ubuntu jaunty Root FS that came with the box.

In any case, this was a great journey of discovery ! The thing with Linux is that it makes this process rewarding and enjoyable. I'd gladly miss work to debug this (which
I did). Thanks to this, I've validated my cross-compilation tools, gdb, and seen the insides of /bin/login, telnetd, and to some extent bash.