Debian running natively atop Android, the Matrix Way.
Last year I picked up a decent mid-end and relatively “open” LG Optimus phone while I was in Florida. I rooted it, installed a custom ROM with the latest security patch level, hardened it in every feasible way, and kept it as FLOSS as possible. It worked out great for my needs and traveled with me quite a bit. A few weeks ago, I got a new OnePlus One and decided to turn the LG phone into a development device / playdough.
“Why don’t I turn this phone into a full-blown Linux server and run web services on it as if it were a Raspberry Pi?” The thought kept popping into my head. So I spent much of the first week of September investigating the possibilities.
There are numerous tutorials on how to run a Linux OS in a chroot environment on an Android device, and there are countless apps (e.g., Servers Ultimate, PAW Server) which allow you to run web servers (e.g., VNC, VPN, SIP, FTP, proxy) directly from an Android device. However, I wanted a real full-blown Linux server (e.g., Debian) running seamlessly on my Android phone, where I could access the Android OS from Debian without restrictions and at the same time make no modifications to the Android system itself.
I was thinking that perhaps I could implement a new init procedure mounting a new root file system at boot, which then transfers control to the Android <em class="md">init</em> in a chroot environment.
The advantages of this approach over the others:
Full Debian installation with lots of apt-get-able packages
Full control of the Android environment from Debian
Simultaneous use of Debian and Android
Access the Android file system from my desktop via SSH/SFTP
No need to unmount/remount the SD card; simply access it via SSH/SFTP
Easy to backup both the Android and the Debian systems
Android system remains untouched and unaware of any modifications
Android root file system is no longer volatile; edits are kept between reboots
Critical file systems could be kept on SD card for easy access in case of major mess-up
Graphic X11 user interface, on both client and server, local and remote, natively, over SSH or VNC
Zero performance impact
Easy to modify the Android ROM selectively, without the need to reflash the entire device
Manage the Android device just like any other Linux system
Replace the initramfs shipped with the Android device with your own modified. Then use an init to mount a new root file system from the SD card’s Linux partition and transfer control to this.
$ adb shell shell@w7:/ $ mount
.../dev/block/vold/public:179_65 on /mnt/media_rw/F409-DD80 type vfat (rw,dirsync,nosuid,nodev,noexec,relatime,uid=1023,gid=1023,fmask=0007,dmask=0007,allow_utime=0020,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)...
/dev/block/vold/public:179_66 on /mnt/media_rw/e0c17d6f-efcd-47eb-9f4e-bc5530f76269 type ext4 (rw,dirsync,context=u:object_r:sdcard_posix:s0,nosuid,nodev,noatime,data=ordered)
...shell@w7:/ $ ls -la /dev/block ...
brw------- 1 root root 179, 64 1970-02-14 03:33 mmcblk1
brw------- 1 root root 179, 65 1970-02-14 03:33 mmcblk1p1
brw------- 1 root root 179, 66 1970-02-14 03:33 mmcblk1p2
...
Here on the LG L90, the FAT partition /mnt/media_rw/F409-DD80 is device 179_65, so the next partition must be 179_66 and it’s named mmcblk1p2.
Here is an example of the /init in my new initramfs file system. It must be named /init because this is hard-coded into the Android kernel to execute on boot.
#!/sbin/busybox sh# initramfs pre-boot init script# Mount the /proc and /sys filesystems/sbin/busybox mount -t proc none /proc/sbin/busybox mount -t sysfs none /sys/sbin/busybox mount -t tmpfs none /dev# Something (what?) needs a few cycles here/sbin/busybox sleep 1# Populate /dev/sbin/busybox mdev -s# Mount the root filesystem, second partition on micro SDcard/sbin/busybox mount -t ext4 -o noatime,nodiratime,errors=panic /dev/mmcblk1p2 /mnt/root# Clean up/sbin/busybox umount /proc/sbin/busybox umount /sys/sbin/busybox umount /dev# Transfer root to SDcardexec /sbin/busybox switch_root /mnt/root /etc/init
This initramfs has a very minimalist filesystem and only contains the /sbin/busybox and the mount points /proc, /sys, /dev and /mnt/root. To be on the safe side, you can use the original initramfs and just add /sbin/busybox and a mount point /mnt/root, and replace init with the script above. You can download pre-compiled busybox here or elsewhere on the Internet.
We’ll need the system’s base address, i.e. where the RAM begins. To get it from your original kernel zImage, check for /proc/config.gz in your running kernel or use the extract-ikconfig script on the kernel binary. If neither exists / applies, try looking for “System RAM” in /proc/iomem on the Android device for a clue of the base address.
root@w7:/ # cat /proc/iomem 00000000-083fffff : System RAM
00008000-0108c71b : Kernel code
0120c000-014fd9eb : Kernel data
0c400000-0d1fffff : System RAM
0f500000-0f9fffff : System RAM
0ff00000-3f7fffff : System RAM
Here, the base address is 00000000.
Now, create the new boot image. I will get the original boot image for this phone and modify it. First, get the CM13 ROM for L90w7 and unzip the image. Then grab the unmkbootimg tool which helps you unpack the boot image.
Next, place unmkbootimg in the same directory as the unzipped CM13 image. To unpack the boot image:
./unmkbootimg boot.imgunmkbootimg version 1.2 - Mikael Q Kuisma <kuisma@ping.se>Kernel size 8019648Kernel address 0x8000Ramdisk size 872992Ramdisk address 0x1000000Secondary size 0Secondary address 0xf00000Kernel tags address 0x100Flash page size 2048Board name is ""Command line "console=ttyHSL0,115200,n8 androidboot.console=ttyHSL0 user_debug=31 msm_rtb.filter=0x37 androidboot.hardware=qcom androidboot.selinux=enforcing"This image is built using standard mkbootimgExtracting kernel to file zImage ...Extracting root filesystem to file initramfs.cpio.gz ...All done.---------------To recompile this image, use:mkbootimg --kernel zImage --ramdisk initramfs.cpio.gz --base 0x0 --cmdline 'console=ttyHSL0,115200,n8 androidboot.console=ttyHSL0 user_debug=31 msm_rtb.filter=0x37 androidboot.hardware=qcom androidboot.selinux=enforcing' -o new_boot.img---------------
Make a new initramfs directory:
$ mkdir initramfs && cd initramfs
Unpack the contents of the ramdisk (un-gzip it and then un-cpio it) into the new initramfs directory
$ gzip -cd ../initramfs.cpio.gz | cpio -i
This will place all of the files from the ramdisk in your current working directory. Now you can alter init as discussed above.
Re-create the ramdisk. Re-cpio and then re-gzip these files. Remember, cpio will include everything in the current working directory, so you probably want to remove any other cruft you might have in there.
Clean the directory so that only initramfs.cpio.gz and zImage remain. There’s no official tool to split the image, but it’s quite trivial and lots of scripts available to do this. The image is basically just a concatenation of the kernel zImage and initramfs.cpio.gz.
Combine the kernel and your new ramdisk into the full image, using the Android OS build kit mkbootimg which can be found pre-compiled at various site. Alternatively, you can compile it from source as follow:
Now copy system/core/mkbootimg/mkbootimg and system/core/cpio/mkbootfs to a directory in your path (e.g., ~/bin). After the compilation, you should see only 3 files in the current working directory:
As we are creating a Debian system for a different architecture than an x86 system in which debootstrap is run, the –arch armel argument is used to instruct debootstrap to create the Debian base system for the ARM architecture. –foreign instructs it to do initial unpacking only, and a second stage install will be done later on the actual hardware. jessie instructs it to download the packages to directory named “jessie” in the current directory from where debootstrap is run. And finally a repository url to fetch the packages from. You can use your local repository here, but make sure it has packages for the architecture armel. See man page of debootstrap if you need more information.
Once this is done, unmount the SD card and insert it back into the Android phone.
# umount /mnt/debian
Boot phone into Recovery Mode and start adb.
root@w7:/ # mount /dev/block/mmcblk1p2 /rootroot@w7:/ # export PATH=/usr/local/sbin:/usr/local/bin:/sbin:/usr/sbin:/bin:/usr/bin:/system/bin:$PATHroot@w7:/ # echo 'deb http://ftp.se.debian.org/debian jessie main contrib non-free' > /root/etc/apt/sources.listroot@w7:/ # for f in dev dev/pts proc sys ; do mount -o bind /$f /root/$f ; doneroot@w7:/ # export TMPDIR=/tmproot@w7:/ # export HOME=/rootroot@w7:/ # busybox chroot /root /bin/bash -lbash-4.3# debootstrap/debootstrap --second-stage...I: Base system installed successfully.
This takes a while to complete. Once it’s done, your Debian system is ready.
Here you’ve actually run Debian on your Android device! But it’s chrooted below Android, and we want the reverse. But now we got a complete Debian system with SSH server and all. Still some tinkering needs to be done. (If apt-get update didn’t work, check your /etc/named.conf.)
4. Create the new Android root file system
Mount the SD card on the desktop again. Unpack the original boot image initramfs to /android on the SD card’s Linux partition. This is the new Android root — in the Debian filesystem tree. Create directory called /android/log. Note that since the new Android root here isn’t a mount point but a subdirectory, Android will not succeed re-mounting it as read-only. If you believe this will cause a problem, you can instead create the Android root on a separate partition on the SD card and mount it as /mnt/root/android in the init on the initramfs above, right after mounting /mnt/root. However, in this case, /android/log is read-only and may not be used for boot logs by /etc/init below. You may solve this by mounting a tmpfs or simply remove logging by /etc/init.
Android normally only accepts 4 partitions on the SD card due to vold limitations. If you don’t want to waste one of them for the small root file system, you can loopback mount (using the –bind option) /android to /mnt/android to make it a mount point. You then can set this mount point to read-only using remount. Note that you must do a remount, because a bind-mount cannot change the flags of the original file system initially. You’ll have to do this remount explicitly yourself in init.stage2 using /bin/mount in this case. For now, just let the root be writable until you get everything up and running. This can be done later — or not at all.
5. Finish it up
The new initramfs transfer init control to /etc/init on the Linux partition.
A sample Debian boot init script looks like this:
#!/sbin/busybox sh## Debian environment boot init script## Leave all the initialization process to the Android init to handle## Launch delayed init script/etc/init.stage2 > /android/log/boot.log 2>&1 &# Transfer control to Android init - never returnsexec /sbin/busybox chroot /android /init
Make sure to copy busybox to /sbin. Note that log from init.stage2 is stored in the Android file tree so you can access it from Android in case the Debian-level SSH server doesn’t start due to some mistake in /etc/rc.local, for example.
Then create the second script called init.stage2 — a forking of a secondary delayed script the Debian environment executes once the Android init is done. It then transfers control to Android’s original init, still running as pid 1, of course.
#!/sbin/busybox sh## Delayed Debian environment boot init script# Not really init (not pid 1) but a fork of it.# The real init is right now executing in Android chroot#/sbin/busybox echo "`/sbin/busybox date` Debian init stage2 started"# Wait for Android init to set up everything# wait for dev to be mounted by Android init/sbin/busybox echo "`/sbin/busybox date` Waiting on Android to mount /dev"while [ ! -e /android/dev/.coldboot_done ]; do/sbin/busybox sleep 1done# wait for Android init to signal all done/sbin/busybox echo "`/sbin/busybox date` Waiting on Android init to finish"while [ -e /android/dev/.booting ]; do/sbin/busybox sleep 1done# Mount the /proc, /sys etc filesystems/sbin/busybox echo "`/sbin/busybox date` Mounting /proc /sys and /dev"/sbin/busybox mount -t proc none /proc/sbin/busybox mount -t sysfs none /sys# Mount /dev from the Android world/sbin/busybox mount -o bind /android/dev /dev/sbin/busybox mount -o bind /android/dev/pts /dev/pts/sbin/busybox mount -o bind /android/dev/socket /dev/socket# All done, now we can start running stuffexport PATH=/sbin:/usr/sbin:/bin:/usr/bin/sbin/busybox echo "`/sbin/busybox date` Running /etc/rc.local"# Start selected servers/etc/init.d/rc.local start/sbin/busybox echo "`/sbin/busybox date` All done"exit 0
Basically, this only waits on Android init and then sets up everything necessary for Debian such as dev, proc and sys mounts, and executes /etc/rc.local.
Because we mount /dev loopback from the Android root, we must remove any devices in /dev populated by debootstrap, or else this mount will fail.
My /etc/rc.local looks like this:
#!/bin/sh -e## rc.local## Executed at the end of each multiuser runlevel.# Make sure the script will "exit 0" on success or print# any other value on error.## To enable or disable this script, just change the execution# bits.## By default this script does nothing./etc/init.d/hostname.sh start/etc/init.d/ssh startexit 0
Note that init make sure everything here is logged to /android/log/boot.log. This is in the case the ssh-server does not start, you may see why in the file /log/boot.log by adb shell to Android.
6. Install the new boot image
If everything went well, it’s time to install the customized boot image. Here the LG L90 phone has an unlocked bootloader supporting fastboot.
Enter Fastboot Mode on the phone using a key combination (holding VolumeUp while the phone is off and connecting it via a USB wire to the desktop).
All done! You’re now running Debian integrated with Android the Matrix Way. ssh to it as user root with the password you specified.
After setup
Additional tinkering
<strong class="ln om">/etc/group</strong>
The Android environment is quite restricted. If you plan to run as non-root in the Debian environment, you’ll need to add yourself to some Android groups to get access to network and such. The groups of the Android user shell serves as a template. Most important are the inet group 3003 to get network access and 1015 to write to the SD card.
The complete set of Android user uid and group gid can be found online. Yes, it’s hard-coded.
<strong class="ln om">/etc/mtab</strong>
To make df happy, make a symlink to /proc/mounts:
ln -s /proc/mounts /etc/mtab
Still, df will produce a somewhat confusing output due to the double mounts of devices in the different roots. Don’t worry, this is only cosmetic.
<strong class="ln om">locales</strong>
You don’t get any localized locale installed by default. If you’d like that, run apt-get install locales, edit /etc/locale.gen to select what locale you want, then run locale-gen. Just like how you configure a new Debian system.
Setting the system default time zone
Simply run
# dpkg-reconfigure tzdata
Use the Debian environment
Connectivity
To get a Debian native terminal, download the ConnectBot app and ssh-connect to localhost.
Note that if you use the “local” connection in ConnectBot, you’ll enter The Matrix, i.e. the chroot Android environment, and can see no signs of the Debian environment whatsoever.
X11
If you want to run X11 on your device, apt-get install tightvncserver and get the free app VNC Viewer for Android.
First, install some desktop environment. LXDE is great for such a small system. On your Android device in the Debian system via SSH:
You should be able to view the Debian desktop environment using VNC Viewer for Android. For better ergonomics, ssh (optionally with X11 forwarding) from your desktop. Running the mouse pointer with the index finger over the touch screen can be somewhat challenging.
File access
With this setup, you can access the Android files without have to unmount/remount the SD card.
As a user in the sdcard_rw group, you have full access to the SD card. As the root user, you have access to all the files in the filesystem. This also makes backups easy (e.g., nightly automated backup via Duplicity running tar over ssh.)
The Android Media Scanner normally runs automatically each time Android remounts the SD card. Since you are now transferring the media to the SD card using SSH/SFTP instead of remounting the card, the media scanner won’t run except for boot time. Download an app to start the Media Scanner manually if needed — there are lots of them.
The Media Scanner is an index service used to catalog media files such as MP3s and images for Android apps. If you transfer a media file using SSH/SFTP but can’t find it in your app, initiate a Media Scan.
Modify the system
With the real Linux distribution on top, it’s trivial to customize the device to your likings. For example, you can simply move /data from the internal partition to a new partition on the SD card, edit Android’s init.rc (or init.vendor.rc) to mount the new one instead, and then restart. Now you have extra storage without having to re-flash the device or anything.
Root
With this setup, the Android environment is not “rooted”. This is trivial to achieve, but very much less needed, unless you have some app needing root privileges you still need to use.
To enter the Android Matrix from the Debian world, use chroot:
This is seldom needed, since you’ll perform all the work (e.g. edits) of the Android file system directly from the Debian environment, using the full set of tools Linux provides you.
Performance
Note that this is not emulation or virtualization but a runtime environment. Because of this, no performance penalty whatsoever occurs in either the Android or the Debian environment — not counting the extra 1.5 seconds to boot the device.
If you move partitions (e.g., /system and/or /data) to the SD card for safety or extra storage, the speed of the SD card may affect the performance. Benchmarks shows that a Class 10 SD card gives about the same I/O performance as the internal NAND disk, though. Don’t except more than 15–20 MB/s. USBs may offer you more.
Don’t expect laptop performance, though. If it’s primarily a GNU/Linux workstation you want, get an x86 based machine instead. The Android platform is design with resource conservation in mind, not high performance.
Caveats
Environment variables and file descriptors
When sshing into the device, remember that neither the SSH server nor your login shell is a child of Android’s init. Therefore you have no access to either file descriptors or environment variables created by init, especially not ANDROID_PROPERTY_WORKSPACE with corresponding file descriptor. Because of this, you can’t use getprop / setprop or any command relying on Android properties from the ssh session (e.g. restart adb). In order to accomplish this, you must enter the Android world via a child of init, e.g., adb or a local ssh-server in the Android root (e.g. dropbear).
You can always enter the Android world only to adb shell back to itself:
If the device is not rooted (ro.secure is 1), you end up as the shell user.
<strong class="ln om">apt-get upgrade</strong>
When you run apt-get upgrade, many installation scripts restart their corresponding daemons. Since no daemons except for those you explicitly start in /etc/rc.local are supposed to be running in the Debian environment, it might be a good idea to restart the system after the upgrade.
Backup
Although Debian is the root, both systems are heavily dependent on the Android system and its init, since it’s the “owner” of the hardware (i.e. runs init). If Android init fails, you will not be able to ssh into the Debian system. Even if the root is transferred to the SD card, the Android init mounts internal partitions, most importantly, /system . Certain changes you make to this partition might get yourself locked out. This can be solved by restoring a backup copy of the Android system to the SD card and editing /android/init.*.rc to not mount /system from internal flash but instead use the one you just restored to the SD card. Running /system from the SD card, to begin with, may be a good idea if you plan to change it frequently. This way the original system partition can be left untouched. This, of course, goes for all the Android partitions. For example, you can easily increase the space for your apps by moving /data to a much bigger partition on the SD card.
Always keep a backup
Alternatively, you can implement some fail-safe in the init scripts, e.g., populating /dev, mounting /proc, /sys, setting IP address, etc. if the Android init fails, but I find it more practical to just run /system from SD card instead.
The Android Java virtual machine (Dalvik), on the other hand, is quite non-critical at the OS level, so removing bundled apps (bloatware) on the /data partition is quite harmless. If you happen to remove the Home Application, you’ll still be able to ssh into the Debian system to restore it from your backup.
<strong class="ln om">pstree</strong>
A typical ps tree shows the following process hierarchy:
Note the sshd running this pstree and a sftp server in the Debian root. All the other processes are chrooted to the Android root.
Remember there is no connection between the process tree and the chroot filesystem structure. Here the init process lives in the chroot environment despite being top of the process tree, and the ssh-server and sshd reside in the genuine top root, although they are shown below init in the process tree.
This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish.AcceptRead More
Privacy & Cookies Policy
Privacy Overview
This website uses cookies to improve your experience while you navigate through the website. Out of these, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may affect your browsing experience.
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
Paolo Redaelli
