Building The Android Open Source Project – Part 4 – GIT Repositories

Before venturing any further into the land of source code changes, it is probably best to have somewhere to store your code and changes in an incremental fashion. This provides a lot of benefits and is always advisable when working with a large project like the AOSP but the main benefit for me is the ability to see which files you have edited at any given time and what changes you have made. Before I setup my own repositories I made a lot of little changes here and there and then completely forgot about what I had done a few days later.

So the two main (and free) GIT repository sites are github and Bitbucket. So go ahead and sign up for one of those; I use bitbucket as they allow you free private repos that nobody else can see and this is useful for some of my projects but the whole point of AOSP is that it’s open source so a public repo will do just fine.

Once you have signed up to one of those the process is pretty straight forward. The two sections of code we want to store for now are the frameworks/base structure and the device/htc/flounder (or whatever your specific device structure is).

Create a repo inside your account and all it ‘frameworks/base’ or something similar.

On both sites you will be given basic instructions on how to initialize and populate the GIT repo you have just created. All we want for now is the address of it I.e. https://github.com/username/test

Open up a terminal and go to frameworks/base

Run the commands below to set up your repo and push your code to the remote repository. This can take quite awhile the first time depending on your internet connection but it is completely worth it.

Note: if you don’t have Git installed you can do so by running the following from a terminal:

sudo apt-get update
sudo apt-get install git

Agree to any prompts that may appear.

git init
git checkout -b master
git add --all
git commit -m "initial commit with power menu changes"
git remote add origin [address of the repo you created]
git push -u master

‘git init’ initialises that directory and everything below it as a git repo.

‘git checkout -b master’ create a local branch called master and checks it out.

‘git add –all’ stages all the changes we have made. Gets them ready to commit.

‘git commit -m …” commits are changes to the branch with a message in between the double quotes.

‘git remote add…’ adds the remote repository that you created to this repos list of remote repositories.

‘git push -u master’ pushes all of our changes and everything under the frameworks/base directory to our remote git repository.

If you run ‘git remote -v’ you will see a list of remote repos that this local repo can communicate with. Currently you should see AOSP (the official repo that you can pull from) and origin which you have just created to represent your own remote repo.

Repeat the same steps for your device branch. These are the two most common places that you will be making changes for now.

From now on, any time you make changes below the frameworks/base folder for example, you can go to the frameworks/base folder and run:

cash status

to see what files you have changed. You can then see exactly what changes you have made by running:

git diff path/to/file/filename

Once you’re happy with your changes you can add everything to your commit by running:

git add --all

Or just add some files one-by-one with:

git add path/to/file/filename

Once you are ready to commit simply run:

git commit -m "I have changed....x..y..z"

And finally push it to your remote repo with:

git push origin
Advertisements

Building The Android Open Source Project – Part 3 – Flashing

Now that we’ve made some changes, it’d be nice to try them out on a real device. By the end of this tutorial you will have an AOSP rom running on your device and you’ll be able to see the changes made in part 2.

Quick note up front: The Nexus 9 (and possibly the Nexus 5X and Nexus 6P) have a ‘vendor’ partition. This makes life a lot easier and removes some steps involving proprietary ‘blobs’ (drivers) that you would need to carry out for devices such as the Nexus 4/5. If anyone would like a tutorial on this I can post one but as I am using the Nexus 9, the steps below will reflect this.

Another warning, AOSP ROMs do not contain Google apps and they must be manually flashed. I’ll upload a post about this very soon, but for now your rom will be pure, non-google-ised AOSP!

Tools & Setup

If you haven’t already got the fastboot and adb tools set up on your machine, you can follow the steps at: http://bernaerts.dyndns.org/linux/74-ubuntu/328-ubuntu-trusty-android-adb-fastboot-qtadb

Once that’s dealt with, create a directory somewhere convenient where we are going to store a script which will automate the flashing process for us.

I’ve tried flashing AOSP builds numerous different ways but have found the method I’m going to describe to be the most successful and stable for me.

Stock ROMs

As I’m using a Nexus device, I have access to the Google-made stock ROMs at this location: https://developers.google.com/android/nexus/images?hl=en

I am using the latest Marshmallow build ‘MRA58N’ so my download link is: https://dl.google.com/dl/android/aosp/volantis-mra58n-factory-80f36936.tgz

I make a folder in my documents, copy this tgz file to it and run the following command to untar it:

tar -zxvf filename

This will give you a bunch of files including a .zip file starting with ‘image’. We want to unzip that file too so run:

unzip imageFilename

Zip & Flash

What we want from the top level folder is the bootloader.img file, and from the image* folder we want the android-info.txt file. Copy both of these files into your ‘out’ folder, for me it’s at /AOSP/target/product/flounder.

The following script zip up all of the appropriate .img files, the android-info.txt and bootloader.img file that you copied over, wipe your device and then flash the build.

Warning: Just in case I haven’t been clear enough, this will wipe everything on your device!!! So backup anything that is important enough to you! I personally use my tablet for browsing, reading and media so I don’t have much to lose.

So, make a bash script file in your documents folder or somewhere else convenient. For example:

touch makeBuild.sh

Then open the file and paste in the following script, putting your paths and/or filenames in as required:

cd [Your 'out' folder]

#Zip them up into one file (ive chosen apathy.zip as my filename)
#Your device may not have vendor.img - The Nexus 9 does. Simply remove this if you don't need it.
zip [Your chosen filename].zip boot.img cache.img recovery.img system.img android-info.txt vendor.img

adb reboot bootloader
sleep 10

fastboot erase system
fastboot erase recovery
fastboot erase cache
fastboot erase boot
fastboot erase userdata

#Delete the line below if you don't have a vendor.img file
fastboot erase vendor
fastboot flash system system.img
fastboot flash recovery recovery.img
fastboot flash cache cache.img
fastboot flash boot boot.img

#Delete the line below if you don't have a vendor.img file
fastboot flash vendor vendor.img

#Reboot device to start it up - be patient!
fastboot reboot

Save this file somewhere convenient and not your ‘out’ folder.

Make the script executable

chmod +x yourScriptsFileName.sh

Warning: before running this script you need to enable some options in the ‘Developer Options’ menu.

To enable the developer options menu, go to Settings -> About (or about tablet) -> Tap the Build number entry until you see a message telling you that you’re a developer.

Once these are enabled, go back to settings and you will see an extra menu ‘developer options’. Tap it. Once inside, you want to enable both ‘USB Debugging’ and ‘OEM Unlocking’. The ‘OEM Unlocking’ is extremely important and if it is not enabled and left enabled there is a chance you could permanently brick your device. Without this option enabled you will not be able to flash anything.

With that said, plug your device into your computer and run:

adb devices

You should see output like the following:

alittlelost@alittlelost-samsung:~$ adb devices
List of devices attached
HT4CAJT02477    device

Now it’s time to run the script you made:

./scriptName.sh

Once this is finished your device will restart. The first start may take about 10-15 minutes to finish.

That’s it; You should now have your own AOSP rom running on your device. When it starts up you can hold in the power button to see the changes that were made in part 2:

part2changes

 

You will also see a warning when you startup your device like the below. This is nothing to worry about and I’ll show how to remove it in a future post.

 

manuWarning

Building The Android Open Source Project – Part 2 – Start Off Small

I want to make a change that is generic to all Android builds so we can ignore (for now) any device specific changes.

 

The change I want to make is to add extra options to the ‘Power Menu’. This is the menu that shows up when you hold down the power button on your device, usually to turn it off. On pure Android devices like the Nexus range, this menu only contains one option ‘Power off’. I want to add some extra options such as an option to reboot your device, toggle airplane mode and put your phone on silent. The functionality for for the airplane mode toggle and silent are already in the code, we’re simply going to expose them; The reboot option is something we will implement ourselves.

 

The change is pretty simple and is all under the folder frameworks/base which you’ll be spending a lot of time in. All paths mentioned below assume you are in this folder.

 

First things first, let’s configure our menu. Go to core/res/res/values/config.xml and find the string-array called ‘config_globalActionsList’. Here we are going to add our new options.

 

Add the following items to the string array:

 

<item>reboot</item>
<item>airplane</item>
<item>silent</item>

 

Next, open the file core/res/res/values/strings.xml and find the string named ‘global_action_power_off’. We’re going to add a new string below here called ‘global_action_reboot’. You’ll notice that global actions already exist for the other two functions if you look down the code a little from here. So add:

 

<string name="global_action_reboot">Reboot</string>

 

This is simply the string that will be shown in the ‘Power Menu’.

 

Now we need to update core/res/res/values/symbols.xml so that we can reference our newly created string from the framework code.

 

Find the ‘global_action_power_off’ java-symbol and add a similar line below it:

 

<java-symbol type="string" name="global_action_reboot"/>

 

Now for the last and most interesting step. We’re going to implement the functionality for our reboot option.

 

Open up services/core/java/com/android/server/policy/GlobalActions.java:

 

First we need to add a new import to make our changes compile successfully, so at the top with all the other imports, add:

 

import android.os.*;

 

Then head down the code to around line 85 and you will see a comment saying that we have some variables that are valid settings for the global actions key. Here is where we need to add a string object for our new reboot action. So below the GLOBAL_ACTION_KEY_POWER add:

 

private static final String GLOBAL_ACTION_KEY_REBOOT = "reboot";

 

Notice that this matches the string we defined back in our first step and the ‘r’ is not capatilized.

 

Further down the class around line 257 you will see an ArraySet being declared called ‘addedKeys’. Right below this there is a for loop which sets up handlers for each type of action declared in the defaultActions string array above. Here is where we’re going to tell the system to handle our new reboot action. So as usual, right below the if statement for GLOBAL_ACTION_KEY_POWER, we want to add an else if statement for our GLOBAL_ACTION_KEY_REBOOT action like so:

 

if (GLOBAL_ACTION_KEY_POWER.equals(actionKey)) {
    mItems.add(new PowerAction());
} else if (GLOBAL_ACTION_KEY_REBOOT.equals(actionKey)) {
    mItems.add(new RebootAction());
}

 

You’ll notice that we have instantiated a new instance of the RebootAction class which doesn’t exist yet; Our next and final step is to create this class.

 

Find the PowerAction class declaration around line 325 and make some space below it to declare our new class as follows:

 

private final class RebootAction extends SinglePressAction {

    private RebootAction(){
        super(R.drawable.ic_lock_power_off, R.string.global_action_power_off);
    }

    @Override
    public void onPress() {
        try {
            IPowerManager powerManager = IPowerManager.Stub.asInterface(ServiceManager.getService(Context.POWER_SERVICE));
            powerManager.reboot(false, null, false);
        }catch(RemoteException e){
            Log.e(TAG,&quot;PowerManager service failed! : &quot; +e);
        }
    }

    @Override
    public boolean showDuringKeyguard() {
        return true;
    }

    @Override
    public boolean showBeforeProvisioning() {
        return true;
    }
}

 

So in our constructor we are calling the SinglePressAction constructor that we inherit by extending the SinglePressAction class which gave us our onPress(), showDuringKeyguard() and showBeforeProvisioning() methods that we need to implement. For now we have gone with the default of return true for showDuringKeyguard() and showBeforeProvisioning(). The onPress() method is where the real work is done.

 

Using the PowerAction class as an example, we get a reference to the IPowerManager interface and call the reboot method. The reboot method is defined as follows:

 

void reboot(boolean confirm, String reason, boolean wait);

 

I personally don’t want a confirmation dialog to popup when I hit reboot, so I pass in false for the first parameter; I don’t really want to give a reason for why this is being called so I pass in null for the second parameter, and lastly if we pass true in for the ‘wait’ parameter, the call will wait for the operation to complete and not return, this doesn’t really matter either way to us so i’m just going to pass in false.

 

And that’s it. Go back to the top level directory and run your build command (for me it is ‘make -j5’).
N.B d3m0li5h3r rightly pointed out that you can reduce your build time here to minutes by going to frameworks/base and running ‘mm’ to build everything below that directory only. Then going to your root directory and running ‘make snod’ (system no update). This is a huge time saver so I highly recommend doing this.

 

This time around I have the benefit of the CCACHE we setup in part 1 and also the make tool is intelligent enough to not have to rebuild everything so it took around 30 mins (with the make -j5 method, the make snod method should take minutes) to build again with my updates.

Building The Android Open Source Project – Part 1

I wanted to document how I go about building AOSP because I regularly wipe my computer to install a new OS, try something out and then revert back to Ubuntu after awhile. Also, when I was doing this for the very first time I struggled to find good documentation on anything but the very basics.

 

N.B. In the past I built some ROMs for the Nexus 9 so that will be my goal here but the target device won’t have any bearing on the initial steps and only really comes into play in the final stages, so this should work for anyone.

 

First start by doing a clean install of Ubuntu 14.04.3 LTS downloaded from ‘here’ (if you don’t have it already). This is not the latest and greatest version of Ubuntu but the official AOSP documentation at least mentions Ubuntu 14.04 so it gives me a little more confidence that I won’t run into any trouble.

 

Once the OS up and running you need to install some pre-requisites:

 

Installing Java 7

Sadly Android is not ‘down with’ Java 8 yet. Load up a terminal by hitting the windows/start key, typing terminal and choosing it from the search results, or hit the alt+F2 keys together to start a new terminal.

 

Once it’s loaded up you should probably right click the icon in your launcher and choose to keep it there because you will be using this a lot.

 

In the terminal, run the following commands:
sudo apt-get update
sudo apt-get install openjdk-7-jdk

 

Installing Required Libraries

Once complete, you need to install a bunch of libraries by running the below command which you can copy and paste into your terminal (sometimes this will require ‘ctrl+shift+v’ rather than ‘ctrl+v’ to paste depending on your setup)

 

sudo apt-get install git-core gnupg flex bison gperf build-essential \ zip curl zlib1g-dev gcc-multilib g++-multilib libc6-dev-i386 \ lib32ncurses5-dev x11proto-core-dev libx11-dev lib32z-dev ccache \ libgl1-mesa-dev libxml2-utils xsltproc unzip

 

Due to limitations placed on regular users in Ubuntu around accessing USB devices directly, we need to create a rules file to allow us to do so by running this command, replacing ‘username’ with your actual username:
wget -S -O - http://source.android.com/source/51-android.rules | sed "s/<username>/$USER/" | sudo tee >/dev/null /etc/udev/rules.d/51-android.rules; sudo udevadm control --reload-rules

 

Directory Setup

Now that the pre-requisites are installed you can go ahead and pull the source down from the repositories. To do this you need a tool called ‘repo’ which is basically a nice wrapper and some extra functionality on top of Git. To get this set up you need to:
Create a directory to store the repo tool in your home (~) directory:
mkdir ~/bin
Add this new directory to your PATH so that when you run commands from the terminal, the system knows what you want it to do. The most common way of doing this is to add a command to your .bashrc file. This is a file that is read each time you open up a new terminal and in here you can setup aliases, run commands etc. Open the ~/.bashrc file with your editor of choice. If you’re comfortable with vim then use that ‘vim ~/.bashrc’ or simply run ‘gedit ~/.bashrc’ if you don’t want to install and learn to use vim. When the file is open, add the following line to the very bottom on a new line:

 

PATH=~/bin:$PATH
Save and close the file.

 

Download the repo tool using Curl:
curl https://storage.googleapis.com/git-repo-downloads/repo >  ~/bin/repo

 

Make sure the repo tool is executable:
chmod a+x ~/bin/repo

 

We now need to make a directory to hold all of the source code that we will pull down onto our computer using the repo tool. I have named mine ‘AOSP’ but feel free to call it whatever you wish; Just remember to replace any use of AOSP in the commands below with the name/path of your folder if you do choose a different name.

 

Make the directory:

mkdir ~/AOSP

– this will make the folder AOSP in your home directory.

Change directories so that you’re inside your new folder.
cd ~/AOSP

‘Pulling’ The Code

You now have a decision to make and it will depend on what device you are building for and what version of Android you would like to work with. I will be pulling the latest code for the Nexus 9 which at this time is Android Marshmallow – branch name = android-6.0.0_r2.

 

Check out this page to find out which version you should be using: https://source.android.com/source/build-numbers.html#source-code-tags-and-builds

 

The bit you need is under the ‘Branch’ column and will be used in the next command.
Let’s initialize our working directory (‘AOSP’ in my case) to the branch we want to download.
repo init -u https://android.googlesource.com/platform/manifest -b android-6.0.0_r2

 

Now you need to pull the source code down onto your local machine. When I ran this most recently I had a lot of odd SSL network errors that kept stopping my download/pull, so I suggest you run it with the -f flag as below so that the download will recover from any errors and continue on. Also, depending on what processor you have you can multi-thread this download to speed things up a little using the -j command followed by a number to indicate the amount of threads you wish to use. I will be using -j5.

 

Before we run our ‘sync’ command we need to tell the server who we are. To set our email address and name run the following:
git config --global user.name YourNameHere
git config --global user.email yourEmailAddressHere
Obviously replacing ‘YourNameHere’ with your name and ‘yourEmailAddresHere’ with your email address. You will not be able to sync the source code without doing this (or at least I couldn’t).

 

repo sync -f -j5
This may take quite awhile depending on your processing power and internet connection so be patient.

 

Once you have finished sync’ing you should set up what is called ‘ccache’. This will help speed up future builds (sadly not the first) so it’s very handy to have.

 

Edit your ~/.bashrc file again using your preferred method as mentioned earlier. Add a new line at the bottom of the file and type:
export CCACHE=1
Now load this bashrc into your current terminal:
. ~/.bashrc
Then set the cache size to 50GB by running:
prebuilts/misc/linux-x86/ccache/ccache -M 50G' from your working directory

 

Some last minute setup that you should always run before doing a build to tell the build system what target you want to build for i.e. what device, and also so you don’t spend a long time building for the wrong target, or a generic build which I have done accidentally many times.

 

Run:
build/envsetup.sh
Run
lunch
and choose your target by typing the number and hitting enter. For me it’s ‘aosp_flounder-userdebug’ as I am targeting the Nexus 9 and this is it’s code name, so I enter ’16’ and hit enter.

 

Let’s Finally Build The Code!!

Now you can finally start building the code. This again can take a long time but thankfully it will only do so the very first time or when you intentionally force a clean build.

The -j flag can be used here again to help speed things up but it depends on your processor, memory and hard drive type (more memory, a better processor and an SSD will help things immensely).
For me the command I run is:
make -j5

 

And now begins the long, long wait. You may need to leave it running overnight the first time. My first build took roughly 5Hrs & 36m.

 

When it eventually builds (and there should be no errors as you haven’t changed anything), your build .img files will be located at out/target/product/flounder (or whatever your product is named, out/target/product/shamu for Nexus 6, out/target/product/hammerhead for Nexus 5 etc.).