Using the Android NDK with Android Studio- Part 3

We conclude our series on integrating Android Studio with NDK projects with an example of wrapping the NDK process in an asynchronous task and pushing complex Java result objects back up to an Android Client using Messenger and Binder.

Using Binder and Messenger to create an asynchronous native process

So, after pulling in and building my native library with pre-built dependencies in part 2 of this series, I needed an architecture for reasonable asynchronous communication containing some java-defined data transfer objects. Android’s Messenger and Binder utilities are perfect for creating an asynchronous link between a native-backed service and an Android application.

Adding on to my previous Native class, I pushed the JNI call down a layer to an async task with callbacks like this:

package com.sdgsystems.examples.android.ndk.ndkmodule;

import android.os.AsyncTask;
import android.util.Log;

//These are my DTOs
import com.sdgsystems.examples.android.ndk.ndkmodule.ResultContainer;
import com.sdgsystems.examples.android.ndk.ndkmodule.ResultElement;

/**
 * Created by bfriedberg on 9/24/14.
 */
public class MainNative {

    private static final String TAG = "MainNative";

    private native ResultContainer processDataNative(byte[] data);

    // This interface defines the callbacks that need to be implemented by any calling classes
    interface NdkModuleCallback {
        void processingError();
        void resultsComplete(ResultContainer result);
    }

    NdkModuleCallback mResultsCallback;

    static {
        System.loadLibrary("NdkModule");
    }

    // Construct with a handle to the callback methods
    public MainNative(NdkModuleCallback callback) {
        mResultsCallback = callback;
    }

    public void processData(byte[] data) {
        Log.d(TAG, "Starting data processing task");

        //start a loop checking for the results
        new ProcessDataTask().execute(data);
    }

     private class ProcessDataTask extends AsyncTask<byte[], Integer, ResultContainer> {
         protected ResultContainer doInBackground(byte[]... data) {

            // This process could be broken into more native steps with 
     // progress updates along the way

            ResultContainer results = processDataNative(data[0]);
            return results;
         }

         protected void onProgressUpdate(Integer... progress) {
             //setProgressPercent(progress[0]);
         }

         protected void onPostExecute(ResultContainer results) {
             if(results != null) {
                 Log.d(TAG, "Data processing results complete, sending message to callback");
                 mResultsCallback.resultsComplete(results);
             } else {
                 Log.d(TAG, "There was a data processing error...");
                 mResultsCallback.processingError();
             }
         }
    }
}

My DTOs are simple but both need to implement serializable in order to attach them to a message:

public class ResultContainer implements Serializable {
    public ResultContainer (int resultCount) {
        resultElements = new ResultElement[resultCount];
    }

    public ResultElement[] resultElements;
}

public class ResultElement implements Serializable{
    public String elementName = "";
    public float count;
}

Processing the data in the native code was complicated to get working, but it pays off in having native code that returns data that I know what to do with…

main.h
#ifndef __MAIN_H__
#define __MAIN_H__

#include <jni.h>
#include <stdio.h>
#include <sys/ioctl.h>
#include <android/log.h>

#define LOG_TAG "NativeNdkModule"
#define LOGI(...) __android_log_print(ANDROID_LOG_INFO,LOG_TAG,__VA_ARGS__)
#define LOGE(...) __android_log_print(ANDROID_LOG_ERROR,LOG_TAG,__VA_ARGS__)
#define LOGD(...) __android_log_print(ANDROID_LOG_DEBUG,LOG_TAG,__VA_ARGS__)

#define CLEAR(x) memset(&(x), 0, sizeof(x))

#define ERROR_LOCAL -1
#define SUCCESS_LOCAL 0

int errnoexit(const char *s);
static int DEVICE_DESCRIPTOR = -1;

jobject Java_com_sdgsystems_examples_android_ndk_ndkModule_MainNative_processData(JNIEnv* env,
        jobject thiz, jbyteArray data);

#endif // __MAIN_H__

main.c
#include "main.h"

#include <errno.h>
#include <string.h>
#include <fcntl.h>
#include <malloc.h>
#include <unistd.h>

#include <string.h>

jobject Java_com_sdgsystems_examples_android_ndk_ndkModule_MainNative_processData(JNIEnv* env,
        jobject thiz, jbyteArray data) {
        ///copy the byte array to a local pointer
        jbyte* bufferPtr = (*env)->GetByteArrayElements(env, data, NULL);

        //And it is important to know the length of your array:
        jsize lengthOfArray = (*env)->GetArrayLength(env, spectrum);

        //Having jbyte* and length, you can do all the things in c-array. Finally, releasing it:
        (*env)->ReleaseByteArrayElements(env, spectrum, bufferPtr, JNI_ABORT);


	//NOTE: There is a java utility called javah that can give you the native 
//class signatures of your objects so that you don’t have to guess.  Nested
//objects and structures can be a real pain to figure out...

       //get the class / constructor / field ids for the element result object
       jclass resultElementClass = (*env)->FindClass(env, "com/sdgsystems/examples/android/ndk/ndkmodule/ResultElement");
       jmethodID resultElementClassConstructorMID = (*env)->GetMethodID(env, resultElementClass, "<init>", "()V");
       jfieldID resultElementElementName = (*env)->GetFieldID(env, resultElementClass, "elementName", "Ljava/lang/String;");
       jfieldID resultElementElementCount = (*env)->GetFieldID(env, resultElementClass, "count", "F");


       //get the class / constructor / field ids for the result object
       jclass resultClass = (*env)->FindClass(env, "com/sdgsystems/examples/android/ndk/ndkmodule/ResultContainer");
       jmethodID resultClassConstructorMID = (*env)->GetMethodID(env, resultClass, "<init>", "(I)V");
       jfieldID resultElementsField = (*env)->GetFieldID(env, resultClass, "resultElements", "[Lcom/sdgsystems/examples/ndk/ndkmodule/ResultElement;");




       //TODO implement processing calls to obtain actual results from the data byte array with bufferPtr and lengthOfArray

       //Until then, here is an example of how to put the data into the ResultContainer:

       /*************
       Test Mapping of data:
       **************/
       int numResultElements = 3;


       //instantiate the result object
       jobject result = (*env)->NewObject(env, resultClass, resultClassConstructorMID, numResultElements);

       //instantiate the result object element array
       jobjectArray resultElementsArray = (*env)->NewObjectArray(env, numResultElements, resultElementClass, NULL);

       for(int index = 0; index < numResultElements; index++) {
           jobject tmp = (*env)->NewObject(env, resultElementClass, resultElementClassConstructorMID, numResultElements);

           char elementName[5];
           sprintf(elementName, "test element %d", index );

           jfloat count = index + 2.5;

           jstring name = (*env)->NewStringUTF(env, elementName);
           LOGD("setting name to %s", elementName);

           //populate tmp
           (*env)->SetObjectField(env, tmp, resultElementElementName, name);
           (*env)->SetFloatField(env, tmp, resultElementElementCount, count);

           (*env)->SetObjectArrayElement(env, resultElementsArray, index, tmp);
           (*env)->DeleteLocalRef(env, tmp);
       }

       //Set the array into the result object
       (*env)->SetObjectField(env, result, resultElementsField ,resultElementsArray);
       /******************
       End Mapping Test
       *****************/

       return result;
}

jint JNI_OnLoad(JavaVM* vm, void* reserved) {
    //cache_yuv_lookup_table(YUV_TABLE);
    LOGI("Pulled in the native library");
    return JNI_VERSION_1_6;
}

Now that I have my native class and JNI hooks, I need to be able to put them behind a service so that they can be run with asynchronous results.

First, I defined a set of message types that I wanted to listen for:

public class NdkModuleMessages {
    public static final int MESSAGE_RESULT_ERROR = -1;
    public static final int MESSAGE_RESULT_OK = 0;
    public static final int MESSAGE_RESULT_SCAN_ELEMENT_LIST = 1;
}

Next, I defined a service that would be in charge of running the background task and channeling messages back up to a bound client:

package com.sdgsystems.examples.android.ndk.ndkmodule;


import android.app.Service;
import android.content.Intent;
import android.os.Binder;
import android.os.Bundle;
import android.os.IBinder;
import android.os.Message;
import android.os.Messenger;
import android.os.RemoteException;
import android.util.Log;

import com.sdgsystems.examples.android.ndk.ndkmodule.ResultContainer;
import com.sdgsystems.examples.android.ndk.ndkmodule.ResultElement;

import java.util.List;

public class NdkModuleService extends Service implements MainNative.ResultsCallback{
    private static final boolean DEBUG = true;
    private static String TAG = "NdkModuleService";

    private IBinder mBinder = new NdkModuleBinder();

    private Messenger clientMessenger;

    private MainNative nativeProcessor;

    //Callback methods, these get passed to the async task and executed on completion 
    @Override
    public void processingError() {
        sendElementProcessingErrorMessage();
    }
    @Override
    public void resultsComplete(ResultContainer result) {
        sendScanElementResultMessage(result);
    }
    //--------------------------------------------------------------------------------

    public class NdkModuleBinder extends Binder {
        public NdkModuleService getService() { return NdkModuleService.this; }

        //Let a client register a messenger w/ the service (we will call back to the messenger 
 //with results from the NdkModule Library
        public void registerMessenger(Messenger messenger) { clientMessenger = messenger; }
        public void processData (byte[] data) { nativeProcessor.processData(data); }
    }

    @Override
    public void onCreate() {
        super.onCreate();
        Log.i(TAG, "Service starting");

        nativeProcessor = new MainNative(this);
    }

    @Override
    public synchronized void onDestroy() {
        super.onDestroy();
        Log.i(TAG, "Service being destroyed");
    }

    @Override
    public IBinder onBind(Intent intent) {
        Log.i(TAG, "Service binding in response to " + intent);
        return mBinder;
    }

    //Send the client a set of elements and concentrations detected in a scan
    private void sendScanElementResultMessage(ResultContainer elements) {
        if(DEBUG) Log.d(TAG, "Received element result");
        if(clientMessenger != null) {
            Message elementListMessage = Message.obtain(null, NdkModuleMessages.MESSAGE_RESULT_SCAN_ELEMENT_LIST);

            Bundle bundle = new Bundle();

	     //We can add the elements object because it (and its children) implements
	     //Serializable
            bundle.putSerializable("result", elements);

	     //Add the bundle to the message, it will be handled by the clients message handler
            elementListMessage.setData(bundle);

            try {
                if(DEBUG) Log.d(TAG, "Sending element result message to client messenger");
                clientMessenger.send(elementListMessage);
            } catch (RemoteException e) {
                e.printStackTrace();
            }
        }
    }

    //Alert the client that there was an element processing error
    private void sendElementProcessingErrorMessage() {
        if(clientMessenger != null) {
            Message elementProcessingErrorMessage = Message.obtain(null, NdkModuleMessages.MESSAGE_RESULT_ERROR);

            try {
                clientMessenger.send(elementProcessingErrorMessage);
            } catch (RemoteException e) {
                e.printStackTrace();
            }
        }
    }
}

Next, the service needs to be declared both in the module’s AndroidManifest AND in the CALLING applications’ AndroidManifest

Local AndroidManifest.xml
…
        <service android:name=".NdkModuleService"/>

Remote AndroidManifest.xml

…
        <service android:name="com.sdgsystems.examples.android.ndk.ndkmodule.NdkModuleService" >
            <intent-filter>
                <action android:name="com.sdgsystems.examples.android.ndk.ndkmodule.action.BIND_DRIVER_SERVICE" />
            </intent-filter>
        </service>

Finally, I define an activity that binds to the remote service, starts the asynchronous task and listens for an incoming message with processed data results:

package com.sdgsystems.examples.android.ndk.ndkmodule.ui;

import android.app.Activity;
import android.content.ComponentName;
import android.content.Context;
import android.content.Intent;
import android.content.ServiceConnection;
import android.os.Bundle;
import android.os.Environment;
import android.os.Handler;
import android.os.IBinder;
import android.os.Message;
import android.os.Messenger;
import android.util.Log;
import android.view.Menu;
import android.view.MenuItem;
import android.view.View;
import android.widget.LinearLayout;
import android.widget.TextView;

import com.sdgsystems.examples.android.ndk.ndkmodule.R;
import com.sdgsystems.examples.android.ndk.ndkmodule.NdkModuleMessages;
import com.sdgsystems.examples.android.ndk.ndkmodule.NdkModuleService;
import com.sdgsystems.examples.android.ndk.ndkmodule.ResultContainer;
import com.sdgsystems.examples.android.ndk.ndkmodule.ResultElement;

import java.io.File;
import java.util.Arrays;

public class NdkModuleActivity extends Activity  {

    private static final String TAG = "NdkModuleActivity";

    private final Messenger mIncomingMessenger = new Messenger(new IncomingHandler());


    @Override
    protected void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        setContentView(R.layout.activity_ndkmodule);

	 //Bind to an instance of the remote service and create it if it hasn’t started
        bindService(new Intent(this, NdkModuleService.class),
                mConnection, Context.BIND_AUTO_CREATE);

        getActionBar().setDisplayHomeAsUpEnabled(true);
    }


    //Trash the connection
    @Override
    protected void onDestroy() {
        super.onDestroy();
        if(mConnection != null)
            unbindService(mConnection);
    }

    private NdkModuleService.NdkModuleBinder mBinder;

    //Define a connection object and handle the connectivity events
    private ServiceConnection mConnection = new ServiceConnection() {
        public void onServiceConnected(ComponentName className, IBinder service) {
            Log.i(TAG, "Bound to NdkModuleService");
            mBinder = (NdkModuleService.NdkModuleBinder) service;

	     //Call the registerMessenger method on the binder to send a handle 
	     //to this object’s incoming messenger handler for results
            mBinder.registerMessenger(mIncomingMessenger);
        }

        public void onServiceDisconnected(ComponentName className) {
            Log.w(TAG, "NdkModuleService disconnected unexpectedly");
            mBinder = null;
        }
    };

    //This is called when the button is clicked
    //The onClick is defined in the layout xml
    public void processData(View view) {
        //Clear the result set
        LinearLayout layout = (LinearLayout) findViewById(R.id.dataResults);
        layout.removeAllViews();

        if(mBinder != null) {
	     //Call the binder and fire off the async data processing task
            //TODO: Add some meaningful data...
            mBinder.processData(new byte[0]);
        }
    }

    //This handler will receive incoming messages from the service and process them
    private class IncomingHandler extends Handler {
        @Override
        public void handleMessage(Message msg) {
            ResultContainer data;

            Bundle bundle = msg.getData();
            data = (ResultContainer)(bundle.getSerializable("result"));

            LinearLayout layout = (LinearLayout) findViewById(R.id.dataResults);

            switch (msg.what) {
                case NdkModuleMessages.MESSAGE_RESULT_SCAN_ELEMENT_LIST:
                    if(DEBUG) Log.d(TAG, "Received data results");

                    if(data.resultElements != null) {
                        for (ResultElement element : data.resultElements) {
                            TextView tv_element = new TextView(NdkModuleActivity.this);

                            if(element != null) {
                                if (element.elementName != null) {
                                    tv_element.setText(element.elementName + ":" + element.count);
                                } else {
                                    tv_element.setText("empty name");
                                }
                            } else {
                                tv_element.setText("null element?");
                            }
                            layout.addView(tv_concetration);
                        }
                    } else {
                        TextView tv_element = new TextView(NdkModuleActivity.this);
                        tv_element.setText("Empty results");
                        layout.addView(tv_concentration);
                    }
                    break;
                case NdkModuleMessages.MESSAGE_RESULT_ERROR:
                    if(DEBUG) Log.d(TAG, "Received spectrum analysis error");
                    TextView tv_error = new TextView(NdkModuleActivity.this);
                    tv_error.setText("THERE WAS AN ERROR DURING PROCESSING");
                    layout.addView(tv_error);

                    break;
                case NdkModuleMessages.MESSAGE_RESULT_OK:
                    if(DEBUG) Log.d(TAG, "Received OK Message");
                    break;
                default:
                    super.handleMessage(msg);
                    break;
            }
        }
    }
}

For completeness’ sake, here is the layout file for that activity:

<LinearLayout xmlns:android="http://schemas.android.com/apk/res/android"
    xmlns:tools="http://schemas.android.com/tools"
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:paddingLeft="@dimen/activity_horizontal_margin"
    android:paddingRight="@dimen/activity_horizontal_margin"
    android:paddingTop="@dimen/activity_vertical_margin"
    android:paddingBottom="@dimen/activity_vertical_margin"
    tools:context="com.sdgsystems.examples.android.ndk.ndkmodule.ui.NdkModuleActivity"
    android:orientation="vertical"
    >

    <Button
        android:layout_width="wrap_content"
        android:layout_height="wrap_content"
        android:text="Test Process Data"
        android:id="@+id/button"
        android:onClick="processData"
        />

    <TextView
        android:layout_width="fill_parent"
        android:layout_height="wrap_content"
        android:text="Processing Results:"
        android:textAppearance="@android:style/TextAppearance.Holo.Large"
        android:layout_marginTop="20dp" />

    <ScrollView
        android:layout_width="fill_parent"
        android:layout_height="fill_parent"
        android:id="@+id/scrollView"
        android:layout_below="@+id/button"
        android:layout_marginTop="20dp"
        android:background="#ff89878b"
        android:padding="20dp">

        <LinearLayout
            android:id="@+id/dataResults"
            android:orientation="vertical"
            android:layout_width="fill_parent"
            android:layout_height="fill_parent" />
    </ScrollView>
</LinearLayout>

Thank you so much for taking the time to read through this guide. I hope that it provides a useful direction for you as you work on handling native processing in your Android Studio project. Questions and comments are always more than welcome.

Ben Friedberg, Lead Software Engineer