Android and OpenGL

If you're doing any drawing in Android, you're using OpenGL.  It might not be immediately obvious, but as far as I can tell, even when you're not using the OpenGL-specific API, the work of drawing goes through the OpenGL code.

It makes sense, really.  If you're going to include OpenGL as a core API of your system, might as well make all the drawing functions funnel through it.  It probably makes the underlying code simpler than having multiple channels to the hardware, and if implemented carefully should be reasonably fast.

The problem I'm about to describe isn't something that's a direct result of using OpenGL.  It's more of a mistake that could happen any time one leverages libraries to create other libraries.

I found this while optimizing some drawing code for Idamu Caverns.  I'd decided not to use any game development frameworks for the game, so the drawing code talks to the Android APIs directly.

To summarize the situation:  The code uses a dedicated rendering thread to render the map to a Bitmap, then displays the Bitmap on the screen in the main UI thread.  This ensures that the UI stays snappy, possibly at the loss of a few frames here and there if the rendering thread gets overly busy.  It also makes some operations simpler.  For example, the render thread always draws the entire map, even if the zoom factor will result in only a small portion of it being displayed, so the render code isn't cluttered with checks on the screen extents.  Additionally, the UI thread can zoom and reposition the map without waiting for the render thread to finish updating it.

Unfortunately, these advantages were also causing problems.  At high zoom levels, the rendered bitmap was huge, despite the fact that most of it would never be seen.  But the memory use wasn't the biggest problem; I worked around that by catching OutOfMemoryError during Bitmap allocation and reducing the zoom factor until allocation succeeded.

No.  The problem was that on some devices, at some zoom levels, the screen just disappeared.  No exceptions, no explanation, no crashes, just no display.

Logcat finally revealed the source of the problem.  It seems that Android's drawBitmap() method works by creating a OpenGL texture from the Bitmap, then issuing OpenGL commands to render the Bitmap to the screen (note that I haven't looked at the code, but you'll see why I'm making this claim shortly).

The Logcat message that I was seeing:

W/OpenGLRenderer﹕ Bitmap too large to be uploaded into a texture (4096x4096, max=2048x2048)

A little research explains that OpenGL keeps texture memory separate from other memory, and that there is a hard limit.  This is no surprise.  What is surprising (and frustrating) is that exceeding said limit simply results in a warning message and failure of the code to do anything.  In my mind, this really should generate an OutOfMemoryError so the situation is detectable by the code.  Furthermore, how much texture memory is available is different for every device and it's somewhat tricky to determine how much is available (it involves talking directly to the OpenGL API, which seems cumbersome, especially in code that doesn't use OpenGL directly.)

In the end, I solved the problem by breaking the map into smaller sections and rendering each one independently (think of how Google maps works when you zoom).  This results in improvements all around: sections that aren't on the screen are deallocated and not rendered, and the UI thread doesn't have to wait until the entire map is rendered to start displaying, to name a few.

But best of all, since each map section is small, even at high zoom levels, the size never exceeds the OpenGL texture limit (at least not on any hardware I can find).  So my mysterious blank screens are gone.

If you want to experiment with this a bit for yourself, the following code should give you a place to start playing:

package com.gamesbybill.drawexperiment;
// includes omitted for clarity
public class Display extends View implements Runnable {

  public Display(Context context, AttributeSet aSet) {
    super(context, aSet);
    Thread t = new Thread(this);
    t.start();
  }

  private static int size = 1;

  @Override
  protected void onDraw(Canvas c) {
    size *= 2;
    Bitmap b = Bitmap.createBitmap(
      size,
      size,
      Bitmap.Config.RGB_565
    );
    Canvas bmCanvas = new Canvas(b);
    bmCanvas.drawColor(Color.BLUE);
    c.drawColor(Color.LTGRAY);
    c.drawBitmap(b, 0, 0, null);
  }

  @Override
  public void run() {
    while (true) {
      postInvalidate();
      SystemClock.sleep(50);
    }
  }

}

Add the following to the XML file for your main activity to make it go:

<com.gamesbybill.drawexperiment.Display
        android:layout_width="match_parent"
        android:layout_height="match_parent"
        />

At some point the code will crash with an out of memory error, but look prior to that in logcat to see the OpenGL message that doesn't crash the app, but simply causes drawing to fail. The limits are different for every device, and once you know the limits for the device you're testing on, you can tweak the code to make the effect easier to see.