App Performance Scripts for Fire TV

Performance testing is the process of app testing in areas such as compatibility, reliability speed, response time, stability, and resource usage on Amazon Fire OS devices. You can use this testing to identify and address your app's performance bottlenecks. Performance testing involves gathering and evaluating key performance indicator (KPI) metrics. To gather KPI metrics, you run a specific set of steps on an Amazon device and then find or calculate the metrics using device resources, such as logs.

Make sure to run performance tests before submitting your app to Amazon Appstore. This page provides steps to test different categories of KPIs and includes example code that you can use in automation. This guide covers the following KPIs:

• Latency - time to first frame (TTFF)
• Ready-to-use - time to full display (TTFD)
• Memory after using the app's core functionality (for example, video streaming)

Setup

To get started, install the following software packages on your development computer:

• Amazon Corretto
• Android Studio (make sure to install platform tools during setup)
• Appium

In addition to the installing the software packages, you will need to:

• Set up a path for JAVA_HOME and ANDROID_HOME folders.
• Enable developer mode on the device and enable USB Debugging. For instructions, see Enable Debugging on Amazon Fire TV.
• Capture the serial number of the attached device. To list the serial numbers of physically attached devices, you can use the Android Debug Bridge (ADB) command adb devices -l .

Test strategy

During testing, you launch the app and force stop it several times by using the app launcher intent or the Monkey tool. Between each iteration, you must perform certain actions, such as capturing Atrace logs, performing navigation actions, capturing timer values from Atrace logs, and capturing memory and RAM usage before force stopping the app. This loop continues for the number of iterations you configure. As network conditions, system load, and other factors can impact the test results, use multiple iterations to average out the interference from external factors.

To calculate metric averages, Amazon recommends running a minimum number of iterations for the following test categories.

Devices to test:

• Fire OS 6: Fire TV Stick 4K (2018)
• Fire OS 7: Fire TV Stick with voice remote (2020)

You can use the logs, screenshots, and other artifacts captured from the performance tests for debugging or data use. The Appium device object is force stopped as part of tear-down.

The following sections contain code examples that you can add to your test automation script.

Get device type

The following example code shows how to get the device type of the connected device.

• Java
• Kotlin

public String get_device_type() {
  String deviceType = null;
  try (BufferedReader read = new BufferedReader(new InputStreamReader
                    (Runtime.getRuntime().exec
                    ("adb -s "+ DSN +" shell getprop ro.build.configuration")
                    .getInputStream()))) 
  {
            String outputLines = read.readLine();
            switch (outputLines) {
                case "tv":
                    deviceType = "FTV";
                    break;
                case "tablet":
                    deviceType = "Tablet";
                    break;
            }
  }
  catch (Exception e) {
     System.out.println("Exception while getting device type info: " + e);
  }
  return deviceType;
}

import java.io.BufferedReader
import java.io.InputStreamReader

fun getDeviceType(): String? {
    var deviceType: String? = null
    try {
        BufferedReader(InputStreamReader(Runtime.getRuntime().exec("adb -s $DSN shell getprop ro.build.configuration").inputStream)).use { read ->
            val outputLines = read.readLine()
            when (outputLines) {
                "tv" -> deviceType = "FTV"
                "tablet" -> deviceType = "Tablet"
            }
        }
    } catch (e: Exception) {
        println("Exception while getting device type info: $e")
    }
    return deviceType
}

Retrieve component name of the main launcher activity

The following example code shows how to retrieve the component name of the main launcher activity. This method fetches the main activity of the app under test and constructs the component name by combining the names of the app package and main activity.

• Java
• Kotlin

try (BufferedReader read = new BufferedReader(new InputStreamReader
                    (Runtime.getRuntime().exec("adb -s "+ DSN +" shell pm dump "+ appPackage +" | grep -A 1 MAIN").getInputStream()))) {
            String outputLine = null;
            String line;
            while ((line = read.readLine()) != null) {
                if (line.contains(appPackage + "/")) {
                    outputLine = line;
                    break;
                }
            }
            
            outputLine = outputLine.split("/")[1];
            String mainActivity = outputLine.split(" ")[0];
            String componentName = appPackage + "/" + mainActivity;
            return componentName;
}
catch (Exception e) {
        System.out.println("There was an exception while retrieving App Main Activity" + e);
}

import java.io.BufferedReader
import java.io.InputStreamReader

try {
    val process = Runtime.getRuntime().exec("adb -s $DSN shell pm dump $appPackage | grep -A 1 MAIN")
    val inputStream = process.inputStream
    val reader = BufferedReader(InputStreamReader(inputStream))
    var line: String? = null
    var outputLine: String? = null
    while (reader.readLine().also { line = it } != null) {
        if (line!!.contains("$appPackage/")) {
            outputLine = line
            break
        }
    }
    outputLine = outputLine!!.split("/")[1]
    val mainActivity = outputLine.split(" ")[0]
    val componentName = "$appPackage/$mainActivity"
    componentName
} catch (e: Exception) {
    println("There was an exception while retrieving App Main Activity: $e")
}

Launch an app using the component name of the main launcher activity

Use the following example code to launch an app using the component name of the main launcher activity. The code uses the componentName variable defined in the previous section, which creates the component name by combining app package and main activity.

• Java
• Kotlin

try (BufferedReader read = new BufferedReader(new InputStreamReader
                    (Runtime.getRuntime().exec("adb -s "+ DSN +" shell am start -n " + componentName).getInputStream()))) {
            String deviceName = getDeviceName(DSN);
            String line;
            while ((line = read.readLine()) != null) {
                if (line.startsWith("Starting: Intent")) {
                    System.out.println("App Launch successful using - " + componentName);
                    break;
                } else if (line.contains("Error")) {
                    System.out.println("App Launch Error");
                }
            }
        } catch (Exception e) {
            System.out.println("There was an exception while launching the app:" + e);
        }

import java.io.BufferedReader
import java.io.InputStreamReader

val process = Runtime.getRuntime().exec("adb -s $DSN shell am start -n $componentName")
val inputStream = process.inputStream
val reader = BufferedReader(InputStreamReader(inputStream))

try {
    val deviceName = getDeviceName(DSN)
    var line: String? = null
    while (reader.readLine().also { line = it } != null) {
        if (line!!.startsWith("Starting: Intent")) {
            println("App Launch successful using - $componentName")
            break
        } else if (line!!.contains("Error")) {
            println("App Launch Error")
        }
    }
} catch (e: Exception) {
    println("There was an exception while launching the app: $e")
}

Launch an app using the Monkey tool

The following code shows how to launch an app using the Monkey tool.

• Java
• Kotlin

try {
     String monkeyCommand = null;
     if (DEVICE_TYPE.equals(FTV)) {
          monkeyCommand = " shell monkey --pct-syskeys 0 -p "
     }
     else {
          monkeyCommand = " shell monkey -p "
     }
     
     BufferedReader launchRead = new BufferedReader(new InputStreamReader
            (Runtime.getRuntime().exec("adb -s "+ DSN + monkeyCommand + appPackage +" -c android.intent.category.LAUNCHER 1").getInputStream()));
      
     String line;
     while ((line = launchRead.readLine()) != null) {
         if (line.contains("Events injected")) {
             System.out.println("App Launch successful using Monkey Tool - " + appPackage);
             launchRead.close();
             return true;
         } 
         else if (line.contains("Error") || line.contains("No activities found")) {
             System.out.println("Error while launching app through Monkey Tool, using Intent to launch");
             launchRead.close();
             return false;
         }
     }
}
catch (Exception e) {
     System.out.println("There was an exception while launching the app using Monkey" + e);
     return false;
}  

try {
     val monkeyCommand: String?
     if (DEVICE_TYPE == FTV) {
          monkeyCommand = " shell monkey --pct-syskeys 0 -p "
     }
     else {
          monkeyCommand = " shell monkey -p "
     }
     val launchRead = BufferedReader(InputStreamReader(
            Runtime.getRuntime().exec("adb -s $DSN $monkeyCommand $appPackage -c android.intent.category.LAUNCHER 1").inputStream))
     var line: String?
     while (launchRead.readLine().also { line = it } != null) {
         if (line!!.contains("Events injected")) {
             println("App Launch successful using Monkey Tool - $appPackage")
             launchRead.close()
             return true
         } 
         else if (line!!.contains("Error") || line!!.contains("No activities found")) {
             println("Error while launching app through Monkey Tool, using Intent to launch")
             launchRead.close()
             return false
         }
     }
}
catch (e: Exception) {
     println("There was an exception while launching the app using Monkey $e")
     return false
}

Force stop an app

The following example code shows how to force stop an app.

• Java
• Kotlin

try {
       Runtime.getRuntime().exec("adb -s "+ DSN +" shell am force-stop " + appPackage);
       System.out.println("App force stopped - " + appPackage);
} 
catch (Exception e) {
       System.out.println("There was an exception in force stopping app" + e);
}

try {
    Runtime.getRuntime().exec("adb -s ${DSN} shell am force-stop ${appPackage}")
    println("App force stopped - $appPackage")
} catch (e: Exception) {
    println("There was an exception in force stopping the app: $e")
}

General commands

The following sections provide examples of commands you can use in performance testing.

Start Atrace capturing

The following example code shows how to start Atrace logs with an ADB command.

• Java
• Kotlin

public void startAtrace(String dsn) {
    log.info("Starting atrace collection");

    try {
        String command = String.format(
            "adb -s %s shell atrace -t 15 -b 32000 am wm gfx view input > /sdcard/atrace.log 2>&1 & echo $! > /sdcard/atrace.pid",
            dsn
        );

        Process process = Runtime.getRuntime().exec(new String[]{"bash", "-c", command});

        BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
        String line;
        while ((line = reader.readLine()) != null) {
            log.debug("Atrace Output: {}", line);
        }
        reader.close();

        log.info("Atrace started in background");

    } catch (Exception e) {
        log.error("Error starting atrace: ", e);
        throw new RuntimeException("Failed to start atrace", e);
    }
}

fun startAtrace(dsn: String) {
    val command = "adb -s $dsn shell atrace -t 15 -b 32000 am wm gfx view input > /sdcard/atrace.log 2>&1 & echo \$! > /sdcard/atrace.pid"
    try {
        val process = ProcessBuilder("bash", "-c", command).start()
        process.inputStream.bufferedReader().useLines { lines ->
            lines.forEach { log.debug("Atrace Output: $it") }
        }
        log.info("Atrace started in background")
    } catch (e: Exception) {
        log.error("Error starting atrace", e)
        throw RuntimeException("Failed to start atrace", e)
    }
}

Clear Atrace logs

You can clear the Atrace logs to start a fresh session. The following example code shows how to use an ADB command to clear the trace logs.

• Java
• Kotlin

public void clearTrace() throws IOException, InterruptedException {
    log.info("Clearing existing trace data");

    try {
        String deleteLogCmd = String.format("adb -s %s shell rm -rf /sdcard/atrace.log", dsn);
        String deletePidCmd = String.format("adb -s %s shell rm -rf /sdcard/atrace.pid", dsn);
        String clearTraceBuffer = String.format("adb -s %s shell 'echo 0 > /sys/kernel/debug/tracing/trace'", dsn);
        String disableTracing = String.format("adb -s %s shell 'echo 0 > /sys/kernel/debug/tracing/tracing_on'", dsn);

        Process logProcess = Runtime.getRuntime().exec(new String[]{"bash", "-c", deleteLogCmd});
        Process pidProcess = Runtime.getRuntime().exec(new String[]{"bash", "-c", deletePidCmd});
        Process clearTraceBufferProcess = Runtime.getRuntime().exec(new String[]{"bash", "-c", clearTraceBuffer});
        Process disableTracingProcess = Runtime.getRuntime().exec(new String[]{"bash", "-c", disableTracing});

        int logExitCode = logProcess.waitFor();
        int pidExitCode = pidProcess.waitFor();
        int clearTraceBufferExitCode = clearTraceBufferProcess.waitFor();
        int disableTracingExitCode = disableTracingProcess.waitFor();
        

        if (logExitCode != 0 || pidExitCode != 0 || clearTraceBufferExitCode!=0 || disableTracingExitCode!=0) {
            throw new IOException("Failed to clear one or more trace files. Exit codes: log=" + logExitCode + ", pid=" + pidExitCode);
        }

        log.info("Trace data cleared successfully");
    } catch (Exception e) {
        log.error("Error clearing trace: ", e);
        throw e;
    }
}

@Throws(IOException::class, InterruptedException::class)
fun clearTrace(dsn: String) {
    log.info("Clearing existing trace data")

    try {
        val logCmd = "adb -s $dsn shell rm -rf /sdcard/atrace.log"
        val pidCmd = "adb -s $dsn shell rm -rf /sdcard/atrace.pid"
        val clearTraceBuffer = "adb -s $dsn shell 'echo 0 > /sys/kernel/debug/tracing/trace'"
        val disableTracing = "adb -s $dsn shell 'echo 0 > /sys/kernel/debug/tracing/tracing_on'"

        val logProcess = ProcessBuilder("bash", "-c", logCmd).start()
        val pidProcess = ProcessBuilder("bash", "-c", pidCmd).start()
        val clearTraceBufferProcess = ProcessBuilder("bash", "-c", clearTraceBuffer).start()
        val disableTracingProcess = ProcessBuilder("bash", "-c", disableTracing).start()

        val logExit = logProcess.waitFor()
        val pidExit = pidProcess.waitFor()
        val clearTraceBufferExit = clearTraceBufferProcess.waitFor()
        val disableTracingExit = disableTracingProcess.waitFor()

        if (logExit != 0 || pidExit != 0 || clearTraceBufferExit != 0 || disableTracingExit != 0) {
            throw IOException("Failed to clear trace files: log=$logExit, pid=$pidExit, clearBuffer=$clearTraceBufferExit, disableTracing=$disableTracingExit")
        }
        log.info("Trace data cleared successfully")
    } catch (e: Exception) {
        log.error("Error clearing trace", e)
        throw e
    }
}

Pull Atrace Logs

To save the output of the Atrace logs, use the following example code. The code shows how to pull Atrace logs using an ADB command.

• Java
• Kotlin

public String pullAtraceLog(int iteration, LaunchType launchType) throws IOException, InterruptedException {
    String timestamp = DateTimeFormatter.ofPattern("yyyyMMdd_HHmmss")
            .format(LocalDateTime.now());

    String fileName = String.format("systrace_%s_launch_iter%d_%s_%s.txt",
            launchType.toString().toLowerCase(),
            iteration,
            dsn,
            timestamp
    );

    Path currentPath = Paths.get(System.getProperty("user.dir"));
    Path baseDir = currentPath.resolve("configuration/output/logs/systrace");
    Path outputDir = baseDir.resolve(launchType.toString().toLowerCase());
    Path outputPath = outputDir.resolve(fileName);

    log.info("Working directory: {}", currentPath);
    log.info("Base directory: {}", baseDir);
    log.info("Systrace file path: {}", outputPath);

    Thread.sleep(5000);

    try {
        Files.createDirectories(outputDir);

        String pullCommand = String.format("adb -s %s pull /sdcard/atrace.log %s", dsn, outputPath);
        log.info("Executing pull command: {}", pullCommand);

        Process process = Runtime.getRuntime().exec(new String[]{"bash", "-c", pullCommand});

        BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()));
        String line;
        while ((line = reader.readLine()) != null) {
            log.debug("Pull Output: {}", line);
        }
        reader.close();

        int exitCode = process.waitFor();
        if (exitCode != 0) {
            throw new IOException("ADB pull failed with exit code: " + exitCode);
        }

        log.info("Pulled {} launch atrace log for device {} to: {}", launchType, dsn, outputPath);

        // Wait for command buffer flush (simulating wait after pull)
        Thread.sleep(30000);
        log.info("Waiting for log to be pulled for 30 seconds");

        // Clean up device trace files
        clearTrace();

        return outputPath.toString();

    } catch (Exception e) {
        log.error("Error saving trace file for {} launch on device {}: ", launchType, dsn, e);
        throw new IOException("Failed to save trace file", e);
    }
}

fun pullAtraceLog(dsn: String, iteration: Int, launchType: String): String {
    val timestamp = LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyyMMdd_HHmmss"))
    val fileName = "systrace_${launchType.lowercase()}_launch_iter${iteration}_${dsn}_$timestamp.txt"
    val outputDir = Paths.get(System.getProperty("user.dir"), "configuration/output/logs/systrace", launchType.lowercase())
    val outputPath = outputDir.resolve(fileName)

    Thread.sleep(5000)
    try {
        Files.createDirectories(outputDir)
        val pullCmd = "adb -s $dsn pull /sdcard/atrace.log $outputPath"
        val process = ProcessBuilder("bash", "-c", pullCmd).start()

        process.inputStream.bufferedReader().useLines { lines ->
            lines.forEach { log.debug("Pull Output: $it") }
        }

        if (process.waitFor() != 0) {
            throw IOException("ADB pull failed")
        }

        Thread.sleep(30000)
        clearTrace(dsn)
        return outputPath.toString()
    } catch (e: Exception) {
        log.error("Error saving trace file for $launchType launch on device $dsn", e)
        throw IOException("Failed to save trace file", e)
    }
}

Calculate time difference between start marker and end marker

The following example code shows how to calculate the time difference between the start and end marker.

• Java
• Kotlin

private double calculateLatency(MarkerInfo startMarker, MarkerInfo endMarker, String launchType) {
    if (startMarker == null || endMarker == null) {
        log.error("Missing {} markers - Start: {}, End: {}",
                launchType,
                startMarker != null ? "found" : "missing",
                endMarker != null ? "found" : "missing");
        return -1;
    }

    double latencyMs = (endMarker.timestamp - startMarker.timestamp) * 1000;
    return Double.parseDouble(formatTime(latencyMs / 1000.0));
}

private fun calculateLatency(
    startMarker: MarkerInfo?,
    endMarker: MarkerInfo?,
    launchType: String
): Double {
    if (startMarker == null || endMarker == null) {
        log.error(
            "Missing {} markers - Start: {}, End: {}",
            launchType,
            if (startMarker != null) "found" else "missing",
            if (endMarker != null) "found" else "missing"
        )
        return -1.0
    }

    val latencyMs = (endMarker.timestamp - startMarker.timestamp) * 1000
    return formatTime(latencyMs / 1000.0).toDouble()
}

Latency - time to first frame

The latency KPI, time to first frame (TTFF), measures how long it takes for an app to display its first visual frame after it launches. By taking measurements during cold and warm launches, this KPI aims to replicate realistic user behavior in different scenarios.

Before launching an app programmatically, gather the necessary information such as the app's package name and main activity.

The following example code shows how to measure TTFF latency.

• Java
• Kotlin

private double parseTTFFLaunch(List<String> fileContent, String packageName, String launchType) throws IOException {
    final String START_MARKER = "AMS.startActivityAsUser";
    final String END_MARKER = "eglSwapBuffersWithDamageKHR";
    final Pattern TIMESTAMP_PATTERN =
        Pattern.compile("\\[\\d+]\\s+\\.{3}1\\s+(\\d+\\.\\d+):\\s+tracing_mark_write:\\s+B\\|(\\d+)\\|(.*)");

    String appPid = findAppPid(fileContent, packageName);
    if (appPid == null) {
        log.error("Could not find PID for package: {}", packageName);
        return -1;
    }

    MarkerInfo startMarker = null;
    MarkerInfo endMarker = null;
    int lineCount = 0;

    for (String line : fileContent) {
        lineCount++;
        Matcher matcher = TIMESTAMP_PATTERN.matcher(line);
        if (!matcher.find()) continue;

        double timestamp = Double.parseDouble(matcher.group(1));
        String processId = matcher.group(2);
        String marker = matcher.group(3);

        if (startMarker == null && marker.equals(START_MARKER)) {
            startMarker = new MarkerInfo(timestamp, line, lineCount, processId);
            log.info("Found {} TTFF start marker at line {} | Line: {}", launchType, lineCount, line);
            continue;
        }

        if (endMarker == null && processId.equals(appPid) &&
            (marker.contains("eglSwapBuffers") || marker.contains(END_MARKER))) {
            endMarker = new MarkerInfo(timestamp, line, lineCount, processId);
            log.info("Found {} TTFF end marker at line {} | Line: {}", launchType, lineCount, line);
            break;
        }
    }

    return calculateLatency(startMarker, endMarker, launchType + " TTFF");
}

@Throws(IOException::class)
private fun parseTTFFLaunch(fileContent: List<String>, packageName: String, launchType: String): Double {
    val START_MARKER = "AMS.startActivityAsUser"
    val END_MARKER = "eglSwapBuffersWithDamageKHR"
    val TIMESTAMP_PATTERN = Regex("""\\[\\d+]\\s+\\.{3}1\\s+(\\d+\\.\\d+):\\s+tracing_mark_write:\\s+B\\|(\\d+)\\|(.*)""")

    val appPid = findAppPid(fileContent, packageName)
    if (appPid == null) {
        log.error("Could not find PID for package: {}", packageName)
        return -1.0
    }

    var startMarker: MarkerInfo? = null
    var endMarker: MarkerInfo? = null
    var lineCount = 0

    for (line in fileContent) {
        lineCount++
        val match = TIMESTAMP_PATTERN.matchEntire(line) ?: continue

        val timestamp = match.groupValues[1].toDouble()
        val processId = match.groupValues[2]
        val marker = match.groupValues[3]

        if (startMarker == null && marker == START_MARKER) {
            startMarker = MarkerInfo(timestamp, line, lineCount, processId)
            log.info("Found {} TTFF start marker at line {} | Line: {}", launchType, lineCount, line)
            continue
        }

        if (endMarker == null && processId == appPid &&
            (marker.contains("eglSwapBuffers") || marker.contains(END_MARKER))
        ) {
            endMarker = MarkerInfo(timestamp, line, lineCount, processId)
            log.info("Found {} TTFF end marker at line {} | Line: {}", launchType, lineCount, line)
            break
        }
    }

    return calculateLatency(startMarker, endMarker, "${launchType} TTFF")
}

Measurement instructions for latency tests

1. Download, install, and sign in to the app (if applicable).
2. Run the Atrace clear command.
3. Run the Atrace start command.
4. Launch the app.
5. Wait 30 seconds.
6. Pull the Atrace logs.
7. Perform the appropriate action:
   • For a cold start, force stop the app.
   • For a warm start, send the app to the background.
8. Repeat for the recommended 50 iterations.

The following sections provide more details for the cold start and warm start scenarios.

Scenario: Cold start - time to first frame

A TTFF cold start is the time it takes for an app to launch and display its first frame after the app process has been stopped or the device has been rebooted. When testing a cold start, you start the app after it is force stopped, which simulates a first use or fresh launch scenario. A cold start launch typically takes longer than a warm start launch because the app must reload services.

Test steps

1. Make sure that the app under test isn't running in the background. If so, force stop the app process.

    adb -s %s shell am force-stop %s
   

2. Start Systrace.

    adb shell atrace -o /sdcard/atrace.capture -t 15 -b 32000 -a com.example.page gfx input am view wm
   

3. Launch the app.

    adb -s %s shell monkey --pct-syskeys 0 -p app.package.com.LEANBACK_LAUNCHER -c android.intent.category.%s 1
   

4. Calculate the time taken to draw the first frame of the app.
5. Force close the app.
6. Repeat step 2–5 for 50 iterations.

Example output

Start Marker: Binder:525_1F-3709 ( 525) [002] ...1 2856.568520: tracing_mark_write: B|525|AMS.startActivityAsUser
End Marker: RenderThread-12263 (12229) [002] ...1 2860.298611: tracing_mark_write: B|12229|eglSwapBuffersWithDamageKHR (of the Respective app PID)
Value: Time Difference Between the End Maker and Start Marker

Scenario: Warm start - time to first frame

A TTFF warm start is the time it takes for an app to launch and display its first frame when the app process is already running in the background. The system brings the app from the background to the foreground as part of this launch activity. When testing a warm start, you launch the app after it has been in the background. A warm start launch is typically faster than a cold start launch because the app already has services cached.

Test steps

1. Launch the app under test and press the Home button. Make sure the app under test is in the background.
2. Start Systrace.

    adb shell atrace -o /sdcard/atrace.capture -t 15 -b 32000 -a com.example.page gfx input am view wm
   

3. Launch the app.

    adb -s %s shell monkey --pct-syskeys 0 -p app.package.com.LEANBACK_LAUNCHER -c android.intent.category.%s 1
   

4. Calculate the time taken to draw the first frame of the app.
5. Press the Home button.
6. Repeat steps 2–5 for 50 iterations.

Example output

Start Marker: Binder:525_7-1078 ( 525) [000] ...1 3452.891078: tracing_mark_write: B|525|AMS.startActivityAsUser
End Marker: RenderThread-15305 (15260) [000] ...1 3453.695058: tracing_mark_write: B|15260|eglSwapBuffersWithDamageKHR (of the Respective app PID)
Value: Time Difference Between the End Maker and Start Marker

Ready-to-use - time to full display

The ready-to-use (RTU) KPI, time to full display (TTFD), measures the time an app takes from launch to the ready-to-use state. For example, a ready-to-use state can be when the app's sign-in or home page is usable. RTU metrics can help identify launch performance issues in an app. By taking measurements during cold and warm launches, this KPI aims to replicate realistic user behavior in different scenarios.

The following example code shows how to measure RTU metrics.

• Java
• Kotlin

private double parseRTULaunch(List < String > fileContent, String packageName, String launchType) throws IOException {
    final String START_MARKER = "AMS.startActivityAsUser";
    final String END_MARKER = "eglSwapBuffersWithDamageKHR";
    final Pattern TIMESTAMP_PATTERN =
        Pattern.compile("\\[\\d+]\\s+\\.{3}1\\s+(\\d+\\.\\d+):\\s+tracing_mark_write:\\s+B\\|(\\d+)\\|(.*)");

    String appPid = findAppPid(fileContent, packageName);
    if (appPid == null) {
        log.error("Could not find PID for package: {}", packageName);
        return -1;
    }

    MarkerInfo startMarker = null;
    MarkerInfo lastChoreographer = null;
    MarkerInfo endMarker = null;
    MarkerInfo reportFullyDrawnMarker = null;
    int lineCount = 0;

    // First pass: find start marker, last Choreographer and reportFullyDrawn
    for (String line : fileContent) {
        lineCount++;
        if (line.contains("ExoPlayer")) continue; // Skip ExoPlayer lines

        Matcher matcher = TIMESTAMP_PATTERN.matcher(line);
        if (!matcher.find()) continue;

        double timestamp = Double.parseDouble(matcher.group(1));
        String processId = matcher.group(2);
        String marker = matcher.group(3);

        if (startMarker == null && marker.equals(START_MARKER)) {
            startMarker = new MarkerInfo(timestamp, line, lineCount, processId);
            log.info("Found {} start marker at line {} | Line: {}", launchType, lineCount, line);
            continue;
        }

        if (processId.equals(appPid)) {
            // Track Choreographer frames (excluding ExoPlayer)
            if (marker.contains(GFX_MARKER) && !line.contains("ExoPlayer")) {
                lastChoreographer = new MarkerInfo(timestamp, line, lineCount, processId);
            }
            // Track reportFullyDrawn if present
            else if (marker.contains("Activity.reportFullyDrawn")) {
                reportFullyDrawnMarker = new MarkerInfo(timestamp, line, lineCount, processId);
                log.info("Found reportFullyDrawn at line {} | Line: {}", lineCount, line);
            }
        }
    }
    
    if(reportFullyDrawnMarker == null) {
        // Second pass: find first end marker after last Choreographer
        if (lastChoreographer != null) {
            lineCount = 0;
            for (String line : fileContent) {
                lineCount++;
                if (line.contains("ExoPlayer")) continue; // Skip ExoPlayer lines
    
                Matcher matcher = TIMESTAMP_PATTERN.matcher(line);
                if (!matcher.find()) continue;
    
                double timestamp = Double.parseDouble(matcher.group(1));
                String processId = matcher.group(2);
                String marker = matcher.group(3);
    
                if (processId.equals(appPid) &&
                        timestamp > lastChoreographer.timestamp &&
                        marker.contains(END_MARKER)) {
    
                    endMarker = new MarkerInfo(timestamp, line, lineCount, processId);
                    log.info("Found first {} eglSwapBuffers after Choreographer at line {} | Line: {}",
                            launchType, lineCount, line);
                    break;
                }
            }
        }
    
        // fallback approach  to get last eglSwapBuffers:
        if (endMarker == null) {
            lineCount = 0;
            double timestamp = 0;
            int finalLineCount =0;
            String finalLine = null;
            for (String line : fileContent) {
                lineCount++;
                if (line.contains("ExoPlayer")) continue; // Skip ExoPlayer lines
    
                Matcher matcher = TIMESTAMP_PATTERN.matcher(line);
                if (!matcher.find()) continue;
    
                String processId = matcher.group(2);
    
                String marker = matcher.group(3);
    
                if (processId.equals(appPid) && marker.contains(END_MARKER)) {
                    timestamp = Double.parseDouble(matcher.group(1));
                    finalLine = line;
                    finalLineCount = lineCount;
                }
            }
            if (finalLineCount > 0) {
                endMarker = new MarkerInfo(timestamp, finalLine, finalLineCount, appPid);
                log.info("Fallback Approach : Found last {} eglSwapBuffers  at line {} | Line: {}", launchType,
                         finalLineCount, endMarker.line);
            } else {
                log.info("unable to find egl swapBuffers marker using any approach!!");
            }
        }
    } else {
        endMarker = reportFullyDrawnMarker;
    }

    return calculateLatency(startMarker, endMarker, launchType + " RTU");
}

@Throws(IOException::class)
private fun parseRTULaunch(fileContent: List<String>, packageName: String, launchType: String): Double {
    val START_MARKER = "AMS.startActivityAsUser"
    val END_MARKER = "eglSwapBuffersWithDamageKHR"
    val TIMESTAMP_PATTERN = Pattern.compile("\\[\\d+]\\s+\\.{3}1\\s+(\\d+\\.\\d+):\\s+tracing_mark_write:\\s+B\\|(\\d+)\\|(.*)")

    val appPid = findAppPid(fileContent, packageName)
    if (appPid == null) {
        log.error("Could not find PID for package: {}", packageName)
        return -1.0
    }

    var startMarker: MarkerInfo? = null
    var lastChoreographer: MarkerInfo? = null
    var endMarker: MarkerInfo? = null
    var reportFullyDrawnMarker: MarkerInfo? = null
    var lineCount = 0

    // First pass: find start marker, last Choreographer and reportFullyDrawn
    for (line in fileContent) {
        lineCount++
        if (line.contains("ExoPlayer")) continue

        val matcher = TIMESTAMP_PATTERN.matcher(line)
        if (!matcher.find()) continue

        val timestamp = matcher.group(1).toDouble()
        val processId = matcher.group(2)
        val marker = matcher.group(3)

        if (startMarker == null && marker == START_MARKER) {
            startMarker = MarkerInfo(timestamp, line, lineCount, processId)
            log.info("Found {} start marker at line {} | Line: {}", launchType, lineCount, line)
            continue
        }

        if (processId == appPid) {
            // Track Choreographer frames (excluding ExoPlayer)
            if (marker.contains(GFX_MARKER) && !line.contains("ExoPlayer")) {
                lastChoreographer = MarkerInfo(timestamp, line, lineCount, processId)
            }
            // Track reportFullyDrawn if present
            else if (marker.contains("Activity.reportFullyDrawn")) {
                reportFullyDrawnMarker = MarkerInfo(timestamp, line, lineCount, processId)
                log.info("Found reportFullyDrawn at line {} | Line: {}", lineCount, line)
            }
        }
    }

    if (reportFullyDrawnMarker == null) {
        // Second pass: find first end marker after last Choreographer
        if (lastChoreographer != null) {
            lineCount = 0
            for (line in fileContent) {
                lineCount++
                if (line.contains("ExoPlayer")) continue

                val matcher = TIMESTAMP_PATTERN.matcher(line)
                if (!matcher.find()) continue

                val timestamp = matcher.group(1).toDouble()
                val processId = matcher.group(2)
                val marker = matcher.group(3)

                if (processId == appPid &&
                    timestamp > lastChoreographer.timestamp &&
                    marker.contains(END_MARKER)
                ) {
                    endMarker = MarkerInfo(timestamp, line, lineCount, processId)
                    log.info(
                        "Found first {} eglSwapBuffers after Choreographer at line {} | Line: {}",
                        launchType, lineCount, line
                    )
                    break
                }
            }
        }

        // fallback approach to get last eglSwapBuffers
        if (endMarker == null) {
            lineCount = 0
            var timestamp = 0.0
            var finalLineCount = 0
            var finalLine: String? = null
            
            for (line in fileContent) {
                lineCount++
                if (line.contains("ExoPlayer")) continue

                val matcher = TIMESTAMP_PATTERN.matcher(line)
                if (!matcher.find()) continue

                val processId = matcher.group(2)
                val marker = matcher.group(3)

                if (processId == appPid && marker.contains(END_MARKER)) {
                    timestamp = matcher.group(1).toDouble()
                    finalLine = line
                    finalLineCount = lineCount
                }
            }
            
            if (finalLineCount > 0) {
                endMarker = MarkerInfo(timestamp, finalLine!!, finalLineCount, appPid)
                log.info(
                    "Fallback Approach : Found last {} eglSwapBuffers at line {} | Line: {}",
                    launchType, finalLineCount, endMarker.line
                )
            } else {
                log.info("unable to find egl swapBuffers marker using any approach!!")
            }
        }
    } else {
        endMarker = reportFullyDrawnMarker
    }

    return calculateLatency(startMarker, endMarker, "$launchType RTU")
}

Measurement instructions for ready-to-use tests

1. Download, install, and sign in to the app (if applicable).
2. Run the Atrace clear command.
3. Run the Atrace start command.
4. Launch the app.
5. Wait 30 seconds.
6. Pull the Atrace logs.
7. Perform the appropriate action:
   • For a cold start, force stop the app.
   • For a warm start, send the app to the background.
8. Repeat for the recommended 10 iterations.

The following sections provide more details for the cold start and warm start scenarios.

Scenario: RTU cold start - time to full display

An RTU cold start is the time it takes for an app to launch, become fully drawn, and be ready for user interaction after the app process has been stopped or the device has been rebooted. When testing a cold start, you start the app after it is force stopped, which simulates a first use or fresh launch scenario. A cold start launch typically takes longer than a warm start launch because the app must reload services.

Test steps

1. Make sure that the app under test is not running in the background. If so, force stop the app process.

    adb -s %s shell am force-stop %s
   

2. Start Systrace.

    adb shell atrace -o /sdcard/atrace.capture -t 15 -b 32000 -a com.example.page gfx input am view wm 
   

3. Launch the app.

    adb -s %s shell monkey --pct-syskeys 0 -p app.package.com.LEANBACK_LAUNCHER -c android.intent.category.%s 1
   

4. Calculate the time taken for the app to fully load. This is the app state where users can start interacting with the app.
5. Force close the app.
6. Repeat step 2–5 for 10 iterations.

Example output

Start marker: Line 60 (PID 617):    Binder:617_16-6754  (  617) [001] ...1 2370075.156745: tracing_mark_write: B|617|AMS.startActivityAsUser
End Marker : ReportFullyDrawn marker: Line 2647 (PID 20378):     RenderThread-20425 (20378) [001] ...1 2370076.764284: tracing_mark_write: B|20378|reportFullyDrawn
Value: Time Difference Between the End Maker and Start Marker

Scenario: RTU warm start - time to full display

An RTU warm start is the time it takes for an app to launch, become fully drawn, and be ready for user interaction when the app process is already running in the background. The system brings the app from the background to the foreground as part of this launch activity. When testing a warm start, you launch the app after it has been in the background. A warm start launch is typically faster than a cold start launch because the app already has services cached.

Test steps

1. Launch the app under test and press the Home button. Make sure the app under test is in the background.
2. Start Systrace.

    adb shell atrace -o /sdcard/atrace.capture -t 15 -b 32000 -a com.example.page gfx input am view wm 
   

3. Launch the app.

    adb -s %s shell monkey --pct-syskeys 0 -p app.package.com.LEANBACK_LAUNCHER -c android.intent.category.%s 1
   

4. Calculate the time taken for the app to fully load. This is the app state where users can start interacting with the app.
5. Press the Home button.
6. Repeat step 2–5 for 10 iterations.

Example output

Start marker: Line 60 (PID 617):    Binder:617_16-6754  (  617) [001] ...1 2370075.156745: tracing_mark_write: B|617|AMS.startActivityAsUser
End Marker : ReportFullyDrawn marker: Line 2647 (PID 20378):     RenderThread-20425 (20378) [001] ...1 2370076.764284: tracing_mark_write: B|20378|reportFullyDrawn
Value: Time Difference Between the End Maker and Start Marker

Memory

The memory KPI provides a detailed overview of the app's memory consumption. In addition to memory values, this KPI also measures foreground and background CPU usage, RAM usage, RAM free, and other specifics. By taking measurements when the app is in the foreground and background, this KPI aims to replicate realistic user behavior in different scenarios.

Memory script

The following code uses the Maestro tool to execute a script that plays content for 10 minutes and measures the foreground or background memory usage.

public static boolean executeMaestroScript(String maestroScriptPath, String dsn) {
        try {
            File scriptFile = new File(maestroScriptPath);
            if (!scriptFile.exists()) {
                log.error("Maestro script file does not exist at path: {}", maestroScriptPath);
                return false;
            }

            String command = String.format('maestro --device %s test %s', dsn, maestroScriptPath);
            log.info("Executing Maestro command: {}", command);

            Process process = ShellUtils.executeCommand(command, 600);

            List<String> outputLines = ShellUtils.readExecutionOutput(process, new ArrayList<>());
            log.info("Maestro script execution output:\n{}", String.join("\n", outputLines));

            if (process.waitFor() == 0) {
                log.info("Maestro script executed successfully.");
                return true;
            } else {
                log.error("Maestro script terminated with non-zero exit code.");
                return false;
            }

        } catch (Exception e) {
            log.error("Exception while executing Maestro script: ", e);
            return false;
        }
    }

Scenario: Foreground memory

The foreground memory KPI captures the app's memory consumption while in the foreground. To measure this, you open the app and play a video or game for 10 minutes and then calculate the app's memory consumption.

Test steps

1. Download, install, and sign in to the app (if applicable).
2. Make sure that the app under test is not running in the background. If so, force stop the app process.

    adb -s %s shell am force-stop %s
   

3. Launch the app.

    adb -s %s shell monkey --pct-syskeys 0 -p app.package.com.LEANBACK_LAUNCHER -c android.intent.category.%s 1
   

4. Open the app and play video content for 10 minutes.
5. Wait 120 seconds.
6. Calculate the foreground memory usage of app with the following command.

    adb -s %s shell 'cat /proc/%s/statm'
   

7. Force stop the app.
8. Repeat step 2–7 for 5 iterations.

Scenario: Background memory

The background memory KPI captures the app's memory consumption while in the background. To measure this, you open the app and play a video or game for 10 minutes, background the app, and then calculate the app's memory consumption. While there is no defined threshold for background memory consumption, the amount of memory an app uses in the background is a deciding factor for stopping a background app when the system is running low on memory (RAM). Apps with the highest background memory consumption are the first to be stopped when the system needs more memory for its foreground and other priority tasks.

Test steps

1. Download, install, and sign in to the app (if applicable).
2. Launch the app under test and press the Home button. Make sure the app under test is in the background.
3. Launch the app.

    adb -s %s shell monkey --pct-syskeys 0 -p app.package.com.LEANBACK_LAUNCHER -c android.intent.category.%s 1
   

4. Open the app and play video content for 10 minutes.
5. Press the Home button to send the app to the background.
6. Wait 60 seconds.
7. Calculate the background memory usage of app with the following command.

    adb -s %s shell 'cat /proc/%s/statm'
   

8. Repeat step 2–7 for 5 iterations.

Memory ADB dump log

The proportional set size (PSS) total is the amount of memory consumed by the app on the device. PSS total is used to calculate the memory consumption of an app when it is in the foreground or background.

                 Pss      Pss   Shared  Private   Shared  Private  SwapPss     Heap     Heap     Heap
                Total    Clean    Dirty    Dirty    Clean    Clean    Dirty     Size    Alloc     Free
                                   
 Native Heap   115268        0      384   115020      100      208       22   151552   119143    32408
 Dalvik Heap    15846        0      264    15124      140      676       11    21026    14882     6144
Dalvik Other     8864        0       40     8864        0        0        0                           
       Stack      136        0        4      136        0        0        0                           
      Ashmem      132        0      264        0       12        0        0                           
   Other dev       48        0      156        0        0       48        0                           
    .so mmap    15819     9796      656      596    26112     9796       20                           
   .apk mmap     2103      432        0        0    26868      432        0                           
   .dex mmap    39396    37468        0        4    17052    37468        0                           
   .oat mmap     1592      452        0        0    13724      452        0                           
   .art mmap     2699      304      808     1956    12044      304        0                           
  Other mmap      274        0       12        4      636      244        0                           
   GL mtrack    42152        0        0    42152        0        0        0                           
     Unknown     2695        0       92     2684       60        0        0                           
       TOTAL   247077    48452     2680   186540    96748    49628       53   172578   134025    38552

Write a KPI log file

Use the following example code to write a KPI log file for one iteration.

• Java
• Kotlin

public void log_file_writer(BufferedReader bf_read) {
 File adb_log_file = new File(kpi_log_file_path + "/KPI_Log.txt");
 FileWriter fileWriter = new FileWriter(adb_log_file);
 try {
    String reader = null;
     while ((reader = bf_read.readLine()) != null) {
            fileWriter.write(reader.trim() + "\n");
     }
  }
  catch (Exception e) {
     System.out.println(e);
  } 
  finally {
     fileWriter.flush();
     fileWriter.close();
     bf_read.close();
  }   
}

import java.io.File
import java.io.FileWriter
import java.io.BufferedReader


fun logFileWriter(bfRead: BufferedReader) {
    val adbLogFile = File("$kpi_log_file_path/KPI_Log.txt")
    val fileWriter = FileWriter(adbLogFile)
    try {
        var reader: String?
        while (bfRead.readLine().also { reader = it } != null) {
            fileWriter.write(reader?.trim() + "\n")
        }
    } catch (e: Exception) {
        println(e)
    } finally {
        fileWriter.flush()
        fileWriter.close()
        bfRead.close()
    }
}

Related topics

• To learn more about ADB commands, see Android Debug Bridge (adb) in the Android developer documentation.
• For more tests on Fire TV devices, see Test criteria group 2: App behavior on Fire TV device.

Last updated: Jan 30, 2026