I’m trying to run integration tests for my Flutter project (hello_world) on Firebase Test Lab, but they fail during execution. The tests run fine locally but fail in the Test Lab.
Below are my setup details, error logs, and files:
Flutter Version: 3.24.3
Build and Test Execution Command:
flutter build apk --debug
pushd android
./gradlew app:assembleAndroidTest
./gradlew app:assembleDebug -Ptarget="./integration_test/app_test.dart"
popd
gcloud firebase test android run \
--type instrumentation \
--app build/app/outputs/apk/debug/app-debug.apk \
--test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--device model=OP5958L1,version=34,locale=en_US,orientation=portrait \
--timeout 10m
1. Integration Test File (integration_test/app_test.dart)
import 'package:flutter/material.dart';
import 'package:flutter_test/flutter_test.dart';
import 'package:hello_world/main.dart';
import 'package:integration_test/integration_test.dart';
void main() {
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
group('end-to-end test', () {
testWidgets('tap on the floating action button, verify counter',
(tester) async {
// Load app widget.
await tester.pumpWidget(const MyApp());
// Verify the counter starts at 0.
expect(find.text('0'), findsOneWidget);
// Finds the floating action button to tap on.
final fab = find.byKey(const ValueKey('increment'));
// Emulate a tap on the floating action button.
await tester.tap(fab);
// Trigger a frame.
await tester.pumpAndSettle();
// Verify the counter increments by 1.
expect(find.text('1'), findsOneWidget);
});
});
}
2. Main Dart File (lib/main.dart)
import 'package:flutter/material.dart';
void main() => runApp(const MyApp());
class MyApp extends StatelessWidget {
const MyApp({super.key});
@override
Widget build(BuildContext context) {
return const MaterialApp(
title: 'Counter App',
home: MyHomePage(title: 'Counter App Home Page'),
);
}
}
class MyHomePage extends StatefulWidget {
const MyHomePage({super.key, required this.title});
final String title;
@override
State<MyHomePage> createState() => _MyHomePageState();
}
class _MyHomePageState extends State<MyHomePage> {
int _counter = 0;
void _incrementCounter() {
setState(() {
_counter++;
});
}
@override
Widget build(BuildContext context) {
return Scaffold(
appBar: AppBar(
title: Text(widget.title),
),
body: Center(
child: Column(
mainAxisAlignment: MainAxisAlignment.center,
children: <Widget>[
const Text(
'You have pushed the button this many times:',
),
Text(
'$_counter',
style: Theme.of(context).textTheme.headlineMedium,
),
],
),
),
floatingActionButton: FloatingActionButton(
key: const Key('increment'),
onPressed: _incrementCounter,
tooltip: 'Increment',
child: const Icon(Icons.add),
),
);
}
}
3. pubspec.yaml
name: hello_world
description: "A new Flutter project."
publish_to: "none"
version: 1.0.0+1
environment:
sdk: ^3.5.3
dependencies:
flutter:
sdk: flutter
cupertino_icons: ^1.0.8
dev_dependencies:
flutter_test:
sdk: flutter
integration_test:
sdk: flutter
flutter_lints: ^4.0.0
flutter:
uses-material-design: true
4. Android Manifest for Tests (android/app/src/androidTest/AndroidManifest.xml)
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android=";
package="com.example.hello_world.tests">
<application>
<uses-library android:name="android.test.runner" />
</application>
<instrumentation
android:name="androidx.test.runner.AndroidJUnitRunner"
android:targetPackage="com.example.hello_world">
<meta-data
android:name="screenCaptureProcessors"
android:value="com.google.firebase.testlab.screenshot.FirebaseScreenCaptureProcessor" />
</instrumentation>
</manifest>
5. android/build.gradle
plugins {
id "com.android.application"
id "kotlin-android"
id "dev.flutter.flutter-gradle-plugin"
}
android {
namespace = "com.example.hello_world"
compileSdk = flutterpileSdkVersion
ndkVersion = flutter.ndkVersion
compileOptions {
sourceCompatibility = JavaVersion.VERSION_1_8
targetCompatibility = JavaVersion.VERSION_1_8
}
kotlinOptions {
jvmTarget = JavaVersion.VERSION_1_8
}
defaultConfig {
applicationId = "com.example.hello_world"
minSdk = flutter.minSdkVersion
targetSdk = flutter.targetSdkVersion
versionCode = flutter.versionCode
versionName = flutter.versionName
testInstrumentationRunner "androidx.test.runner.AndroidJUnitRunner"
}
buildTypes {
release {
signingConfig = signingConfigs.debug
}
}
}
flutter {
source = "../.."
}
dependencies {
testImplementation 'junit:junit:4.12'
androidTestImplementation 'androidx.test:runner:1.2.0'
androidTestImplementation 'androidx.test.espresso:espresso-core:3.2.0'
}
6. Error Logs
gcloud firebase test android run \
--type instrumentation \
--app build/app/outputs/apk/debug/app-debug.apk \
--test build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk \
--device model=OP5958L1,version=34,locale=en_US,orientation=portrait \
--timeout 10m
Have questions, feedback, or issues? Get support by visiting:
/
Uploading [build/app/outputs/apk/debug/app-debug.apk] to Firebase Test Lab...
Uploading [build/app/outputs/apk/androidTest/debug/app-debug-androidTest.apk] to Firebase Test Lab...
Raw results will be stored in your GCS bucket at [:28:42.953289_EzRK/]
Test [matrix-2tqkwswrvjvc9] has been created in the Google Cloud.
Creating individual test executions...done.
Firebase Test Lab will execute your instrumentation test on 1 device(s). More devices may be added later if flaky test attempts are specified.
Test results will be streamed to [ .xxxxxxxxxxxx/matrices/xxxxxxxxxxxxx ].
22:33:05 Test is Pending
22:33:28 Starting attempt 1.
22:33:28 Setting up Android test.
22:33:28 Test is Running
22:33:35 Starting Android test.
22:37:16 Completed Android test.
22:37:16 Tearing down Android test.
22:37:23 Done. Test time = 1 (secs)
22:37:23 Starting results processing. Attempt: 1
22:37:23 Completed results processing. Time taken = 4 (secs)
22:37:23 Test is Finished
Instrumentation testing complete.
More details are available at [ .xxxxxxxxxx/matrices/xxxxxxxxxxxxxxxxxx ].
┌─────────┬────────────────────────────┬────────────────────┐
│ OUTCOME │ TEST_AXIS_VALUE │ TEST_DETAILS │
├─────────┼────────────────────────────┼────────────────────┤
│ Failed │ OP5958L1-34-en_US-portrait │ Test failed to run │
└─────────┴────────────────────────────┴────────────────────
5. Troubleshooting Steps Taken:
Verified that app-debug.apk and app-debug-androidTest.apk are generated correctly.
Ran tests locally using flutter test integration_test/app_test.dart (works fine).
8. Questions
How do I fix the android:exported issue for Firebase Test Lab?
Do I need any additional configurations in gradle or AndroidManifest.xml?
Has anyone successfully run Flutter integration tests on Firebase Test Lab for Android 12+?