I am using Firebase Auth, and my users have to be authenticated to use the app.
I would like to have an overview of my (web) app usage from my users. Of course, I could implement tracking from the frontend with a tracker or make calls to cloud functions and log those calls, but I don't want to rely on the client at all. Clients can be hacked, HTTP calls can be blocked, etc.
If the user is utilizing the app and privately accessing data from Firestore that is secured by rules, the backend possesses information about which users have read or written specific data; is there any way to access this data ?
I am currently logging user activity in callable functions; however, users can utilize the app and access data from Firestore without invoking a callable, thus remaining untracked.
exports.addmessage = onCall((request) => {
const uid = request.auth.uid; // store this
});
I am using Firebase Auth, and my users have to be authenticated to use the app.
I would like to have an overview of my (web) app usage from my users. Of course, I could implement tracking from the frontend with a tracker or make calls to cloud functions and log those calls, but I don't want to rely on the client at all. Clients can be hacked, HTTP calls can be blocked, etc.
If the user is utilizing the app and privately accessing data from Firestore that is secured by rules, the backend possesses information about which users have read or written specific data; is there any way to access this data ?
I am currently logging user activity in callable functions; however, users can utilize the app and access data from Firestore without invoking a callable, thus remaining untracked.
exports.addmessage = onCall((request) => {
const uid = request.auth.uid; // store this
});
1 Answer
Reset to default 0If you want to know what data is accessed in your Firestore database, you can enable its data access audit logging. From the documentation:
Google Cloud services generate audit logs that record administrative and access activities within your Google Cloud resources.
Administrative access is always logged, while you can enable data access logging yourself on a project. When enabled, Firestore creates extensive logging information for each data access event in Cloud Logging.
You can then either export the data, or (what I typically do) query it through BigQuery.
I wrote up one example of this in my article Counting document reads per user in Firestore. From that article:
An example of the JSON data from the audit log from that article:
{ "protoPayload": { "@type": "type.googleapis.com/google.cloud.audit.AuditLog", "status": {}, "authenticationInfo": { "principalEmail": "[email protected]", "thirdPartyPrincipal": { "@type": "type.googleapis.com/google.cloud.audit.ThirdPartyJwt", "payload": { "payload": { "sign_in_provider": "anonymous", "identities": {} }, "iss": "https://securetoken.google.com/nanochat-20241022-mw8qu9", "iat": 1731091820, "aud": "nanochat-20241022-mw8qu9", "provider_id": "anonymous", "sub": "mzi2JtP9p1fxavXdVvuk2qMyIWB3", "auth_time": 1731007799, "user_id": "mzi2JtP9p1fxavXdVvuk2qMyIWB3", "exp": 1731095420 }, "header": { "typ": "JWT", "alg": "RS256", "kid": "b8cac95b4a5acde0b96572dee8c8c95eee48cccd" } } }, "requestMetadata": { "callerIp": "157.131.245.100", "callerSuppliedUserAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/130.0.0.0 Safari/537.36,gzip(gfe),gzip(gfe)", "requestAttributes": { "time": "2024-11-08T19:14:57.226501Z", "auth": {} }, "destinationAttributes": {} }, "serviceName": "firestore.googleapis.com", "methodName": "google.firestore.v1.Firestore.Listen", "authorizationInfo": [ { "resource": "projects/nanochat-20241022-mw8qu9/databases/(default)", "permission": "datastore.entities.get", "granted": true, "resourceAttributes": { "service": "firestore.googleapis.com", "name": "projects/nanochat-20241022-mw8qu9/databases/(default)", "type": "firestore.googleapis.com/Database" }, "permissionType": "DATA_READ" }, { "resource": "projects/nanochat-20241022-mw8qu9/databases/(default)", "permission": "datastore.entities.list", "granted": true, "resourceAttributes": { "service": "firestore.googleapis.com", "name": "projects/nanochat-20241022-mw8qu9/databases/(default)", "type": "firestore.googleapis.com/Database" }, "permissionType": "DATA_READ" } ], "resourceName": "projects/nanochat-20241022-mw8qu9/databases/(default)", "numResponseItems": "10", "request": { "@type": "type.googleapis.com/google.firestore.v1.ListenRequest", "addTarget": { "targetId": 102, "query": { "parent": "projects/nanochat-20241022-mw8qu9/databases/(default)/documents", "structuredQuery": { "limit": 10, "from": [ { "collectionId": "chat" } ], "orderBy": [ { "direction": "DESCENDING", "field": { "fieldPath": "timestamp" } }, { "direction": "DESCENDING", "field": { "fieldPath": "__name__" } } ] } } } }, "metadata": { "@type": "type.googleapis.com/google.cloud.audit.DatastoreServiceData" } }, "insertId": "-itd58hf2gzijg", "resource": { "type": "audited_resource", "labels": { "project_id": "nanochat-20241022-mw8qu9", "service": "firestore.googleapis.com", "method": "google.firestore.v1.Firestore.Listen" } }, "timestamp": "2024-11-08T19:14:57.217715Z", "severity": "INFO", "logName": "projects/nanochat-20241022-mw8qu9/logs/cloudaudit.googleapis.com%2Fdata_access", "operation": { "id": "3c9374e7-6d46-4619-ad12-a0bfbfac16b1", "producer": "firestore.googleapis.com", "last": true }, "receiveTimestamp": "2024-11-08T19:14:57.639965123Z" }The chart I generate there of reads-per-user-per-day:
And the BigQuery SQL I used to generate this:
SELECT JSON_VALUE(proto_payload.audit_log.authentication_info.third_party_principal.payload.user_id) as uid, TIMESTAMP_TRUNC(timestamp, DAY) as day, SUM(proto_payload.audit_log.num_response_items) as read_count FROM `nanochat-20241022-mw8qu9.global._Default._AllLogs` WHERE proto_payload.audit_log.authorization_info[0].permission_type IN ('DATA_READ', 'DATA_WRITE') AND proto_payload.audit_log.method_name LIKE 'google.firestore.v1.Firestore%' GROUP BY JSON_VALUE(proto_payload.audit_log.authentication_info.third_party_principal.payload.user_id), TIMESTAMP_TRUNC(timestamp, DAY)