We have a Python process which reads a JSON file and pushes the data held therein to a BigQuery table. The files are small and normally would process very quickly. All of a sudden the process has really slowed down. The python resides in a VM which is within Big Query. What we have noticed in the BigQuery logs is a lot of errors similar to the following:

{
  "insertId": "s=3bc18ce95c384588adb5aea97d1ab8fc;i=c645e;b=8f0641e52f4d431db01b9ea986ded58e;m=4ac5bf55c5b;t=62fc1655c06a6;x=91a9a97343928f96",
  "jsonPayload": {
    "message": "2025-03-07T14:25:35.9871Z OSConfigAgent Error main.go:88: code: \"Unauthenticated\", message: \"Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See .\", details: []"
  },

We're not sure if this is related.

I should clarify that the script runs and does eventually push the data but it is very slow do to so, it doesn't actually fail.

We have tried restarting the VM and the process itself. What can we try next?

We have a Python process which reads a JSON file and pushes the data held therein to a BigQuery table. The files are small and normally would process very quickly. All of a sudden the process has really slowed down. The python resides in a VM which is within Big Query. What we have noticed in the BigQuery logs is a lot of errors similar to the following:

{
  "insertId": "s=3bc18ce95c384588adb5aea97d1ab8fc;i=c645e;b=8f0641e52f4d431db01b9ea986ded58e;m=4ac5bf55c5b;t=62fc1655c06a6;x=91a9a97343928f96",
  "jsonPayload": {
    "message": "2025-03-07T14:25:35.9871Z OSConfigAgent Error main.go:88: code: \"Unauthenticated\", message: \"Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google/identity/sign-in/web/devconsole-project.\", details: []"
  },

We're not sure if this is related.

I should clarify that the script runs and does eventually push the data but it is very slow do to so, it doesn't actually fail.

We have tried restarting the VM and the process itself. What can we try next?

• python

1 Answer 1

It looks like the issue might be related to authentication. The logs show an 'Unauthenticated' error, which could be slowing things down. Try checking authentication with gcloud auth and make sure the VM's service account has the right permissions. Also, have you tested the network to see if there are any delays?