I have made a small non-code update to a Python project. Existing unit tests are passing locally, but one of the tests fail when pushing my branch and triggering a CI job.
The actual error indicates that the moto mocking is not working in the CI job. This is the relevant code stub:
import os
import unittest
from unittest.mock import patch, Mock
import boto3
from moto import mock_dynamodb, mock_ssm
os.environ['FLOW_NAME'] = 'test_flow_name'
class TestDynamoDB(unittest.TestCase):
mock_dynamodb2 = mock_dynamodb()
def setUp(self) -> None:
self.mock_dynamodb2.start()
ddb_client = boto3.resource('dynamodb')
table = ddb_client.create_table(
TableName='unit-test-data-integration-metadata',
AttributeDefinitions=[
{
'AttributeName': 'process',
'AttributeType': 'S'
},
{
'AttributeName': 'component',
'AttributeType': 'S'
}
],
KeySchema=[
{
'AttributeName': 'process',
'KeyType': 'HASH'
},
{
'AttributeName': 'component',
'KeyType': 'RANGE'
}
],
ProvisionedThroughput={
'ReadCapacityUnits': 10,
'WriteCapacityUnits': 10
}
)
And error message:
======================================================================
ERROR [0.407s]: test_get_stream_config (tests.test_get_stream_config.TestDynamoDB.test_get_stream_config)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/project/tests/test_get_stream_config.py", line 16, in setUp
table = ddb_client.create_table(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/boto3/resources/factory.py", line 581, in do_action
response = action(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/boto3/resources/action.py", line 88, in __call__
response = getattr(parent.meta.client, operation_name)(*args, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/botocore/client.py", line 570, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/botocore/context.py", line 124, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/botocore/client.py", line 1031, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the CreateTable operation: The security token included in the request is invalid.
----------------------------------------------------------------------
Ran 6 tests in 1.328s
FAILED (errors=1)
My first thought was that a new dependency version had been picked up in the CI that had broken the test but I have validated that all the dependency versions are the same as in my local project. The python versions are both 3.11 but with a differing minor versions.
My other ideas are that it could be something OS-related or an environment variable difference that is causing this but I don't think any of that has changed since the last time it ran.
Any ideas why this might be failing?
I have made a small non-code update to a Python project. Existing unit tests are passing locally, but one of the tests fail when pushing my branch and triggering a CI job.
The actual error indicates that the moto mocking is not working in the CI job. This is the relevant code stub:
import os
import unittest
from unittest.mock import patch, Mock
import boto3
from moto import mock_dynamodb, mock_ssm
os.environ['FLOW_NAME'] = 'test_flow_name'
class TestDynamoDB(unittest.TestCase):
mock_dynamodb2 = mock_dynamodb()
def setUp(self) -> None:
self.mock_dynamodb2.start()
ddb_client = boto3.resource('dynamodb')
table = ddb_client.create_table(
TableName='unit-test-data-integration-metadata',
AttributeDefinitions=[
{
'AttributeName': 'process',
'AttributeType': 'S'
},
{
'AttributeName': 'component',
'AttributeType': 'S'
}
],
KeySchema=[
{
'AttributeName': 'process',
'KeyType': 'HASH'
},
{
'AttributeName': 'component',
'KeyType': 'RANGE'
}
],
ProvisionedThroughput={
'ReadCapacityUnits': 10,
'WriteCapacityUnits': 10
}
)
And error message:
======================================================================
ERROR [0.407s]: test_get_stream_config (tests.test_get_stream_config.TestDynamoDB.test_get_stream_config)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/root/project/tests/test_get_stream_config.py", line 16, in setUp
table = ddb_client.create_table(
^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/boto3/resources/factory.py", line 581, in do_action
response = action(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/boto3/resources/action.py", line 88, in __call__
response = getattr(parent.meta.client, operation_name)(*args, **params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/botocore/client.py", line 570, in _api_call
return self._make_api_call(operation_name, kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/botocore/context.py", line 124, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/root/project/venv/lib/python3.11/site-packages/botocore/client.py", line 1031, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the CreateTable operation: The security token included in the request is invalid.
----------------------------------------------------------------------
Ran 6 tests in 1.328s
FAILED (errors=1)
My first thought was that a new dependency version had been picked up in the CI that had broken the test but I have validated that all the dependency versions are the same as in my local project. The python versions are both 3.11 but with a differing minor versions.
My other ideas are that it could be something OS-related or an environment variable difference that is causing this but I don't think any of that has changed since the last time it ran.
Any ideas why this might be failing?
Share Improve this question asked Mar 18 at 15:10 Connor EllisConnor Ellis 111 bronze badge 2 |1 Answer
Reset to default 1I managed to resolve this - It was caused by a difference in environment variables.
Specifically, The CI job set the AWS_ACCOUNT_ID which I didn't have set locally. That env var was having a some interaction with a change made in botocore 1.37.0 related to DynamoDB URLs.
Setting that env var locally allowed me to reproduce the error locally. Pinning botocore to 1.36.x fixed the issue locally and on the CI server.
... when calling the CreateTable operation: The security token included in the request is invalid.? – jhole Commented Mar 18 at 15:39ddb_client.create_table()is supposed to be mocked byself.mock_dynamodb2.start()using themotolibrary. The fact that it seems to be trying to perform an actual DDB table creation indicates that that mocking is not working in the CI job. – Connor Ellis Commented Mar 18 at 15:45