I am using python cdk to launch a Bedrock Agent which uses anthropic.claude-3-5-sonnet-20241022-v2:0 as a foundation model.

        support_agent = bedrock.CfnAgent(self, "HorizonIQSupportAgent",
        agent_name="support-agent",
        agent_resource_role_arn=agent_role.attr_arn,
        auto_prepare=True,
        description="support-agent",
        foundation_model=foundation_model_name,
        idle_session_ttl_in_seconds=600,
        instruction=prompt,
        knowledge_bases=[bedrock.CfnAgent.AgentKnowledgeBaseProperty(
            description=kb_description,
            knowledge_base_id=knowledge_base.attr_knowledge_base_id,
        )]
    )

this works fine and the agent gets deployed. However, I am not able to test it on the console and I get below error.

Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model

I have to manually edit the model and select it again with an inference profile to make it work. How can i do it automatically using python cdk ?

before

after manual editing

I am using python cdk to launch a Bedrock Agent which uses anthropic.claude-3-5-sonnet-20241022-v2:0 as a foundation model.

        support_agent = bedrock.CfnAgent(self, "HorizonIQSupportAgent",
        agent_name="support-agent",
        agent_resource_role_arn=agent_role.attr_arn,
        auto_prepare=True,
        description="support-agent",
        foundation_model=foundation_model_name,
        idle_session_ttl_in_seconds=600,
        instruction=prompt,
        knowledge_bases=[bedrock.CfnAgent.AgentKnowledgeBaseProperty(
            description=kb_description,
            knowledge_base_id=knowledge_base.attr_knowledge_base_id,
        )]
    )

this works fine and the agent gets deployed. However, I am not able to test it on the console and I get below error.

Invocation of model ID anthropic.claude-3-5-sonnet-20241022-v2:0 with on-demand throughput isn’t supported. Retry your request with the ID or ARN of an inference profile that contains this model

I have to manually edit the model and select it again with an inference profile to make it work. How can i do it automatically using python cdk ?

before

after manual editing

• aws-cdk
• amazon-bedrock

• 1 Can you check if you have enabled model access for this specific model, also check if the model is available by cross region inference in the region where you are trying it. If you are trying this in us-east-1 , you would need to enable the model in us-west-2 as the claude sonnet 3.5 V2 is available only in us-west-2. In us-east-1 it is available as cross region inference – Vikram S Commented 2 days ago
• thanks. That pointed me the right direction. – Naxi Commented 2 days ago

1 Answer 1

Apparently, claude sonnet 3.5 V2 is only available in us-west-2. So in order to use in us-east-1, you have to use the inference profile is as the model id.

So in my cdk, I replaced model from - anthropic.claude-3-5-sonnet-20241022-v2:0 to us.anthropic.claude-3-5-sonnet-20241022-v2:0

You can find this in Inference and Assessment > Cross-region inference