I'll use a synthetic example - the real one is more complex. Consider trying to validate a freeform address. You want to be sure the user provided all the information needed and if not prompt them to do so. Here's a system prompt for same:

@SystemMessage("""
                    Your role is to extract american postal address information ONLY from the user input message.
                    If a piece of the required address information is not provided you MUST NOT infer it.
                    Instead inform the user of the piece[s] of information that is/are missing.
                    """)

Now consider input such as

The address is 123 East Street, Winchester, VA

that you want to be mapped into a data structure similar to this

@Description("Client Address")
public static final class Address {
    @Description("apartment number")
    String apartmentNumber;
    @Description("street number")
    Integer streetNumber;
    @Description("street name")
    String street;
    @Description("city")
    String city;
    @Description("state")
    String state;
    @Description("zip code")
    String zip;

Ok that's probably more detail than you need. On to the actual question. I have a tool to validate the mapped object. By time it gets to the tool [one of] the four possible zip codes for the given address - Winchester, VA has been mapped into it, and the apartment number has been set to be the same as the street number. Which means I can't stop and ask that the zip be furnished.

If I provide less information - a partial address such as 100 Eat St for example, I get an address in Seattle, WA with an appropriate zip.

I've tried looking at the manual, the examples and googling, but can't seem to find a way to stop unwanted extraneous information being mapped into my object. I've tried numerous combinations of system message to attempt to guide the llm to not inject spurious information but with no success.

What am I missing?

I'll use a synthetic example - the real one is more complex. Consider trying to validate a freeform address. You want to be sure the user provided all the information needed and if not prompt them to do so. Here's a system prompt for same:

@SystemMessage("""
                    Your role is to extract american postal address information ONLY from the user input message.
                    If a piece of the required address information is not provided you MUST NOT infer it.
                    Instead inform the user of the piece[s] of information that is/are missing.
                    """)

Now consider input such as

The address is 123 East Street, Winchester, VA

that you want to be mapped into a data structure similar to this

@Description("Client Address")
public static final class Address {
    @Description("apartment number")
    String apartmentNumber;
    @Description("street number")
    Integer streetNumber;
    @Description("street name")
    String street;
    @Description("city")
    String city;
    @Description("state")
    String state;
    @Description("zip code")
    String zip;

Ok that's probably more detail than you need. On to the actual question. I have a tool to validate the mapped object. By time it gets to the tool [one of] the four possible zip codes for the given address - Winchester, VA has been mapped into it, and the apartment number has been set to be the same as the street number. Which means I can't stop and ask that the zip be furnished.

If I provide less information - a partial address such as 100 Eat St for example, I get an address in Seattle, WA with an appropriate zip.

I've tried looking at the manual, the examples and googling, but can't seem to find a way to stop unwanted extraneous information being mapped into my object. I've tried numerous combinations of system message to attempt to guide the llm to not inject spurious information but with no success.

What am I missing?

• langchain4j

1 Answer 1

As a temporary (ugly, but might work) solution you could add boolean containsZip, boolean containsApartmentNumber, etc to the Address and ignore actual value if the flag is false