Why does bash make an additional and unnecessary substitution & for the word that was replaced when replacing a substring?

Test replace string in bash:

test="test string MARKER end"; echo "${test/MARKER/new value '&' why & in new value was replaced}"

I've got output:

test string new value & why MARKER in new value was replaced end

why is there a MARKER here again?

Why does bash make an additional and unnecessary substitution & for the word that was replaced when replacing a substring?

Test replace string in bash:

test="test string MARKER end"; echo "${test/MARKER/new value '&' why & in new value was replaced}"

I've got output:

test string new value & why MARKER in new value was replaced end

why is there a MARKER here again?

• bash
• backreference
• parameter-expansion

• Do you want to know how & works here or why was it introduced to work like that? – Arkadiusz Drabczyk Commented Feb 16 at 21:02
• 1 fwiw, I'm running bash v.5.1.16 and the output from your code shows 2 &'s and no MARKER; what are you expecting the output to look like? – markp-fuso Commented Feb 16 at 21:08
• 2 @markp-fuso: this feature was introduced in 5.2 – Arkadiusz Drabczyk Commented Feb 16 at 21:14
• 3 FYI replacing & with the string that matched the pattern is extremely common behavior that exists in [almost?] all pattern-matching tools. Try echo 'foo MARKER bar' | sed 's/MARKER/<&>/' or echo 'foo MARKER bar' | awk '{sub(/MARKER/,"<&>")}1' for example. It'd be more surprising for bash NOT to behave the same way. – Ed Morton Commented Feb 17 at 1:04
• 1 The issue was with patsub_replacement option shopt, which was active by default in my shell bash shopt -u patsub_replacement – dobordx Commented Feb 17 at 11:09

1 Answer 1

"unnecessary" is your first feeling; but in fact it can be quite useful when you want to include the matched string in the output, without certainty about the matched string.

In your example you don't have uncertainty (you know you are looking for the fixed string "MARKER"),
but let's take another example where the match is polymorphic:

prices="12.34 € + $ 9.45"
echo ${prices//[€$]/(&)} # "Either an euro, or a dollar"
# prints: 12.34 (€) + ($) 9.45

Here I knew I wanted the replaced string to be part of the replacement (between two parenthesis), but I did't know in advance if it would be an euro or a dollar.

With the & I make sure the replacement reproduces the matched string.