I have a config file that (purposely) contains some undefined aliases:
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
Meanwhile I have a cli that takes --foo and --bar as arguments, where --foo can be specified multiple times. I wind up with a dict looking like:
args = {}
args["foo"] = ["foo_val_1", "foo_val_2", "foo_val_3"]
args["bar"] = "bar_val"
I'd like to dump the args to yaml, but with custom anchors for each key in arg. Something like this:
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
I could write my own code to generate the doc, but I'd prefer somehow tell ruamel.yaml to handle adding the anchors before serializing. The question is, is that possible, and if so, how?
The intention is to then concatenate the two files like this:
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
and extract "base" as the final config after loading the concatenated doc.
I have a config file that (purposely) contains some undefined aliases:
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
Meanwhile I have a cli that takes --foo and --bar as arguments, where --foo can be specified multiple times. I wind up with a dict looking like:
args = {}
args["foo"] = ["foo_val_1", "foo_val_2", "foo_val_3"]
args["bar"] = "bar_val"
I'd like to dump the args to yaml, but with custom anchors for each key in arg. Something like this:
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
I could write my own code to generate the doc, but I'd prefer somehow tell ruamel.yaml to handle adding the anchors before serializing. The question is, is that possible, and if so, how?
The intention is to then concatenate the two files like this:
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
and extract "base" as the final config after loading the concatenated doc.
Share Improve this question edited Mar 30 at 9:31 Marc Swingler asked Mar 30 at 9:23 Marc SwinglerMarc Swingler 3331 silver badge11 bronze badges1 Answer
Reset to default 1Since you cannnot load your config file (as it is invalid YAML), the easiest IMO is to generate a a YAML file with the anchor definitions and append the config file to it.
I recommend to first round-trip the expected output:
import sys
import ruamel.yaml
yaml_str = """\
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
"""
yaml = ruamel.yaml.YAML()
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)
which gives:
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar bar_val
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
And then analysing data, especially for the special types that need to
be created (see also this answer):
print('type', type(data['args']['foo']))
print('anchor', data['args']['foo'].anchor)
print('type', type(data['args']['bar']))
print('anchor', data['args']['foo'].anchor)
which gives:
type <class 'ruamel.yamlments.CommentedSeq'>
anchor Anchor('args_foo')
type <class 'ruamel.yaml.scalarstring.PlainScalarString'>
anchor Anchor('args_foo')
Your args data structure needs some elaboration:
args = {}
args["foo"] = map = ruamel.yamlments.CommentedSeq(["foo_val_1", "foo_val_2", "foo_val_3"])
map.yaml_set_anchor('args_foo', always_dump=True)
args["bar"] = scalar = ruamel.yaml.scalarstring.DoubleQuotedScalarString("bar_val")
scalar.yaml_set_anchor('args_bar', always_dump=True)
yaml.dump(args, sys.stdout)
which gives:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
The always_dump is necessary, as spurious anchors are normally not dumped.
Now you can combine the above:
import sys
import io
import ruamel.yaml
from pathlib import Path
config = Path('config.nonyaml')
config.write_text("""\
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
""")
yaml = ruamel.yaml.YAML()
args = {}
args["foo"] = map = ruamel.yamlments.CommentedSeq(["foo_val_1", "foo_val_2", "foo_val_3"])
map.yaml_set_anchor('args_foo', always_dump=True)
args["bar"] = scalar = ruamel.yaml.scalarstring.DoubleQuotedScalarString("bar_val")
scalar.yaml_set_anchor('args_bar', always_dump=True)
buf = io.BytesIO()
yaml.dump(dict(args=args), buf)
buf.write(config.read_bytes())
print(buf.getvalue().decode('utf-8'))
# check if buf contains loadable YAML
data = yaml.load(buf.getvalue())
print('data:', data)
which gives:
args:
foo: &args_foo
- foo_val_1
- foo_val_2
- foo_val_3
bar: &args_bar "bar_val"
base:
foo: *args_foo
bar: *args_bar
fuz: fuz_val
buz: buz_val
data: {'args': {'foo': ['foo_val_1', 'foo_val_2', 'foo_val_3'], 'bar': 'bar_val'}, 'base': {'foo': ['foo_val_1', 'foo_val_2', 'foo_val_3'], 'bar': 'bar_val', 'fuz': 'fuz_val', 'buz': 'buz_val'}}